2024-04-20T10:17:39.051 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-20T10:17:39.051 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair} --archive /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463 --owner scheduled_rishabh@teuthology --verbose -- /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/orig.config.yaml 2024-04-20T10:17:39.096 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:17:39.182 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463 branch: wip-rishabh-testing-20240416.193735 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair} email: null first_in_suite: false job_id: '7664463' kernel: flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240416.193735 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - force file system read-only - bad backtrace - MDS in read-only mode - \(MDS_READ_ONLY\) sha1: a8c84f5997950fd7d008204839253291d52e6eda ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a8c84f5997950fd7d008204839253291d52e6eda selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 subvolume_version: 2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240416.193735 sha1: a8c84f5997950fd7d008204839253291d52e6eda owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7954 sha1: a8c84f5997950fd7d008204839253291d52e6eda sleep_before_teardown: 0 subset: 364/512 suite: fs suite_branch: wip-rishabh-testing-20240416.193735 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: a8c84f5997950fd7d008204839253291d52e6eda syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi047.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGtvGgOa4z63kz9+Cg0/az2X1LuKHDEPB4/vjsPPwWlwfYctZSn3+flXdwjk5TiwAa+oHTBslvdwFxZ0k70J8e0= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPz7C7ZIChXoqlSACubGmD165Bl6znw68xZaLIvEd6qgX1SR1vQe5NsrzYeEQUvJyakvWvBUIyCLRT803TXwkB4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - sequential: - mgrmodules - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_auto_repair teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-20_09:50:15 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-20T10:17:39.183 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa; will attempt to use it 2024-04-20T10:17:39.183 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa/tasks 2024-04-20T10:17:39.183 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-20T10:17:39.185 INFO:teuthology.task.internal:Checking packages... 2024-04-20T10:17:39.208 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a8c84f5997950fd7d008204839253291d52e6eda' 2024-04-20T10:17:39.208 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-20T10:17:39.208 INFO:teuthology.packaging:ref: None 2024-04-20T10:17:39.209 INFO:teuthology.packaging:tag: None 2024-04-20T10:17:39.209 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240416.193735 2024-04-20T10:17:39.209 INFO:teuthology.packaging:sha1: a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:17:39.209 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240416.193735 2024-04-20T10:17:39.427 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3133.ga8c84f59 2024-04-20T10:17:39.429 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-20T10:17:39.435 INFO:teuthology.task.internal:no buildpackages task found 2024-04-20T10:17:39.436 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-20T10:17:39.473 INFO:teuthology.task.internal:Saving configuration 2024-04-20T10:17:39.490 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-20T10:17:39.496 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-20T10:17:39.522 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi047.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:12:54.427664', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGtvGgOa4z63kz9+Cg0/az2X1LuKHDEPB4/vjsPPwWlwfYctZSn3+flXdwjk5TiwAa+oHTBslvdwFxZ0k70J8e0='} 2024-04-20T10:17:39.542 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-20 10:12:54.429179', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPz7C7ZIChXoqlSACubGmD165Bl6znw68xZaLIvEd6qgX1SR1vQe5NsrzYeEQUvJyakvWvBUIyCLRT803TXwkB4='} 2024-04-20T10:17:39.542 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-20T10:17:39.548 INFO:teuthology.task.internal:roles: ubuntu@smithi047.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-20T10:17:39.548 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-20T10:17:39.549 INFO:teuthology.run_tasks:Running task console_log... 2024-04-20T10:17:39.618 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ff62e396160>, signals=[15]) 2024-04-20T10:17:39.618 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-20T10:17:39.625 INFO:teuthology.task.internal:Opening connections... 2024-04-20T10:17:39.625 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:17:39.626 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:17:39.705 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:17:39.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:17:39.778 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-20T10:17:39.785 DEBUG:teuthology.orchestra.run.smithi047:> uname -m 2024-04-20T10:17:39.803 INFO:teuthology.orchestra.run.smithi047.stdout:x86_64 2024-04-20T10:17:39.804 DEBUG:teuthology.orchestra.run.smithi047:> cat /etc/os-release 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:NAME="CentOS Stream" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION="9" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:ID="centos" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:VERSION_ID="9" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:ANSI_COLOR="0;31" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:17:39.861 INFO:teuthology.orchestra.run.smithi047.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:17:39.862 INFO:teuthology.orchestra.run.smithi047.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:17:39.862 INFO:teuthology.orchestra.run.smithi047.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:17:39.862 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:17:39.862 INFO:teuthology.orchestra.run.smithi047.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:17:39.862 INFO:teuthology.lock.ops:Updating smithi047.front.sepia.ceph.com on lock server 2024-04-20T10:17:39.897 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-04-20T10:17:39.915 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-04-20T10:17:39.915 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-20T10:17:39.972 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-04-20T10:17:39.973 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-20T10:17:39.973 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-20T10:17:39.973 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-20T10:17:39.973 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-04-20T10:17:39.997 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-20T10:17:40.006 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-20T10:17:40.012 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-20T10:17:40.012 DEBUG:teuthology.orchestra.run.smithi047:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:17:40.015 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-04-20T10:17:40.031 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-20T10:17:40.037 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-20T10:17:40.037 DEBUG:teuthology.orchestra.run.smithi047:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:17:40.073 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-04-20T10:17:40.184 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-20T10:17:40.243 INFO:teuthology.run_tasks:Running task kernel... 2024-04-20T10:17:40.257 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-04-20T10:17:40.258 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, timeout 300 2024-04-20T10:17:40.258 DEBUG:teuthology.orchestra.run.smithi047:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:17:40.258 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-20T10:17:40.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:17:40.275 DEBUG:teuthology.task.kernel:sha1 for mon.b is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:17:40.275 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:17:40.275 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-04-20T10:17:40.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:17:40.276 DEBUG:teuthology.task.kernel:sha1 for mon.a is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:17:40.276 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-20T10:17:40.276 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-20T10:17:40.331 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:17:40.331 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:17:40.331 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:17:40.331 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:17:40.335 INFO:teuthology.orchestra.run.smithi047.stdout:5.14.0-437.el9.x86_64 2024-04-20T10:17:40.336 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:17:40.336 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-20T10:17:40.336 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:17:40.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum upgrade -y linux-firmware 2024-04-20T10:17:40.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum upgrade -y linux-firmware 2024-04-20T10:17:45.296 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 2.5 MB/s | 8.0 MB 00:03 2024-04-20T10:17:47.155 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 1.6 MB/s | 8.0 MB 00:05 2024-04-20T10:17:47.752 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-20T10:17:49.106 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-20T10:17:51.897 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-04-20T10:17:53.298 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 63 kB/s | 16 kB 00:00 2024-04-20T10:17:55.477 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-04-20T10:18:00.541 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-20T10:18:00.656 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 812 kB/s | 6.2 MB 00:07 2024-04-20T10:18:02.323 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-20T10:18:02.536 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-04-20T10:18:02.537 INFO:teuthology.orchestra.run.smithi175.stdout:Nothing to do. 2024-04-20T10:18:02.537 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-04-20T10:18:02.639 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T10:18:02.640 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:18:02.859 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T10:18:02.859 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T10:18:02.862 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi175.front.sepia.ceph.com to complete... 2024-04-20T10:18:04.049 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-20T10:18:09.224 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-04-20T10:18:11.295 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T10:18:11.296 INFO:teuthology.orchestra.run.smithi047.stdout:Nothing to do. 2024-04-20T10:18:11.296 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T10:18:11.426 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T10:18:11.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-20T10:18:11.581 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-20T10:18:11.581 DEBUG:teuthology.orchestra.run.smithi047:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-20T10:18:11.589 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi047.front.sepia.ceph.com to complete... 2024-04-20T10:18:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20 10:18:22 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T10:18:22.200 INFO:teuthology.orchestra.run.smithi175.stderr:FINISHED --2024-04-20 10:18:22-- 2024-04-20T10:18:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:Total wall clock time: 19s 2024-04-20T10:18:22.201 INFO:teuthology.orchestra.run.smithi175.stderr:Downloaded: 1 files, 341M in 19s (17.9 MB/s) 2024-04-20T10:18:22.204 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.b... 2024-04-20T10:18:22.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T10:18:24.304 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying... ######################################## 2024-04-20T10:18:24.445 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing... ######################################## 2024-04-20T10:18:24.559 INFO:teuthology.orchestra.run.smithi175.stdout:Updating / installing... 2024-04-20T10:18:32.566 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T10:18:50.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20 10:18:50 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-20T10:18:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:FINISHED --2024-04-20 10:18:50-- 2024-04-20T10:18:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:Total wall clock time: 39s 2024-04-20T10:18:50.862 INFO:teuthology.orchestra.run.smithi047.stderr:Downloaded: 1 files, 341M in 39s (8.80 MB/s) 2024-04-20T10:18:50.865 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on mon.a... 2024-04-20T10:18:50.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-20T10:18:53.081 INFO:teuthology.orchestra.run.smithi047.stdout:Verifying... ######################################## 2024-04-20T10:18:53.206 INFO:teuthology.orchestra.run.smithi047.stdout:Preparing... ######################################## 2024-04-20T10:18:53.326 INFO:teuthology.orchestra.run.smithi047.stdout:Updating / installing... 2024-04-20T10:19:01.696 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-20T10:19:11.444 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T10:19:11.444 DEBUG:teuthology.orchestra.run.smithi175:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T10:19:11.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T10:19:11.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T10:19:11.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T10:19:11.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T10:19:11.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T10:19:11.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T10:19:11.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T10:19:11.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T10:19:11.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T10:19:11.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T10:19:11.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T10:19:11.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T10:19:11.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T10:19:11.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T10:19:11.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T10:19:11.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T10:19:11.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T10:19:11.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T10:19:11.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T10:19:11.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T10:19:11.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T10:19:11.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T10:19:11.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T10:19:11.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T10:19:11.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T10:19:11.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T10:19:11.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T10:19:11.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T10:19:11.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T10:19:11.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T10:19:11.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T10:19:11.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T10:19:11.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T10:19:11.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T10:19:11.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T10:19:11.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T10:19:11.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T10:19:11.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T10:19:11.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T10:19:11.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T10:19:11.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T10:19:11.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T10:19:11.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T10:19:11.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T10:19:11.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T10:19:11.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T10:19:11.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T10:19:11.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T10:19:11.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T10:19:11.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T10:19:11.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T10:19:11.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T10:19:11.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T10:19:11.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T10:19:11.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T10:19:11.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T10:19:11.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T10:19:11.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T10:19:11.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T10:19:11.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T10:19:11.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T10:19:11.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T10:19:11.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T10:19:11.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T10:19:11.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T10:19:11.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T10:19:11.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T10:19:11.582 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T10:19:11.583 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T10:19:11.584 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T10:19:11.585 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T10:19:11.586 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T10:19:11.587 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T10:19:11.588 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T10:19:11.589 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T10:19:11.590 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T10:19:11.591 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T10:19:11.592 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T10:19:11.593 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T10:19:11.594 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T10:19:11.595 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T10:19:11.596 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T10:19:11.597 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T10:19:11.598 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T10:19:11.599 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T10:19:11.600 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T10:19:11.601 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T10:19:11.602 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T10:19:11.603 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T10:19:11.604 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T10:19:11.605 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T10:19:11.606 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T10:19:11.607 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T10:19:11.608 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T10:19:11.609 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T10:19:11.610 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T10:19:11.611 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T10:19:11.612 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T10:19:11.613 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T10:19:11.614 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T10:19:11.615 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T10:19:11.616 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T10:19:11.617 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T10:19:11.618 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T10:19:11.619 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T10:19:11.620 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T10:19:11.621 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T10:19:11.622 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T10:19:11.623 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T10:19:11.624 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T10:19:11.625 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T10:19:11.626 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T10:19:11.627 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T10:19:11.628 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T10:19:11.629 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T10:19:11.630 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T10:19:11.631 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T10:19:11.632 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T10:19:11.633 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T10:19:11.634 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T10:19:11.635 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T10:19:11.636 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T10:19:11.637 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T10:19:11.638 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T10:19:11.639 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T10:19:11.640 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T10:19:11.641 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T10:19:11.642 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T10:19:11.643 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T10:19:11.644 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T10:19:11.645 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T10:19:11.646 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T10:19:11.647 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T10:19:11.648 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T10:19:11.649 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T10:19:11.650 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T10:19:11.651 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T10:19:11.652 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T10:19:11.653 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T10:19:11.654 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T10:19:11.655 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T10:19:11.656 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T10:19:11.657 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T10:19:11.658 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T10:19:11.659 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T10:19:11.660 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T10:19:11.661 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T10:19:11.662 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T10:19:11.663 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T10:19:11.664 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T10:19:11.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T10:19:11.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T10:19:11.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T10:19:11.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T10:19:11.665 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:19:11.669 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:11.669 DEBUG:teuthology.orchestra.run.smithi175:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T10:19:11.692 INFO:teuthology.orchestra.run.smithi175.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T10:19:11.692 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:for file in vmlinuz System.map config; do 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: fi 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:done 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T10:19:11.693 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:11.694 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T10:19:11.694 INFO:teuthology.orchestra.run.smithi175.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T10:19:11.694 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:19:11.694 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:11.694 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-04-20T10:19:11.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-04-20T10:19:11.767 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 68.el9 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8264254 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T10:19:11.768 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:19:11.769 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T10:19:11.770 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T10:19:11.770 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:11.773 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T10:19:12.338 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-04-20T10:19:13.746 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T10:19:13.774 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-04-20T10:19:13.777 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-04-20T10:19:13.792 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.lX6XawIUYa 2024-04-20T10:19:13.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lX6XawIUYa 2024-04-20T10:19:13.867 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.lX6XawIUYa 2024-04-20T10:19:14.007 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.lX6XawIUYa is 6KB 2024-04-20T10:19:14.056 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.lX6XawIUYa 2024-04-20T10:19:14.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T10:19:14.131 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T10:19:14.132 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T10:19:14.132 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T10:19:14.132 INFO:teuthology.orchestra.run.smithi175.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T10:19:14.133 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:14.285 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-04-20T10:19:39.909 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi047.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-20T10:19:39.909 DEBUG:teuthology.orchestra.run.smithi047:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-20T10:19:39.996 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-20T10:19:39.997 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-20T10:19:39.998 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-20T10:19:39.999 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-20T10:19:40.000 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-20T10:19:40.001 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-20T10:19:40.002 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-20T10:19:40.003 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-20T10:19:40.004 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-20T10:19:40.005 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-20T10:19:40.006 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-20T10:19:40.007 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-20T10:19:40.008 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-20T10:19:40.009 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-20T10:19:40.010 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-20T10:19:40.011 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-20T10:19:40.012 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-20T10:19:40.013 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-20T10:19:40.014 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-20T10:19:40.015 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-20T10:19:40.016 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-20T10:19:40.017 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-20T10:19:40.018 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-20T10:19:40.019 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-20T10:19:40.020 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-20T10:19:40.021 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-20T10:19:40.022 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-20T10:19:40.023 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-20T10:19:40.024 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-20T10:19:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-20T10:19:40.026 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-20T10:19:40.027 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-20T10:19:40.028 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-20T10:19:40.029 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-20T10:19:40.030 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-20T10:19:40.031 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-20T10:19:40.032 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-20T10:19:40.033 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-20T10:19:40.034 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-20T10:19:40.035 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-20T10:19:40.036 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-20T10:19:40.037 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-20T10:19:40.038 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-20T10:19:40.039 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-20T10:19:40.040 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-20T10:19:40.041 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-20T10:19:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-20T10:19:40.043 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-20T10:19:40.044 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-20T10:19:40.045 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-20T10:19:40.046 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-20T10:19:40.047 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-20T10:19:40.048 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-20T10:19:40.049 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-20T10:19:40.050 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-20T10:19:40.051 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-20T10:19:40.051 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-20T10:19:40.051 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-20T10:19:40.051 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-20T10:19:40.053 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-20T10:19:40.053 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-20T10:19:40.053 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-20T10:19:40.053 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-20T10:19:40.054 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-20T10:19:40.055 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-20T10:19:40.056 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-20T10:19:40.057 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-20T10:19:40.058 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-20T10:19:40.059 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-20T10:19:40.060 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-20T10:19:40.061 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-20T10:19:40.062 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-20T10:19:40.063 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-20T10:19:40.064 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-20T10:19:40.065 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-20T10:19:40.066 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-20T10:19:40.067 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-20T10:19:40.068 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-20T10:19:40.069 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-20T10:19:40.070 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-20T10:19:40.071 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-20T10:19:40.072 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-20T10:19:40.073 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-20T10:19:40.074 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-20T10:19:40.075 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-20T10:19:40.076 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-20T10:19:40.077 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-20T10:19:40.078 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-20T10:19:40.079 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-20T10:19:40.080 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-20T10:19:40.081 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-20T10:19:40.082 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-20T10:19:40.083 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-20T10:19:40.084 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-20T10:19:40.085 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-20T10:19:40.086 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-20T10:19:40.087 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-20T10:19:40.088 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-20T10:19:40.089 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-20T10:19:40.090 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-20T10:19:40.091 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-20T10:19:40.092 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-20T10:19:40.093 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-20T10:19:40.094 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-20T10:19:40.095 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-20T10:19:40.096 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-20T10:19:40.097 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-20T10:19:40.098 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-20T10:19:40.099 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-20T10:19:40.100 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-20T10:19:40.101 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-20T10:19:40.102 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-20T10:19:40.103 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-20T10:19:40.104 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-20T10:19:40.105 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-20T10:19:40.106 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-20T10:19:40.107 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-20T10:19:40.108 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-20T10:19:40.109 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-20T10:19:40.110 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-20T10:19:40.111 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-20T10:19:40.112 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-20T10:19:40.113 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-20T10:19:40.114 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-20T10:19:40.115 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-20T10:19:40.116 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-20T10:19:40.117 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-20T10:19:40.118 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-20T10:19:40.119 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-20T10:19:40.120 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-20T10:19:40.121 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-20T10:19:40.122 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-20T10:19:40.123 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-20T10:19:40.124 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-20T10:19:40.125 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-20T10:19:40.126 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-20T10:19:40.127 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-20T10:19:40.128 INFO:teuthology.orchestra.run.smithi047.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:19:40.131 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:40.131 DEBUG:teuthology.orchestra.run.smithi047:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-20T10:19:40.155 INFO:teuthology.orchestra.run.smithi047.stdout:postinstall scriptlet (using /bin/sh): 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout:for file in vmlinuz System.map config; do 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: fi 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout:done 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-20T10:19:40.156 INFO:teuthology.orchestra.run.smithi047.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-20T10:19:40.157 INFO:teuthology.orchestra.run.smithi047.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:40.157 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-20T10:19:40.157 INFO:teuthology.orchestra.run.smithi047.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-20T10:19:40.157 INFO:teuthology.orchestra.run.smithi047.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-20T10:19:40.157 INFO:teuthology.orchestra.run.smithi047.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:40.157 INFO:teuthology.orchestra.run.smithi047.stdout:fi 2024-04-20T10:19:40.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo rpm -qi grub2-tools 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Name : grub2-tools 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Epoch : 1 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Version : 2.06 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Release : 68.el9 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Architecture: x86_64 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Group : Unspecified 2024-04-20T10:19:40.232 INFO:teuthology.orchestra.run.smithi047.stdout:Size : 8264254 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:License : GPLv3+ 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Packager : builder@centos.org 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Vendor : CentOS 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Summary : Support tools for GRUB. 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:Description : 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-20T10:19:40.233 INFO:teuthology.orchestra.run.smithi047.stdout:hardware devices. 2024-04-20T10:19:40.234 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:19:40.234 INFO:teuthology.orchestra.run.smithi047.stdout:This subpackage provides tools for support of all platforms. 2024-04-20T10:19:40.234 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-20T10:19:40.235 INFO:teuthology.task.kernel:Updating grub on smithi047 to boot 6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:40.235 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-20T10:19:40.960 INFO:teuthology.orchestra.run.smithi047.stderr:Generating grub configuration file ... 2024-04-20T10:19:42.397 INFO:teuthology.orchestra.run.smithi047.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-20T10:19:42.423 INFO:teuthology.orchestra.run.smithi047.stderr:done 2024-04-20T10:19:42.425 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-20T10:19:42.480 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.DaGLrikvBU 2024-04-20T10:19:42.480 DEBUG:teuthology.orchestra.run.smithi047:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DaGLrikvBU 2024-04-20T10:19:42.546 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.DaGLrikvBU 2024-04-20T10:19:42.685 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.DaGLrikvBU is 6KB 2024-04-20T10:19:42.734 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.DaGLrikvBU 2024-04-20T10:19:42.749 DEBUG:teuthology.orchestra.run.smithi047:> sudo /bin/ls /boot/loader/entries || true 2024-04-20T10:19:42.816 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-20T10:19:42.817 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-20T10:19:42.817 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-20T10:19:42.817 INFO:teuthology.orchestra.run.smithi047.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-20T10:19:42.818 DEBUG:teuthology.orchestra.run.smithi047:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-20T10:19:42.983 DEBUG:teuthology.orchestra.run.smithi047:> sudo shutdown -r now 2024-04-20T10:19:44.289 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-20T10:19:44.289 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T10:19:44.290 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:20:02.718 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-04-20T10:20:11.727 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T10:20:11.727 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:20:12.986 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-20T10:20:12.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T10:20:12.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:20:14.782 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.175 2024-04-20T10:20:26.794 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T10:20:26.795 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:20:31.390 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.47 2024-04-20T10:20:40.398 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T10:20:40.399 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:20:58.613 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-04-20T10:20:59.260 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T10:20:59.260 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T10:20:59.260 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-04-20T10:20:59.307 INFO:teuthology.orchestra.run.smithi175.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T10:20:59.307 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T10:20:59.307 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T10:20:59.307 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T10:21:00.308 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-20T10:21:00.309 DEBUG:teuthology.orchestra.run.smithi175:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T10:21:00.446 INFO:teuthology.orchestra.run.smithi175.stdout:ttyS1 2024-04-20T10:21:00.486 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:40.402 DEBUG:teuthology.orchestra.remote:timed out 2024-04-20T10:21:52.403 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T10:21:52.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:21:52.799 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-20T10:21:53.493 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T10:21:53.493 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.9.0-rc3-g806ec36b2112"... 2024-04-20T10:21:53.493 DEBUG:teuthology.orchestra.run.smithi047:> uname -r 2024-04-20T10:21:53.548 INFO:teuthology.orchestra.run.smithi047.stdout:6.9.0-rc3-g806ec36b2112 2024-04-20T10:21:53.548 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-20T10:21:53.548 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-20T10:21:53.548 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-20T10:21:54.549 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-20T10:21:54.550 DEBUG:teuthology.orchestra.run.smithi047:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-20T10:21:54.675 INFO:teuthology.orchestra.run.smithi047.stdout:ttyS1 2024-04-20T10:21:54.742 DEBUG:teuthology.parallel:result is None 2024-04-20T10:21:54.742 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-20T10:21:54.752 INFO:teuthology.task.internal:Creating test directory... 2024-04-20T10:21:54.752 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:21:54.760 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-20T10:21:54.820 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-20T10:21:54.827 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-20T10:21:54.842 INFO:teuthology.task.internal:Creating archive directory... 2024-04-20T10:21:54.842 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:21:54.850 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-20T10:21:54.926 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-20T10:21:54.937 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-20T10:21:54.937 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:21:54.945 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-20T10:21:55.034 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:21:55.039 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:21:55.064 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:21:55.073 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-20T10:21:55.077 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-20T10:21:55.085 INFO:teuthology.task.internal:Configuring sudo... 2024-04-20T10:21:55.085 DEBUG:teuthology.orchestra.run.smithi047:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:21:55.093 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-20T10:21:55.180 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-20T10:21:55.190 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-20T10:21:55.191 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:21:55.198 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-20T10:21:55.251 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:21:55.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:21:55.438 DEBUG:teuthology.orchestra.run.smithi047:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:21:55.523 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:21:55.613 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:21:55.613 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:21:55.700 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:21:55.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-20T10:21:55.905 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:21:55.987 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-20T10:21:56.070 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:21:56.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-20T10:21:56.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo service rsyslog restart 2024-04-20T10:21:56.156 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-04-20T10:21:56.266 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:21:56.317 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:21:56.814 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-20T10:21:56.823 INFO:teuthology.task.internal:Starting timer... 2024-04-20T10:21:56.823 INFO:teuthology.run_tasks:Running task pcp... 2024-04-20T10:21:56.833 INFO:teuthology.run_tasks:Running task selinux... 2024-04-20T10:21:56.842 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-20T10:21:56.842 DEBUG:teuthology.orchestra.run.smithi047:> sudo service auditd rotate 2024-04-20T10:21:56.971 INFO:teuthology.orchestra.run.smithi047.stdout:Rotating logs: 2024-04-20T10:21:56.976 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-04-20T10:21:57.102 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-04-20T10:21:57.107 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-20T10:21:57.107 DEBUG:teuthology.orchestra.run.smithi047:> /usr/sbin/getenforce 2024-04-20T10:21:57.167 INFO:teuthology.orchestra.run.smithi047.stdout:Permissive 2024-04-20T10:21:57.168 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-04-20T10:21:57.222 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-04-20T10:21:57.223 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi047.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-04-20T10:21:57.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:21:57.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:21:57.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:21:57.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:21:57.386 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-20T10:21:57.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:21:57.464 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-04-20T10:21:57.538 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-20T10:21:57.549 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-20T10:21:57.566 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-20T10:21:57.567 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi047.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-20T10:24:19.811 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi047.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-04-20T10:24:19.813 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T10:24:19.814 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi047.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:24:19.929 DEBUG:teuthology.orchestra.run.smithi047:> true 2024-04-20T10:24:20.098 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi047.front.sepia.ceph.com' 2024-04-20T10:24:20.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T10:24:20.099 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-20T10:24:20.203 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-04-20T10:24:20.362 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-04-20T10:24:20.363 INFO:teuthology.run_tasks:Running task clock... 2024-04-20T10:24:20.376 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-20T10:24:20.377 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:24:20.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:24:20.385 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-20T10:24:20.385 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:24:20.473 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:24:20.478 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-20T10:24:20.516 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:24:20.524 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-20T10:24:20.593 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-04-20T10:24:20.607 INFO:teuthology.orchestra.run.smithi047.stderr:sudo: ntpd: command not found 2024-04-20T10:24:20.625 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-04-20T10:24:20.641 INFO:teuthology.orchestra.run.smithi047.stdout:506 Cannot talk to daemon 2024-04-20T10:24:20.666 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:24:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-20T10:24:20.708 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:24:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-20T10:24:20.832 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:24:20.840 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:24:20.840 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-04-20T10:24:20.840 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.840 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.840 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.841 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.871 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:24:20.881 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:24:20.881 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-20T10:24:20.881 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.881 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.881 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.881 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:24:20.883 INFO:teuthology.run_tasks:Running task install... 2024-04-20T10:24:20.891 DEBUG:teuthology.task.install:project ceph 2024-04-20T10:24:20.892 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}} 2024-04-20T10:24:20.892 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'} 2024-04-20T10:24:20.892 INFO:teuthology.task.install:Using flavor: default 2024-04-20T10:24:20.898 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-20T10:24:20.898 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-20T10:24:20.898 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:24:20.899 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:24:20.901 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'tag': None, 'wait_for_package': False} 2024-04-20T10:24:20.901 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:24:21.045 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:24:21.046 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:24:21.121 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/ 2024-04-20T10:24:21.121 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3133.ga8c84f59 2024-04-20T10:24:21.379 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:24:21.380 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:24:21.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:24:21.442 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240416.193735/a8c84f5997950fd7d008204839253291d52e6eda/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-20T10:24:21.442 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:24:21.442 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-20T10:24:21.471 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:24:21.471 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:24:21.539 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-20T10:24:21.539 DEBUG:teuthology.orchestra.run.smithi047:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a8c84f5997950fd7d008204839253291d52e6eda/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-20T10:24:21.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:24:21.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-20T10:24:21.710 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:24:21.789 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-04-20T10:24:21.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-04-20T10:24:21.797 DEBUG:teuthology.orchestra.run.smithi047:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-20T10:24:21.881 INFO:teuthology.orchestra.run.smithi047.stdout:check_obsoletes = 1 2024-04-20T10:24:21.886 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum clean all 2024-04-20T10:24:22.113 INFO:teuthology.orchestra.run.smithi175.stdout:48 files removed 2024-04-20T10:24:22.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:24:22.235 INFO:teuthology.orchestra.run.smithi047.stdout:48 files removed 2024-04-20T10:24:22.275 DEBUG:teuthology.orchestra.run.smithi047:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-20T10:24:23.270 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 122 kB/s | 84 kB 00:00 2024-04-20T10:24:23.578 INFO:teuthology.orchestra.run.smithi047.stdout:ceph packages for x86_64 97 kB/s | 84 kB 00:00 2024-04-20T10:24:23.887 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 22 kB/s | 12 kB 00:00 2024-04-20T10:24:24.145 INFO:teuthology.orchestra.run.smithi047.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-20T10:24:24.379 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-20T10:24:24.613 INFO:teuthology.orchestra.run.smithi047.stdout:ceph source packages 4.9 kB/s | 1.9 kB 00:00 2024-04-20T10:24:25.694 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.0 MB 00:01 2024-04-20T10:24:27.695 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-20T10:24:32.104 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-20T10:24:34.165 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - BaseOS 870 kB/s | 8.0 MB 00:09 2024-04-20T10:24:34.788 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 12 kB/s | 16 kB 00:01 2024-04-20T10:24:35.949 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-04-20T10:24:36.278 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-20T10:24:41.429 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - CRB 5.3 MB/s | 6.2 MB 00:01 2024-04-20T10:24:41.983 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-20T10:24:43.221 INFO:teuthology.orchestra.run.smithi047.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-20T10:24:44.467 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:24:44.469 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:24:44.490 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:24:44.491 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:24:44.492 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:24:44.493 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:24:44.496 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:24:44.499 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:24:44.501 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:24:44.570 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:24:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:24:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:24:44.600 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:24:44.601 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:24:44.602 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:24:44.603 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:24:44.604 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:24:44.605 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:24:44.606 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:24:44.606 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:24:44.606 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:24:44.606 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:24:44.606 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:24:44.607 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:24:44.607 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:24:44.607 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:24:44.607 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:24:44.607 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:24:44.607 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:24:44.608 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:24:44.609 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:24:44.610 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout:Install 148 Packages 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 2 Packages 2024-04-20T10:24:44.611 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:24:44.612 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 218 M 2024-04-20T10:24:44.612 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-04-20T10:24:45.249 INFO:teuthology.orchestra.run.smithi047.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-20T10:24:45.601 INFO:teuthology.orchestra.run.smithi175.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 22 kB/s | 6.6 kB 00:00 2024-04-20T10:24:45.918 INFO:teuthology.orchestra.run.smithi175.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 2.8 MB/s | 893 kB 00:00 2024-04-20T10:24:46.010 INFO:teuthology.orchestra.run.smithi175.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 1.6 MB/s | 145 kB 00:00 2024-04-20T10:24:46.518 INFO:teuthology.orchestra.run.smithi175.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 4.4 MB/s | 5.4 MB 00:01 2024-04-20T10:24:46.593 INFO:teuthology.orchestra.run.smithi175.stdout:(5/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 4.1 MB/s | 2.4 MB 00:00 2024-04-20T10:24:46.760 INFO:teuthology.orchestra.run.smithi175.stdout:(6/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 6.2 MB/s | 1.5 MB 00:00 2024-04-20T10:24:47.161 INFO:teuthology.orchestra.run.smithi175.stdout:(7/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 11 MB/s | 21 MB 00:01 2024-04-20T10:24:47.310 INFO:teuthology.orchestra.run.smithi175.stdout:(8/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-04-20T10:24:47.386 INFO:teuthology.orchestra.run.smithi175.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 338 kB/s | 25 kB 00:00 2024-04-20T10:24:48.178 INFO:teuthology.orchestra.run.smithi175.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 12 MB/s | 17 MB 00:01 2024-04-20T10:24:48.253 INFO:teuthology.orchestra.run.smithi175.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 3.0 MB/s | 225 kB 00:00 2024-04-20T10:24:48.328 INFO:teuthology.orchestra.run.smithi175.stdout:(12/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 367 kB/s | 27 kB 00:00 2024-04-20T10:24:48.412 INFO:teuthology.orchestra.run.smithi175.stdout:(13/150): libcephfs-devel-19.0.0-3133.ga8c84f59 383 kB/s | 32 kB 00:00 2024-04-20T10:24:48.587 INFO:teuthology.orchestra.run.smithi175.stdout:(14/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 4.2 MB/s | 748 kB 00:00 2024-04-20T10:24:48.687 INFO:teuthology.orchestra.run.smithi175.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 1.6 MB/s | 163 kB 00:00 2024-04-20T10:24:48.787 INFO:teuthology.orchestra.run.smithi175.stdout:(16/150): librados-devel-19.0.0-3133.ga8c84f59. 1.2 MB/s | 126 kB 00:00 2024-04-20T10:24:48.921 INFO:teuthology.orchestra.run.smithi175.stdout:(17/150): libradosstriper1-19.0.0-3133.ga8c84f5 3.7 MB/s | 501 kB 00:00 2024-04-20T10:24:49.555 INFO:teuthology.orchestra.run.smithi175.stdout:(18/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 4.4 MB/s | 11 MB 00:02 2024-04-20T10:24:49.671 INFO:teuthology.orchestra.run.smithi175.stdout:(19/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 7.2 MB/s | 5.4 MB 00:00 2024-04-20T10:24:49.697 INFO:teuthology.orchestra.run.smithi175.stdout:(20/150): python3-ceph-argparse-19.0.0-3133.ga8 321 kB/s | 45 kB 00:00 2024-04-20T10:24:49.739 INFO:teuthology.orchestra.run.smithi175.stdout:(21/150): python3-ceph-common-19.0.0-3133.ga8c8 2.0 MB/s | 135 kB 00:00 2024-04-20T10:24:49.772 INFO:teuthology.orchestra.run.smithi175.stdout:(22/150): python3-cephfs-19.0.0-3133.ga8c84f59. 2.1 MB/s | 162 kB 00:00 2024-04-20T10:24:49.814 INFO:teuthology.orchestra.run.smithi175.stdout:(23/150): python3-rados-19.0.0-3133.ga8c84f59.e 4.2 MB/s | 321 kB 00:00 2024-04-20T10:24:49.848 INFO:teuthology.orchestra.run.smithi175.stdout:(24/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 3.9 MB/s | 299 kB 00:00 2024-04-20T10:24:49.881 INFO:teuthology.orchestra.run.smithi175.stdout:(25/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.5 MB/s | 99 kB 00:00 2024-04-20T10:24:49.898 INFO:teuthology.orchestra.run.smithi175.stdout:(26/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.7 MB/s | 85 kB 00:00 2024-04-20T10:24:49.957 INFO:teuthology.orchestra.run.smithi175.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.9 MB/s | 170 kB 00:00 2024-04-20T10:24:50.024 INFO:teuthology.orchestra.run.smithi175.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3133.g 521 kB/s | 34 kB 00:00 2024-04-20T10:24:50.099 INFO:teuthology.orchestra.run.smithi175.stdout:(29/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 1.9 MB/s | 142 kB 00:00 2024-04-20T10:24:50.266 INFO:teuthology.orchestra.run.smithi175.stdout:(30/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 8.1 MB/s | 3.1 MB 00:00 2024-04-20T10:24:51.233 INFO:teuthology.orchestra.run.smithi175.stdout:(31/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 13 MB/s | 49 MB 00:03 2024-04-20T10:24:51.300 INFO:teuthology.orchestra.run.smithi047.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-20T10:24:51.317 INFO:teuthology.orchestra.run.smithi175.stdout:(32/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 2.9 MB/s | 3.5 MB 00:01 2024-04-20T10:24:51.350 INFO:teuthology.orchestra.run.smithi175.stdout:(33/150): ceph-mgr-modules-core-19.0.0-3133.ga8 2.1 MB/s | 249 kB 00:00 2024-04-20T10:24:51.375 INFO:teuthology.orchestra.run.smithi175.stdout:(34/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 858 kB/s | 49 kB 00:00 2024-04-20T10:24:51.409 INFO:teuthology.orchestra.run.smithi175.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3133.ga 286 kB/s | 16 kB 00:00 2024-04-20T10:24:51.551 INFO:teuthology.orchestra.run.smithi175.stdout:(36/150): ceph-mgr-diskprediction-local-19.0.0- 5.8 MB/s | 7.4 MB 00:01 2024-04-20T10:24:51.585 INFO:teuthology.orchestra.run.smithi175.stdout:(37/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.3 MB/s | 273 kB 00:00 2024-04-20T10:24:51.601 INFO:teuthology.orchestra.run.smithi175.stdout:(38/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 368 kB/s | 18 kB 00:00 2024-04-20T10:24:51.651 INFO:teuthology.orchestra.run.smithi175.stdout:(39/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 3.1 MB/s | 763 kB 00:00 2024-04-20T10:24:51.686 INFO:teuthology.orchestra.run.smithi175.stdout:(40/150): fuse-2.9.9-15.el9.x86_64.rpm 807 kB/s | 80 kB 00:00 2024-04-20T10:24:51.752 INFO:teuthology.orchestra.run.smithi175.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2024-04-20T10:24:51.794 INFO:teuthology.orchestra.run.smithi175.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 4.1 MB/s | 803 kB 00:00 2024-04-20T10:24:51.819 INFO:teuthology.orchestra.run.smithi175.stdout:(43/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 253 kB 00:00 2024-04-20T10:24:51.878 INFO:teuthology.orchestra.run.smithi175.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2024-04-20T10:24:51.903 INFO:teuthology.orchestra.run.smithi175.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 1.6 MB/s | 135 kB 00:00 2024-04-20T10:24:51.953 INFO:teuthology.orchestra.run.smithi175.stdout:(46/150): python3-cryptography-36.0.1-4.el9.x86 6.1 MB/s | 1.2 MB 00:00 2024-04-20T10:24:51.987 INFO:teuthology.orchestra.run.smithi175.stdout:(47/150): python3-requests-2.25.1-8.el9.noarch. 1.1 MB/s | 125 kB 00:00 2024-04-20T10:24:52.020 INFO:teuthology.orchestra.run.smithi175.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-20T10:24:52.204 INFO:teuthology.orchestra.run.smithi175.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 583 kB/s | 107 kB 00:00 2024-04-20T10:24:52.263 INFO:teuthology.orchestra.run.smithi175.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 377 kB/s | 21 kB 00:00 2024-04-20T10:24:52.297 INFO:teuthology.orchestra.run.smithi175.stdout:(51/150): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 689 kB 00:00 2024-04-20T10:24:52.346 INFO:teuthology.orchestra.run.smithi175.stdout:(52/150): bison-3.7.4-5.el9.x86_64.rpm 2.6 MB/s | 943 kB 00:00 2024-04-20T10:24:52.372 INFO:teuthology.orchestra.run.smithi175.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 443 kB/s | 33 kB 00:00 2024-04-20T10:24:52.405 INFO:teuthology.orchestra.run.smithi175.stdout:(54/150): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-20T10:24:52.431 INFO:teuthology.orchestra.run.smithi175.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 315 kB/s | 18 kB 00:00 2024-04-20T10:24:52.464 INFO:teuthology.orchestra.run.smithi175.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-04-20T10:24:52.498 INFO:teuthology.orchestra.run.smithi175.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-04-20T10:24:52.523 INFO:teuthology.orchestra.run.smithi175.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 787 kB/s | 45 kB 00:00 2024-04-20T10:24:52.590 INFO:teuthology.orchestra.run.smithi175.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-04-20T10:24:52.657 INFO:teuthology.orchestra.run.smithi175.stdout:(60/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 9.6 MB/s | 3.0 MB 00:00 2024-04-20T10:24:52.682 INFO:teuthology.orchestra.run.smithi175.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 279 kB/s | 25 kB 00:00 2024-04-20T10:24:52.724 INFO:teuthology.orchestra.run.smithi175.stdout:(62/150): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-04-20T10:24:52.758 INFO:teuthology.orchestra.run.smithi175.stdout:(63/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 476 kB/s | 47 kB 00:00 2024-04-20T10:24:52.792 INFO:teuthology.orchestra.run.smithi175.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.8 MB/s | 188 kB 00:00 2024-04-20T10:24:52.817 INFO:teuthology.orchestra.run.smithi175.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-04-20T10:24:52.850 INFO:teuthology.orchestra.run.smithi175.stdout:(66/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-04-20T10:24:52.967 INFO:teuthology.orchestra.run.smithi175.stdout:(67/150): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-20T10:24:53.068 INFO:teuthology.orchestra.run.smithi175.stdout:(68/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 17 MB/s | 4.1 MB 00:00 2024-04-20T10:24:53.126 INFO:teuthology.orchestra.run.smithi175.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-04-20T10:24:53.160 INFO:teuthology.orchestra.run.smithi175.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-20T10:24:53.268 INFO:teuthology.orchestra.run.smithi175.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 10 MB/s | 4.7 MB 00:00 2024-04-20T10:24:53.335 INFO:teuthology.orchestra.run.smithi175.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-20T10:24:53.360 INFO:teuthology.orchestra.run.smithi175.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 963 kB/s | 87 kB 00:00 2024-04-20T10:24:53.427 INFO:teuthology.orchestra.run.smithi175.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-20T10:24:53.494 INFO:teuthology.orchestra.run.smithi175.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-20T10:24:53.603 INFO:teuthology.orchestra.run.smithi175.stdout:(76/150): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-20T10:24:53.728 INFO:teuthology.orchestra.run.smithi175.stdout:(77/150): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-20T10:24:53.753 INFO:teuthology.orchestra.run.smithi175.stdout:(78/150): python3-jsonpatch-1.21-16.el9.noarch. 99 kB/s | 26 kB 00:00 2024-04-20T10:24:53.778 INFO:teuthology.orchestra.run.smithi175.stdout:(79/150): python3-jsonpointer-2.0-4.el9.noarch. 109 kB/s | 19 kB 00:00 2024-04-20T10:24:53.780 INFO:teuthology.orchestra.run.smithi047.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:24:53.781 INFO:teuthology.orchestra.run.smithi047.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-20T10:24:53.799 INFO:teuthology.orchestra.run.smithi047.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-20T10:24:53.800 INFO:teuthology.orchestra.run.smithi047.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-20T10:24:53.802 INFO:teuthology.orchestra.run.smithi047.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-20T10:24:53.803 INFO:teuthology.orchestra.run.smithi047.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-20T10:24:53.805 INFO:teuthology.orchestra.run.smithi047.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-20T10:24:53.808 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-20T10:24:53.809 INFO:teuthology.orchestra.run.smithi047.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-20T10:24:53.812 INFO:teuthology.orchestra.run.smithi175.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-20T10:24:53.837 INFO:teuthology.orchestra.run.smithi175.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 605 kB/s | 35 kB 00:00 2024-04-20T10:24:53.879 INFO:teuthology.orchestra.run.smithi175.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-20T10:24:53.882 INFO:teuthology.orchestra.run.smithi047.stdout:Dependencies resolved. 2024-04-20T10:24:53.898 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: Package Arch Version Repository Size 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout:Installing: 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 6.6 k 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 893 k 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 145 k 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 1.5 M 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 142 k 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 3.5 M 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 7.4 M 2024-04-20T10:24:53.899 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 49 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 11 M 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 49 M 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 273 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 763 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 225 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 27 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 18 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 32 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 748 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 126 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-20T10:24:53.900 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 162 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 321 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 299 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 99 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 85 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 170 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrading: 2024-04-20T10:24:53.901 INFO:teuthology.orchestra.run.smithi047.stdout: librados2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.4 M 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 3.1 M 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout:Installing dependencies: 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 21 M 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 34 k 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 2.4 M 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 249 k 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 4.7 M 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 17 M 2024-04-20T10:24:53.902 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3133.ga8c84f59.el9 ceph-noarch 16 k 2024-04-20T10:24:53.905 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 25 k 2024-04-20T10:24:53.905 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 163 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-20T10:24:53.906 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 501 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2 x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 5.4 M 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-20T10:24:53.907 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 45 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common x86_64 2:19.0.0-3133.ga8c84f59.el9 ceph 135 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-20T10:24:53.908 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-20T10:24:53.909 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-20T10:24:53.910 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-20T10:24:53.911 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-20T10:24:53.912 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout:Installing weak dependencies: 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction Summary 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout:======================================================================================= 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout:Install 148 Packages 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout:Upgrade 2 Packages 2024-04-20T10:24:53.913 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:24:53.914 INFO:teuthology.orchestra.run.smithi047.stdout:Total download size: 218 M 2024-04-20T10:24:53.914 INFO:teuthology.orchestra.run.smithi047.stdout:Downloading Packages: 2024-04-20T10:24:53.921 INFO:teuthology.orchestra.run.smithi175.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-20T10:24:53.946 INFO:teuthology.orchestra.run.smithi175.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-04-20T10:24:54.010 INFO:teuthology.orchestra.run.smithi175.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 482 kB/s | 42 kB 00:00 2024-04-20T10:24:54.047 INFO:teuthology.orchestra.run.smithi175.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-20T10:24:54.081 INFO:teuthology.orchestra.run.smithi175.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 159 kB 00:00 2024-04-20T10:24:54.114 INFO:teuthology.orchestra.run.smithi175.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-20T10:24:54.148 INFO:teuthology.orchestra.run.smithi175.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 770 kB/s | 51 kB 00:00 2024-04-20T10:24:54.173 INFO:teuthology.orchestra.run.smithi175.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 925 kB/s | 54 kB 00:00 2024-04-20T10:24:54.232 INFO:teuthology.orchestra.run.smithi175.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 723 kB/s | 42 kB 00:00 2024-04-20T10:24:54.349 INFO:teuthology.orchestra.run.smithi175.stdout:(92/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.0 MB/s | 5.1 MB 00:00 2024-04-20T10:24:54.374 INFO:teuthology.orchestra.run.smithi175.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 313 kB/s | 44 kB 00:00 2024-04-20T10:24:54.416 INFO:teuthology.orchestra.run.smithi175.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.3 MB/s | 221 kB 00:00 2024-04-20T10:24:54.441 INFO:teuthology.orchestra.run.smithi175.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 543 kB/s | 36 kB 00:00 2024-04-20T10:24:54.475 INFO:teuthology.orchestra.run.smithi175.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 822 kB/s | 47 kB 00:00 2024-04-20T10:24:54.508 INFO:teuthology.orchestra.run.smithi175.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-20T10:24:54.533 INFO:teuthology.orchestra.run.smithi175.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-04-20T10:24:54.567 INFO:teuthology.orchestra.run.smithi175.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-20T10:24:54.592 INFO:teuthology.orchestra.run.smithi175.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-20T10:24:54.626 INFO:teuthology.orchestra.run.smithi175.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-04-20T10:24:54.651 INFO:teuthology.orchestra.run.smithi175.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 386 kB/s | 22 kB 00:00 2024-04-20T10:24:54.668 INFO:teuthology.orchestra.run.smithi047.stdout:(1/150): ceph-19.0.0-3133.ga8c84f59.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-20T10:24:54.768 INFO:teuthology.orchestra.run.smithi175.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 6.0 MB/s | 863 kB 00:00 2024-04-20T10:24:54.852 INFO:teuthology.orchestra.run.smithi175.stdout:(104/150): gperftools-libs-2.9.1-2.el9.x86_64.r 3.8 MB/s | 309 kB 00:00 2024-04-20T10:24:54.885 INFO:teuthology.orchestra.run.smithi175.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 2.3 MB/s | 548 kB 00:00 2024-04-20T10:24:55.027 INFO:teuthology.orchestra.run.smithi047.stdout:(2/150): ceph-fuse-19.0.0-3133.ga8c84f59.el9.x8 2.4 MB/s | 893 kB 00:00 2024-04-20T10:24:55.027 INFO:teuthology.orchestra.run.smithi175.stdout:(106/150): libarrow-9.0.0-10.el9.x86_64.rpm 31 MB/s | 4.4 MB 00:00 2024-04-20T10:24:55.053 INFO:teuthology.orchestra.run.smithi175.stdout:(107/150): grpc-data-1.46.7-10.el9.noarch.rpm 99 kB/s | 19 kB 00:00 2024-04-20T10:24:55.078 INFO:teuthology.orchestra.run.smithi175.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 528 kB/s | 26 kB 00:00 2024-04-20T10:24:55.101 INFO:teuthology.orchestra.run.smithi047.stdout:(3/150): ceph-immutable-object-cache-19.0.0-313 1.9 MB/s | 145 kB 00:00 2024-04-20T10:24:55.103 INFO:teuthology.orchestra.run.smithi175.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 990 kB/s | 49 kB 00:00 2024-04-20T10:24:55.128 INFO:teuthology.orchestra.run.smithi175.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-04-20T10:24:55.195 INFO:teuthology.orchestra.run.smithi175.stdout:(111/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2024-04-20T10:24:55.260 INFO:teuthology.orchestra.run.smithi047.stdout:(4/150): ceph-base-19.0.0-3133.ga8c84f59.el9.x8 6.1 MB/s | 5.4 MB 00:00 2024-04-20T10:24:55.421 INFO:teuthology.orchestra.run.smithi175.stdout:(112/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-04-20T10:24:55.462 INFO:teuthology.orchestra.run.smithi175.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 2.0 MB/s | 548 kB 00:00 2024-04-20T10:24:55.469 INFO:teuthology.orchestra.run.smithi047.stdout:(5/150): ceph-mgr-19.0.0-3133.ga8c84f59.el9.x86 7.2 MB/s | 1.5 MB 00:00 2024-04-20T10:24:55.487 INFO:teuthology.orchestra.run.smithi175.stdout:(114/150): luarocks-3.9.2-1.el9.noarch.rpm 395 kB/s | 151 kB 00:00 2024-04-20T10:24:55.513 INFO:teuthology.orchestra.run.smithi175.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 504 kB/s | 43 kB 00:00 2024-04-20T10:24:55.538 INFO:teuthology.orchestra.run.smithi175.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 305 kB/s | 15 kB 00:00 2024-04-20T10:24:55.563 INFO:teuthology.orchestra.run.smithi175.stdout:(117/150): python3-cachetools-4.2.4-1.el9.noarc 323 kB/s | 32 kB 00:00 2024-04-20T10:24:55.586 INFO:teuthology.orchestra.run.smithi047.stdout:(6/150): ceph-mds-19.0.0-3133.ga8c84f59.el9.x86 4.9 MB/s | 2.4 MB 00:00 2024-04-20T10:24:55.589 INFO:teuthology.orchestra.run.smithi175.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-20T10:24:55.622 INFO:teuthology.orchestra.run.smithi175.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.3 MB/s | 303 kB 00:00 2024-04-20T10:24:55.656 INFO:teuthology.orchestra.run.smithi175.stdout:(120/150): python3-cherrypy-18.6.1-2.el9.noarch 3.0 MB/s | 358 kB 00:00 2024-04-20T10:24:55.682 INFO:teuthology.orchestra.run.smithi175.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 3.6 MB/s | 203 kB 00:00 2024-04-20T10:24:55.706 INFO:teuthology.orchestra.run.smithi175.stdout:(122/150): python3-colorama-0.4.4-12.el9.noarch 310 kB/s | 35 kB 00:00 2024-04-20T10:24:55.749 INFO:teuthology.orchestra.run.smithi175.stdout:(123/150): python3-jaraco-8.2.1-3.el9.noarch.rp 257 kB/s | 11 kB 00:00 2024-04-20T10:24:55.816 INFO:teuthology.orchestra.run.smithi175.stdout:(124/150): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-04-20T10:24:55.841 INFO:teuthology.orchestra.run.smithi175.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 214 kB/s | 18 kB 00:00 2024-04-20T10:24:55.883 INFO:teuthology.orchestra.run.smithi175.stdout:(126/150): python3-grpcio-tools-1.46.7-10.el9.x 719 kB/s | 144 kB 00:00 2024-04-20T10:24:55.925 INFO:teuthology.orchestra.run.smithi175.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 215 kB/s | 23 kB 00:00 2024-04-20T10:24:55.950 INFO:teuthology.orchestra.run.smithi175.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 181 kB/s | 19 kB 00:00 2024-04-20T10:24:55.975 INFO:teuthology.orchestra.run.smithi175.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 217 kB/s | 20 kB 00:00 2024-04-20T10:24:56.025 INFO:teuthology.orchestra.run.smithi175.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 14 MB/s | 1.0 MB 00:00 2024-04-20T10:24:56.050 INFO:teuthology.orchestra.run.smithi175.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 469 kB/s | 46 kB 00:00 2024-04-20T10:24:56.076 INFO:teuthology.orchestra.run.smithi175.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 882 kB/s | 79 kB 00:00 2024-04-20T10:24:56.101 INFO:teuthology.orchestra.run.smithi175.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 1.2 MB/s | 58 kB 00:00 2024-04-20T10:24:56.134 INFO:teuthology.orchestra.run.smithi175.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.3 MB/s | 272 kB 00:00 2024-04-20T10:24:56.162 INFO:teuthology.orchestra.run.smithi175.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 199 kB/s | 16 kB 00:00 2024-04-20T10:24:56.185 INFO:teuthology.orchestra.run.smithi175.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-20T10:24:56.281 INFO:teuthology.orchestra.run.smithi175.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 195 kB/s | 28 kB 00:00 2024-04-20T10:24:56.327 INFO:teuthology.orchestra.run.smithi175.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-04-20T10:24:56.352 INFO:teuthology.orchestra.run.smithi175.stdout:(139/150): python3-repoze-lru-0.7-16.el9.noarch 186 kB/s | 31 kB 00:00 2024-04-20T10:24:56.377 INFO:teuthology.orchestra.run.smithi175.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 630 kB/s | 59 kB 00:00 2024-04-20T10:24:56.386 INFO:teuthology.orchestra.run.smithi047.stdout:(7/150): ceph-common-19.0.0-3133.ga8c84f59.el9. 11 MB/s | 21 MB 00:02 2024-04-20T10:24:56.411 INFO:teuthology.orchestra.run.smithi175.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 1.5 MB/s | 51 kB 00:00 2024-04-20T10:24:56.436 INFO:teuthology.orchestra.run.smithi175.stdout:(142/150): python3-tempora-5.0.0-2.el9.noarch.r 333 kB/s | 36 kB 00:00 2024-04-20T10:24:56.461 INFO:teuthology.orchestra.run.smithi175.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-04-20T10:24:56.495 INFO:teuthology.orchestra.run.smithi175.stdout:(144/150): python3-werkzeug-2.0.3-3.el9.1.noarc 7.3 MB/s | 427 kB 00:00 2024-04-20T10:24:56.503 INFO:teuthology.orchestra.run.smithi047.stdout:(8/150): ceph-mon-19.0.0-3133.ga8c84f59.el9.x86 4.6 MB/s | 4.7 MB 00:01 2024-04-20T10:24:56.529 INFO:teuthology.orchestra.run.smithi175.stdout:(145/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.5 MB/s | 230 kB 00:00 2024-04-20T10:24:56.562 INFO:teuthology.orchestra.run.smithi175.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.9 MB/s | 192 kB 00:00 2024-04-20T10:24:56.595 INFO:teuthology.orchestra.run.smithi047.stdout:(9/150): ceph-selinux-19.0.0-3133.ga8c84f59.el9 277 kB/s | 25 kB 00:00 2024-04-20T10:24:56.596 INFO:teuthology.orchestra.run.smithi175.stdout:(147/150): python3-zc-lockfile-2.0-10.el9.noarc 154 kB/s | 20 kB 00:00 2024-04-20T10:24:56.646 INFO:teuthology.orchestra.run.smithi175.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-20T10:24:57.513 INFO:teuthology.orchestra.run.smithi175.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 3.4 MB/s | 3.1 MB 00:00 2024-04-20T10:24:57.588 INFO:teuthology.orchestra.run.smithi175.stdout:(150/150): librados2-19.0.0-3133.ga8c84f59.el9. 3.4 MB/s | 3.4 MB 00:01 2024-04-20T10:24:57.596 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:24:57.597 INFO:teuthology.orchestra.run.smithi175.stdout:Total 17 MB/s | 218 MB 00:12 2024-04-20T10:24:57.862 INFO:teuthology.orchestra.run.smithi047.stdout:(10/150): ceph-osd-19.0.0-3133.ga8c84f59.el9.x8 7.4 MB/s | 17 MB 00:02 2024-04-20T10:24:57.945 INFO:teuthology.orchestra.run.smithi047.stdout:(11/150): cephfs-mirror-19.0.0-3133.ga8c84f59.e 2.7 MB/s | 225 kB 00:00 2024-04-20T10:24:58.104 INFO:teuthology.orchestra.run.smithi047.stdout:(12/150): ceph-radosgw-19.0.0-3133.ga8c84f59.el 6.2 MB/s | 11 MB 00:01 2024-04-20T10:24:58.146 INFO:teuthology.orchestra.run.smithi047.stdout:(13/150): cephfs-shell-19.0.0-3133.ga8c84f59.el 135 kB/s | 27 kB 00:00 2024-04-20T10:24:58.171 INFO:teuthology.orchestra.run.smithi047.stdout:(14/150): libcephfs-devel-19.0.0-3133.ga8c84f59 482 kB/s | 32 kB 00:00 2024-04-20T10:24:58.238 INFO:teuthology.orchestra.run.smithi047.stdout:(15/150): libcephsqlite-19.0.0-3133.ga8c84f59.e 2.4 MB/s | 163 kB 00:00 2024-04-20T10:24:58.280 INFO:teuthology.orchestra.run.smithi047.stdout:(16/150): libcephfs2-19.0.0-3133.ga8c84f59.el9. 5.5 MB/s | 748 kB 00:00 2024-04-20T10:24:58.314 INFO:teuthology.orchestra.run.smithi047.stdout:(17/150): librados-devel-19.0.0-3133.ga8c84f59. 1.7 MB/s | 126 kB 00:00 2024-04-20T10:24:58.397 INFO:teuthology.orchestra.run.smithi047.stdout:(18/150): libradosstriper1-19.0.0-3133.ga8c84f5 4.2 MB/s | 501 kB 00:00 2024-04-20T10:24:58.464 INFO:teuthology.orchestra.run.smithi047.stdout:(19/150): python3-ceph-argparse-19.0.0-3133.ga8 687 kB/s | 45 kB 00:00 2024-04-20T10:24:58.540 INFO:teuthology.orchestra.run.smithi047.stdout:(20/150): python3-ceph-common-19.0.0-3133.ga8c8 1.8 MB/s | 135 kB 00:00 2024-04-20T10:24:58.623 INFO:teuthology.orchestra.run.smithi047.stdout:(21/150): python3-cephfs-19.0.0-3133.ga8c84f59. 1.9 MB/s | 162 kB 00:00 2024-04-20T10:24:58.715 INFO:teuthology.orchestra.run.smithi047.stdout:(22/150): python3-rados-19.0.0-3133.ga8c84f59.e 3.5 MB/s | 321 kB 00:00 2024-04-20T10:24:58.915 INFO:teuthology.orchestra.run.smithi047.stdout:(23/150): python3-rbd-19.0.0-3133.ga8c84f59.el9 1.5 MB/s | 299 kB 00:00 2024-04-20T10:24:58.990 INFO:teuthology.orchestra.run.smithi047.stdout:(24/150): python3-rgw-19.0.0-3133.ga8c84f59.el9 1.3 MB/s | 99 kB 00:00 2024-04-20T10:24:59.066 INFO:teuthology.orchestra.run.smithi047.stdout:(25/150): rbd-fuse-19.0.0-3133.ga8c84f59.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-20T10:24:59.391 INFO:teuthology.orchestra.run.smithi047.stdout:(26/150): librgw2-19.0.0-3133.ga8c84f59.el9.x86 5.0 MB/s | 5.4 MB 00:01 2024-04-20T10:24:59.450 INFO:teuthology.orchestra.run.smithi047.stdout:(27/150): rbd-nbd-19.0.0-3133.ga8c84f59.el9.x86 2.9 MB/s | 170 kB 00:00 2024-04-20T10:24:59.533 INFO:teuthology.orchestra.run.smithi047.stdout:(28/150): rbd-mirror-19.0.0-3133.ga8c84f59.el9. 6.6 MB/s | 3.1 MB 00:00 2024-04-20T10:24:59.559 INFO:teuthology.orchestra.run.smithi047.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3133.g 318 kB/s | 34 kB 00:00 2024-04-20T10:24:59.600 INFO:teuthology.orchestra.run.smithi047.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3133.ga8c84f5 2.1 MB/s | 142 kB 00:00 2024-04-20T10:25:00.184 INFO:teuthology.orchestra.run.smithi047.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3133.ga8c84 5.7 MB/s | 3.5 MB 00:00 2024-04-20T10:25:00.284 INFO:teuthology.orchestra.run.smithi047.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3133.ga8 2.5 MB/s | 249 kB 00:00 2024-04-20T10:25:00.351 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-04-20T10:25:00.359 INFO:teuthology.orchestra.run.smithi047.stdout:(33/150): ceph-mgr-rook-19.0.0-3133.ga8c84f59.e 669 kB/s | 49 kB 00:00 2024-04-20T10:25:00.418 INFO:teuthology.orchestra.run.smithi047.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3133.ga 287 kB/s | 16 kB 00:00 2024-04-20T10:25:00.572 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-04-20T10:25:00.572 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-04-20T10:25:00.577 INFO:teuthology.orchestra.run.smithi047.stdout:(35/150): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-04-20T10:25:00.610 INFO:teuthology.orchestra.run.smithi047.stdout:(36/150): ceph-volume-19.0.0-3133.ga8c84f59.el9 1.4 MB/s | 273 kB 00:00 2024-04-20T10:25:00.662 INFO:teuthology.orchestra.run.smithi047.stdout:(37/150): cephfs-top-19.0.0-3133.ga8c84f59.el9. 370 kB/s | 18 kB 00:00 2024-04-20T10:25:00.702 INFO:teuthology.orchestra.run.smithi047.stdout:(38/150): cephadm-19.0.0-3133.ga8c84f59.el9.noa 6.0 MB/s | 763 kB 00:00 2024-04-20T10:25:00.820 INFO:teuthology.orchestra.run.smithi047.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 511 kB/s | 80 kB 00:00 2024-04-20T10:25:01.336 INFO:teuthology.orchestra.run.smithi047.stdout:(40/150): ceph-test-19.0.0-3133.ga8c84f59.el9.x 10 MB/s | 49 MB 00:04 2024-04-20T10:25:01.370 INFO:teuthology.orchestra.run.smithi047.stdout:(41/150): libquadmath-11.4.1-3.el9.x86_64.rpm 355 kB/s | 195 kB 00:00 2024-04-20T10:25:01.420 INFO:teuthology.orchestra.run.smithi047.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-20T10:25:01.488 INFO:teuthology.orchestra.run.smithi047.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-04-20T10:25:01.521 INFO:teuthology.orchestra.run.smithi047.stdout:(44/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-04-20T10:25:01.555 INFO:teuthology.orchestra.run.smithi047.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-04-20T10:25:01.588 INFO:teuthology.orchestra.run.smithi047.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-04-20T10:25:01.639 INFO:teuthology.orchestra.run.smithi047.stdout:(47/150): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-04-20T10:25:01.681 INFO:teuthology.orchestra.run.smithi047.stdout:(48/150): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-04-20T10:25:02.031 INFO:teuthology.orchestra.run.smithi047.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 306 kB/s | 107 kB 00:00 2024-04-20T10:25:02.131 INFO:teuthology.orchestra.run.smithi047.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 217 kB/s | 21 kB 00:00 2024-04-20T10:25:02.173 INFO:teuthology.orchestra.run.smithi047.stdout:(51/150): automake-1.16.2-8.el9.noarch.rpm 1.2 MB/s | 689 kB 00:00 2024-04-20T10:25:02.265 INFO:teuthology.orchestra.run.smithi047.stdout:(52/150): flexiblas-3.0.4-8.el9.x86_64.rpm 360 kB/s | 33 kB 00:00 2024-04-20T10:25:02.298 INFO:teuthology.orchestra.run.smithi047.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-04-20T10:25:02.390 INFO:teuthology.orchestra.run.smithi047.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 199 kB/s | 18 kB 00:00 2024-04-20T10:25:02.466 INFO:teuthology.orchestra.run.smithi047.stdout:(55/150): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-04-20T10:25:02.491 INFO:teuthology.orchestra.run.smithi047.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 112 kB/s | 11 kB 00:00 2024-04-20T10:25:02.599 INFO:teuthology.orchestra.run.smithi047.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 420 kB/s | 45 kB 00:00 2024-04-20T10:25:02.633 INFO:teuthology.orchestra.run.smithi047.stdout:(58/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 962 kB/s | 160 kB 00:00 2024-04-20T10:25:02.767 INFO:teuthology.orchestra.run.smithi047.stdout:(59/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.0 MB/s | 3.0 MB 00:00 2024-04-20T10:25:02.808 INFO:teuthology.orchestra.run.smithi047.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-04-20T10:25:02.850 INFO:teuthology.orchestra.run.smithi047.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 308 kB/s | 25 kB 00:00 2024-04-20T10:25:02.876 INFO:teuthology.orchestra.run.smithi047.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 719 kB/s | 47 kB 00:00 2024-04-20T10:25:02.934 INFO:teuthology.orchestra.run.smithi047.stdout:(63/150): libtool-2.4.6-45.el9.x86_64.rpm 1.9 MB/s | 582 kB 00:00 2024-04-20T10:25:02.968 INFO:teuthology.orchestra.run.smithi047.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-04-20T10:25:03.001 INFO:teuthology.orchestra.run.smithi047.stdout:(65/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-20T10:25:03.027 INFO:teuthology.orchestra.run.smithi047.stdout:(66/150): openblas-0.3.21-2.el9.x86_64.rpm 386 kB/s | 35 kB 00:00 2024-04-20T10:25:03.232 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-04-20T10:25:03.232 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-04-20T10:25:03.352 INFO:teuthology.orchestra.run.smithi047.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 12 MB/s | 4.1 MB 00:00 2024-04-20T10:25:03.460 INFO:teuthology.orchestra.run.smithi047.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 3.7 MB/s | 1.6 MB 00:00 2024-04-20T10:25:03.527 INFO:teuthology.orchestra.run.smithi047.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 5.1 MB/s | 911 kB 00:00 2024-04-20T10:25:03.578 INFO:teuthology.orchestra.run.smithi047.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-04-20T10:25:03.704 INFO:teuthology.orchestra.run.smithi047.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 6.4 MB/s | 4.7 MB 00:00 2024-04-20T10:25:03.779 INFO:teuthology.orchestra.run.smithi047.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.2 MB/s | 87 kB 00:00 2024-04-20T10:25:03.837 INFO:teuthology.orchestra.run.smithi047.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-04-20T10:25:03.921 INFO:teuthology.orchestra.run.smithi047.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-04-20T10:25:04.004 INFO:teuthology.orchestra.run.smithi047.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-04-20T10:25:04.071 INFO:teuthology.orchestra.run.smithi047.stdout:(76/150): python3-jsonpatch-1.21-16.el9.noarch. 388 kB/s | 26 kB 00:00 2024-04-20T10:25:04.138 INFO:teuthology.orchestra.run.smithi047.stdout:(77/150): python3-jsonpointer-2.0-4.el9.noarch. 290 kB/s | 19 kB 00:00 2024-04-20T10:25:04.239 INFO:teuthology.orchestra.run.smithi047.stdout:(78/150): postgresql-server-13.14-1.el9.x86_64. 8.1 MB/s | 5.8 MB 00:00 2024-04-20T10:25:04.355 INFO:teuthology.orchestra.run.smithi047.stdout:(79/150): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-20T10:25:04.389 INFO:teuthology.orchestra.run.smithi047.stdout:(80/150): python3-mako-1.1.4-6.el9.noarch.rpm 688 kB/s | 172 kB 00:00 2024-04-20T10:25:04.414 INFO:teuthology.orchestra.run.smithi047.stdout:(81/150): python3-markupsafe-1.1.1-12.el9.x86_6 199 kB/s | 35 kB 00:00 2024-04-20T10:25:04.498 INFO:teuthology.orchestra.run.smithi047.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.3 MB/s | 250 kB 00:00 2024-04-20T10:25:04.640 INFO:teuthology.orchestra.run.smithi047.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 990 kB/s | 222 kB 00:00 2024-04-20T10:25:04.690 INFO:teuthology.orchestra.run.smithi047.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 404 kB/s | 77 kB 00:00 2024-04-20T10:25:04.715 INFO:teuthology.orchestra.run.smithi047.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 569 kB/s | 42 kB 00:00 2024-04-20T10:25:04.782 INFO:teuthology.orchestra.run.smithi047.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 2.9 MB/s | 269 kB 00:00 2024-04-20T10:25:04.816 INFO:teuthology.orchestra.run.smithi047.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-20T10:25:04.902 INFO:teuthology.orchestra.run.smithi047.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-04-20T10:25:04.925 INFO:teuthology.orchestra.run.smithi047.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 474 kB/s | 51 kB 00:00 2024-04-20T10:25:04.975 INFO:teuthology.orchestra.run.smithi047.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 750 kB/s | 54 kB 00:00 2024-04-20T10:25:05.100 INFO:teuthology.orchestra.run.smithi047.stdout:(91/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.9 MB/s | 5.1 MB 00:00 2024-04-20T10:25:05.150 INFO:teuthology.orchestra.run.smithi047.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 239 kB/s | 42 kB 00:00 2024-04-20T10:25:05.176 INFO:teuthology.orchestra.run.smithi047.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 596 kB/s | 44 kB 00:00 2024-04-20T10:25:05.234 INFO:teuthology.orchestra.run.smithi047.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-04-20T10:25:05.259 INFO:teuthology.orchestra.run.smithi047.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 434 kB/s | 36 kB 00:00 2024-04-20T10:25:05.310 INFO:teuthology.orchestra.run.smithi047.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 637 kB/s | 47 kB 00:00 2024-04-20T10:25:05.352 INFO:teuthology.orchestra.run.smithi047.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 3.3 MB/s | 305 kB 00:00 2024-04-20T10:25:05.402 INFO:teuthology.orchestra.run.smithi047.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 635 kB/s | 58 kB 00:00 2024-04-20T10:25:05.436 INFO:teuthology.orchestra.run.smithi047.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-20T10:25:05.486 INFO:teuthology.orchestra.run.smithi047.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 770 kB/s | 64 kB 00:00 2024-04-20T10:25:05.511 INFO:teuthology.orchestra.run.smithi047.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 603 kB/s | 45 kB 00:00 2024-04-20T10:25:05.895 INFO:teuthology.orchestra.run.smithi047.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 55 kB/s | 22 kB 00:00 2024-04-20T10:25:06.112 INFO:teuthology.orchestra.run.smithi047.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-04-20T10:25:06.178 INFO:teuthology.orchestra.run.smithi047.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 1.3 MB/s | 863 kB 00:00 2024-04-20T10:25:06.421 INFO:teuthology.orchestra.run.smithi047.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2024-04-20T10:25:06.679 INFO:teuthology.orchestra.run.smithi047.stdout:(106/150): abseil-cpp-20211102.0-3.el9.x86_64.r 699 kB/s | 548 kB 00:00 2024-04-20T10:25:06.721 INFO:teuthology.orchestra.run.smithi047.stdout:(107/150): gperftools-libs-2.9.1-2.el9.x86_64.r 508 kB/s | 309 kB 00:00 2024-04-20T10:25:06.780 INFO:teuthology.orchestra.run.smithi047.stdout:(108/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 261 kB/s | 26 kB 00:00 2024-04-20T10:25:06.813 INFO:teuthology.orchestra.run.smithi047.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-04-20T10:25:06.889 INFO:teuthology.orchestra.run.smithi047.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 624 kB/s | 67 kB 00:00 2024-04-20T10:25:06.922 INFO:teuthology.orchestra.run.smithi047.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-04-20T10:25:07.114 INFO:teuthology.orchestra.run.smithi047.stdout:(112/150): libarrow-9.0.0-10.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-04-20T10:25:07.148 INFO:teuthology.orchestra.run.smithi047.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 2.4 MB/s | 548 kB 00:00 2024-04-20T10:25:07.190 INFO:teuthology.orchestra.run.smithi047.stdout:(114/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.7 MB/s | 838 kB 00:00 2024-04-20T10:25:07.215 INFO:teuthology.orchestra.run.smithi047.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 436 kB/s | 43 kB 00:00 2024-04-20T10:25:07.257 INFO:teuthology.orchestra.run.smithi047.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 297 kB/s | 32 kB 00:00 2024-04-20T10:25:07.282 INFO:teuthology.orchestra.run.smithi047.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 164 kB/s | 15 kB 00:00 2024-04-20T10:25:07.316 INFO:teuthology.orchestra.run.smithi047.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-20T10:25:07.366 INFO:teuthology.orchestra.run.smithi047.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-20T10:25:07.399 INFO:teuthology.orchestra.run.smithi047.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-04-20T10:25:07.425 INFO:teuthology.orchestra.run.smithi047.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 326 kB/s | 35 kB 00:00 2024-04-20T10:25:07.467 INFO:teuthology.orchestra.run.smithi047.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-20T10:25:07.525 INFO:teuthology.orchestra.run.smithi047.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 1.4 MB/s | 144 kB 00:00 2024-04-20T10:25:07.559 INFO:teuthology.orchestra.run.smithi047.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 117 kB/s | 11 kB 00:00 2024-04-20T10:25:07.651 INFO:teuthology.orchestra.run.smithi047.stdout:(125/150): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-04-20T10:25:07.684 INFO:teuthology.orchestra.run.smithi047.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 112 kB/s | 18 kB 00:00 2024-04-20T10:25:07.709 INFO:teuthology.orchestra.run.smithi047.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 155 kB/s | 23 kB 00:00 2024-04-20T10:25:07.743 INFO:teuthology.orchestra.run.smithi047.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 213 kB/s | 19 kB 00:00 2024-04-20T10:25:07.802 INFO:teuthology.orchestra.run.smithi047.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 216 kB/s | 20 kB 00:00 2024-04-20T10:25:07.835 INFO:teuthology.orchestra.run.smithi047.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 507 kB/s | 46 kB 00:00 2024-04-20T10:25:07.877 INFO:teuthology.orchestra.run.smithi047.stdout:(131/150): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-20T10:25:07.986 INFO:teuthology.orchestra.run.smithi047.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 430 kB/s | 79 kB 00:00 2024-04-20T10:25:08.003 INFO:teuthology.orchestra.run.smithi047.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 346 kB/s | 58 kB 00:00 2024-04-20T10:25:08.053 INFO:teuthology.orchestra.run.smithi047.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-04-20T10:25:08.078 INFO:teuthology.orchestra.run.smithi047.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 180 kB/s | 16 kB 00:00 2024-04-20T10:25:08.104 INFO:teuthology.orchestra.run.smithi047.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 898 kB/s | 90 kB 00:00 2024-04-20T10:25:08.146 INFO:teuthology.orchestra.run.smithi047.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-20T10:25:08.171 INFO:teuthology.orchestra.run.smithi047.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 335 kB/s | 31 kB 00:00 2024-04-20T10:25:08.205 INFO:teuthology.orchestra.run.smithi047.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-20T10:25:08.247 INFO:teuthology.orchestra.run.smithi047.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 591 kB/s | 59 kB 00:00 2024-04-20T10:25:08.272 INFO:teuthology.orchestra.run.smithi047.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 358 kB/s | 36 kB 00:00 2024-04-20T10:25:08.305 INFO:teuthology.orchestra.run.smithi047.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 510 kB/s | 51 kB 00:00 2024-04-20T10:25:08.347 INFO:teuthology.orchestra.run.smithi047.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-04-20T10:25:08.373 INFO:teuthology.orchestra.run.smithi047.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 897 kB/s | 90 kB 00:00 2024-04-20T10:25:08.415 INFO:teuthology.orchestra.run.smithi047.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.9 MB/s | 427 kB 00:00 2024-04-20T10:25:08.473 INFO:teuthology.orchestra.run.smithi047.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2024-04-20T10:25:08.507 INFO:teuthology.orchestra.run.smithi047.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-04-20T10:25:08.841 INFO:teuthology.orchestra.run.smithi047.stdout:(148/150): librados2-19.0.0-3133.ga8c84f59.el9. 9.3 MB/s | 3.4 MB 00:00 2024-04-20T10:25:09.007 INFO:teuthology.orchestra.run.smithi047.stdout:(149/150): librbd1-19.0.0-3133.ga8c84f59.el9.x8 6.3 MB/s | 3.1 MB 00:00 2024-04-20T10:25:09.287 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-04-20T10:25:09.290 INFO:teuthology.orchestra.run.smithi047.stdout:(150/150): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-04-20T10:25:09.303 INFO:teuthology.orchestra.run.smithi047.stdout:-------------------------------------------------------------------------------- 2024-04-20T10:25:09.304 INFO:teuthology.orchestra.run.smithi047.stdout:Total 14 MB/s | 218 MB 00:15 2024-04-20T10:25:09.386 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:25:09.416 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:25:09.758 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:25:10.209 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:25:10.324 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:25:10.499 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:25:10.533 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:25:10.661 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:25:10.720 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:25:10.757 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:25:10.793 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:25:11.156 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:25:11.214 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:25:11.239 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:25:11.398 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:25:11.431 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:25:11.518 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:25:11.549 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:25:11.611 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:25:12.056 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:25:12.093 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction check 2024-04-20T10:25:12.107 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:25:12.299 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:25:12.323 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction check succeeded. 2024-04-20T10:25:12.323 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction test 2024-04-20T10:25:12.423 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:25:12.466 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:25:12.514 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:25:12.615 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:25:12.662 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:25:12.720 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:25:12.765 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:25:12.806 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:25:12.914 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:25:12.975 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:25:13.016 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:25:13.071 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:25:13.165 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:25:13.220 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:25:13.270 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:25:13.301 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:25:13.414 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:25:13.471 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:25:13.569 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:25:13.751 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:25:13.816 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:25:14.506 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:25:14.614 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:25:14.723 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:25:14.830 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:25:14.888 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:25:14.945 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:25:15.053 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:25:15.079 INFO:teuthology.orchestra.run.smithi047.stdout:Transaction test succeeded. 2024-04-20T10:25:15.080 INFO:teuthology.orchestra.run.smithi047.stdout:Running transaction 2024-04-20T10:25:15.144 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:25:15.191 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:25:15.228 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:25:15.296 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:25:15.338 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:25:15.398 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:25:15.457 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:25:15.658 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:25:15.711 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:25:15.748 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:25:15.804 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:25:15.935 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:25:15.980 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:25:16.055 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:25:16.097 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:25:16.130 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:25:16.336 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:25:16.517 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:25:16.561 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:25:16.601 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:25:16.670 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:25:16.813 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:25:16.884 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:25:16.943 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:25:16.990 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:25:17.061 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:25:17.112 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:25:17.189 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:25:17.241 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:25:17.360 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:25:17.420 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:25:17.464 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:25:17.525 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:25:17.676 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:25:17.756 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:25:18.653 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:25:18.724 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:25:18.764 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:25:18.893 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:25:18.987 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:25:19.100 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:25:19.156 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:25:19.382 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:25:19.419 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:25:19.535 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:25:20.375 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:25:20.525 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:25:21.833 INFO:teuthology.orchestra.run.smithi047.stdout: Preparing : 1/1 2024-04-20T10:25:21.972 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-20T10:25:22.154 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:25:22.250 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:25:22.273 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-20T10:25:22.292 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:25:22.329 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:25:22.582 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-20T10:25:22.718 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:25:22.788 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:25:22.915 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:25:22.953 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:25:23.011 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:25:23.042 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-20T10:25:23.165 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:25:23.353 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 5/152 2024-04-20T10:25:23.379 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:25:23.512 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:25:23.560 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:25:23.598 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:25:23.690 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:25:23.725 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:25:23.725 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:25:23.763 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-20T10:25:23.796 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-20T10:25:24.234 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-20T10:25:24.312 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-20T10:25:24.342 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:25:24.515 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:25:24.552 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:25:24.650 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 13/152 2024-04-20T10:25:24.678 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:25:24.742 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-20T10:25:25.225 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:25:25.305 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-20T10:25:25.509 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-20T10:25:25.634 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-20T10:25:25.771 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-20T10:25:25.832 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-20T10:25:25.965 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-20T10:25:26.018 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-20T10:25:26.102 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-20T10:25:26.154 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-20T10:25:26.198 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-20T10:25:26.312 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-20T10:25:26.388 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-20T10:25:26.388 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:25:26.437 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:25:26.448 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-20T10:25:26.536 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-20T10:25:26.671 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-20T10:25:26.710 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 30/152 2024-04-20T10:25:26.766 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 31/152 2024-04-20T10:25:26.800 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 32/152 2024-04-20T10:25:26.947 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-20T10:25:27.012 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-20T10:25:27.093 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:25:27.125 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-20T10:25:27.274 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:25:27.304 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-20T10:25:27.388 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-20T10:25:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:25:27.433 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:25:27.433 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:25:27.433 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:27.483 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:25:28.151 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-20T10:25:28.274 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-20T10:25:28.314 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-20T10:25:28.359 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-20T10:25:28.425 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 42/152 2024-04-20T10:25:28.469 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-20T10:25:28.553 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-20T10:25:28.659 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-20T10:25:28.698 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-20T10:25:28.738 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-20T10:25:28.797 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-20T10:25:28.844 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-20T10:25:28.906 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-20T10:25:28.957 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-20T10:25:29.156 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-20T10:25:29.235 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-20T10:25:29.272 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-20T10:25:29.330 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-20T10:25:29.473 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-20T10:25:29.521 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-20T10:25:29.594 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-20T10:25:29.638 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-20T10:25:29.670 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-20T10:25:29.880 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-20T10:25:30.063 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-20T10:25:30.109 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-20T10:25:30.148 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-20T10:25:30.196 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-20T10:25:30.279 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-20T10:25:30.343 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-20T10:25:30.422 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-20T10:25:30.473 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-20T10:25:30.528 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-20T10:25:30.578 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-20T10:25:30.655 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-20T10:25:30.701 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-20T10:25:30.851 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-20T10:25:30.913 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-20T10:25:30.957 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-20T10:25:31.029 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-20T10:25:31.190 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-20T10:25:31.263 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-20T10:25:32.280 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-20T10:25:32.361 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-20T10:25:32.395 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-20T10:25:32.529 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-20T10:25:32.590 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-20T10:25:32.698 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-20T10:25:32.754 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-20T10:25:33.000 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-20T10:25:33.025 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-20T10:25:33.131 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-20T10:25:34.031 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-20T10:25:34.178 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-20T10:25:35.919 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-20T10:25:36.031 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-20T10:25:36.075 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-20T10:25:36.118 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-20T10:25:36.521 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-20T10:25:36.569 INFO:teuthology.orchestra.run.smithi047.stdout: Upgrading : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:25:36.727 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 97/152 2024-04-20T10:25:36.760 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 98/152 2024-04-20T10:25:36.808 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-20T10:25:37.377 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-20T10:25:37.413 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:25:37.514 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 101/152 2024-04-20T10:25:37.540 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 102/152 2024-04-20T10:25:40.091 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-04-20T10:25:40.092 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:40.328 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:25:40.363 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:25:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:25:40.375 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:40.375 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:25:40.375 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:40.403 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:25:41.079 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 103/152 2024-04-20T10:25:41.123 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:25:41.313 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 104/152 2024-04-20T10:25:41.313 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-20T10:25:41.314 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-20T10:25:41.314 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:25:41.357 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:25:41.996 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:25:42.089 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:25:42.089 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:42.089 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:25:42.089 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:25:42.089 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:25:42.089 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:42.338 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:25:42.404 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:25:42.404 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:42.405 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:25:42.405 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:25:42.405 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:25:42.405 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:42.888 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:25:42.962 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:25:42.963 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:42.963 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:25:42.963 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:25:42.963 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:25:42.963 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:43.106 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:25:43.170 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:25:43.256 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:25:43.508 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:25:43.568 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:25:43.657 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:25:43.777 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:25:43.960 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:25:44.012 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:25:44.050 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:25:44.082 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:25:44.728 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:25:44.768 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:25:44.824 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:25:44.930 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:25:44.962 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:25:46.276 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:25:46.322 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:25:47.415 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:25:47.439 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:25:47.581 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:25:47.781 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:25:47.807 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:25:47.884 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:25:47.884 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:47.884 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:25:47.884 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:25:47.884 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:25:47.885 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:47.950 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:25:47.989 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:25:48.135 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:25:48.213 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:25:49.291 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:25:49.327 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:25:49.394 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:25:49.394 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:49.394 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:25:49.394 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:25:49.394 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:25:49.394 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:49.444 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:25:49.534 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:25:49.534 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:49.534 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:25:49.534 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:49.603 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:25:49.677 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:25:49.677 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:49.678 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:25:49.678 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:25:49.678 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:25:49.678 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:50.037 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:25:50.109 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:25:50.109 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:50.109 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:25:50.109 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:25:50.110 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:25:50.110 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:55.006 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 105/152 2024-04-20T10:25:55.006 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /sys 2024-04-20T10:25:55.006 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /proc 2024-04-20T10:25:55.006 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /mnt 2024-04-20T10:25:55.006 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /var/tmp 2024-04-20T10:25:55.006 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /home 2024-04-20T10:25:55.007 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /root 2024-04-20T10:25:55.007 INFO:teuthology.orchestra.run.smithi047.stdout:skipping the directory /tmp 2024-04-20T10:25:55.007 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:25:55.155 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:25:55.211 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:25:55.253 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:25:55.293 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:25:55.307 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 106/152 2024-04-20T10:25:55.307 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:55.307 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-20T10:25:55.307 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:25:55.339 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:25:55.387 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:25:55.692 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:25:55.766 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:25:55.793 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:25:56.683 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:25:56.733 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:25:56.788 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:25:56.989 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:25:57.071 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 107/152 2024-04-20T10:25:57.071 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:57.072 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-20T10:25:57.072 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:25:57.072 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-20T10:25:57.072 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:25:57.331 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:25:57.408 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 108/152 2024-04-20T10:25:57.409 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:57.409 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-20T10:25:57.409 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:25:57.409 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-20T10:25:57.409 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:25:57.948 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:25:58.022 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 109/152 2024-04-20T10:25:58.022 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:25:58.022 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-20T10:25:58.023 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:25:58.023 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-20T10:25:58.023 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:25:58.045 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:25:58.147 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:25:58.158 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-20T10:25:58.222 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-20T10:25:58.312 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-20T10:25:58.507 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:25:58.569 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-20T10:25:58.628 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-20T10:25:58.715 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-20T10:25:58.827 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-20T10:25:58.907 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:25:58.961 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:25:58.961 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:25:59.005 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-20T10:25:59.053 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-20T10:25:59.086 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:25:59.087 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:25:59.124 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-20T10:25:59.158 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-20T10:25:59.575 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:25:59.575 INFO:teuthology.orchestra.run.smithi175.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T10:25:59.576 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:25:59.813 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:25:59.854 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 121/152 2024-04-20T10:25:59.933 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 122/152 2024-04-20T10:26:00.055 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 123/152 2024-04-20T10:26:00.105 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:26:01.476 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 124/152 2024-04-20T10:26:01.524 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:26:02.689 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 125/152 2024-04-20T10:26:02.724 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:26:02.878 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 126/152 2024-04-20T10:26:03.077 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 127/152 2024-04-20T10:26:03.101 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:26:03.178 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 128/152 2024-04-20T10:26:03.179 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:26:03.179 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-20T10:26:03.179 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:26:03.179 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-20T10:26:03.179 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:03.243 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:26:03.285 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 129/152 2024-04-20T10:26:03.422 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 130/152 2024-04-20T10:26:03.496 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 131/152 2024-04-20T10:26:04.622 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-20T10:26:04.654 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:26:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 133/152 2024-04-20T10:26:04.743 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:26:04.743 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-20T10:26:04.743 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:26:04.743 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-20T10:26:04.743 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:04.796 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:26:04.861 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:26:04.862 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:26:04.863 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:26:04.875 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:26:04.876 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:26:04.877 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:26:04.878 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:26:04.879 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:26:04.882 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:26:04.882 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:26:04.882 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:26:04.882 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:26:04.882 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:26:04.883 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:26:04.884 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:26:04.885 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:26:04.886 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:26:04.887 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:26:04.888 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:26:04.888 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:26:04.888 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:26:04.888 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:26:04.888 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:26:04.894 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 134/152 2024-04-20T10:26:04.894 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:26:04.894 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-20T10:26:04.895 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:04.973 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:26:05.081 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 135/152 2024-04-20T10:26:05.081 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:26:05.081 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-20T10:26:05.081 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:26:05.082 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-20T10:26:05.082 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:05.481 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:26:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 136/152 2024-04-20T10:26:05.563 INFO:teuthology.orchestra.run.smithi047.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-20T10:26:05.563 INFO:teuthology.orchestra.run.smithi047.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-20T10:26:05.563 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:26:05.563 INFO:teuthology.orchestra.run.smithi047.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-20T10:26:05.563 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:07.214 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:26:07.214 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:07.214 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-04-20T10:26:07.214 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.215 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.216 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:26:07.217 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:26:07.218 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:26:07.219 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:26:07.220 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:26:07.221 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:26:07.222 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:26:07.223 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:26:07.224 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:26:07.225 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:26:07.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:07.226 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-04-20T10:26:07.800 DEBUG:teuthology.parallel:result is None 2024-04-20T10:26:10.931 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 137/152 2024-04-20T10:26:10.987 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 138/152 2024-04-20T10:26:11.063 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 139/152 2024-04-20T10:26:11.107 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-20T10:26:11.156 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-20T10:26:11.484 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 142/152 2024-04-20T10:26:11.533 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-20T10:26:11.561 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 144/152 2024-04-20T10:26:12.538 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:26:12.584 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:26:12.641 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-20T10:26:13.970 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 146/152 2024-04-20T10:26:14.115 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-20T10:26:14.199 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-20T10:26:14.378 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-20T10:26:14.459 INFO:teuthology.orchestra.run.smithi047.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-20T10:26:14.459 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:26:14.597 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-20T10:26:14.597 INFO:teuthology.orchestra.run.smithi047.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:26:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:26:15.114 INFO:teuthology.orchestra.run.smithi047.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-20T10:26:15.114 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 1/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 3/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 4/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3133.ga8c84 5/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 6/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 7/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 8/152 2024-04-20T10:26:20.737 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 9/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 10/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 11/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 12/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 13/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 14/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_ 15/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 16/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 17/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_6 18/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86 19/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 20/152 2024-04-20T10:26:20.738 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el 21/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9. 22/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_6 23/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 24/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 25/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 26/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 27/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 28/152 2024-04-20T10:26:20.739 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 29/152 2024-04-20T10:26:20.740 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59. 30/152 2024-04-20T10:26:20.740 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noa 31/152 2024-04-20T10:26:20.740 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.n 32/152 2024-04-20T10:26:20.740 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c 33/152 2024-04-20T10:26:20.740 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el 34/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 35/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.e 36/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 37/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 38/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 39/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-20T10:26:20.741 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-20T10:26:20.742 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-20T10:26:20.743 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-20T10:26:20.744 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-20T10:26:20.745 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-20T10:26:20.746 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-20T10:26:20.747 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-20T10:26:20.748 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-20T10:26:20.749 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-20T10:26:20.750 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-20T10:26:20.750 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-20T10:26:20.750 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-20T10:26:20.750 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-20T10:26:20.750 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 149/152 2024-04-20T10:26:20.750 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-20T10:26:20.750 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 151/152 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout:Upgraded: 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: librados2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: librbd1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout:Installed: 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: automake-1.16.2-8.el9.noarch 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: bison-3.7.4-5.el9.x86_64 2024-04-20T10:26:23.271 INFO:teuthology.orchestra.run.smithi047.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-base-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-grafana-dashboards-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-immutable-object-cache-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mds-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-dashboard-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-modules-core-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.272 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mgr-rook-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-mon-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-osd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-prometheus-alerts-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-radosgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-selinux-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-test-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: ceph-volume-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: cephadm-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-shell-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: cephfs-top-2:19.0.0-3133.ga8c84f59.el9.noarch 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: flex-2.6.4-9.el9.x86_64 2024-04-20T10:26:23.273 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: libcephfs2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: libcephsqlite-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.274 INFO:teuthology.orchestra.run.smithi047.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: librados-devel-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: libradosstriper1-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: librgw2-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: lua-5.4.4-4.el9.x86_64 2024-04-20T10:26:23.275 INFO:teuthology.orchestra.run.smithi047.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-13.14-1.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-20T10:26:23.276 INFO:teuthology.orchestra.run.smithi047.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-argparse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ceph-common-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cephfs-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-20T10:26:23.277 INFO:teuthology.orchestra.run.smithi047.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-20T10:26:23.278 INFO:teuthology.orchestra.run.smithi047.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-ply-3.11-14.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-20T10:26:23.279 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rados-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rgw-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-20T10:26:23.280 INFO:teuthology.orchestra.run.smithi047.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-fuse-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-mirror-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: rbd-nbd-2:19.0.0-3133.ga8c84f59.el9.x86_64 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: re2-1:20211101-3.el9.x86_64 2024-04-20T10:26:23.281 INFO:teuthology.orchestra.run.smithi047.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-20T10:26:23.282 INFO:teuthology.orchestra.run.smithi047.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-20T10:26:23.282 INFO:teuthology.orchestra.run.smithi047.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-20T10:26:23.282 INFO:teuthology.orchestra.run.smithi047.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-20T10:26:23.282 INFO:teuthology.orchestra.run.smithi047.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-20T10:26:23.282 INFO:teuthology.orchestra.run.smithi047.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-20T10:26:23.282 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:23.282 INFO:teuthology.orchestra.run.smithi047.stdout:Complete! 2024-04-20T10:26:23.858 DEBUG:teuthology.parallel:result is None 2024-04-20T10:26:23.858 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:26:24.085 DEBUG:teuthology.orchestra.run.smithi047:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:26:24.141 INFO:teuthology.orchestra.run.smithi047.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:26:24.141 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:26:24.141 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:26:24.143 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a8c84f5997950fd7d008204839253291d52e6eda 2024-04-20T10:26:24.305 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-20T10:26:24.379 INFO:teuthology.orchestra.run.smithi175.stdout:19.0.0-3133.ga8c84f59.el9 2024-04-20T10:26:24.379 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3133.ga8c84f59.el9 2024-04-20T10:26:24.379 INFO:teuthology.task.install:The correct ceph version 19.0.0-3133.ga8c84f59 is installed. 2024-04-20T10:26:24.381 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-20T10:26:24.381 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:24.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:26:24.450 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:24.450 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-20T10:26:24.543 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-20T10:26:24.543 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:24.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:26:24.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:26:24.717 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:24.717 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-04-20T10:26:24.806 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-20T10:26:24.891 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-20T10:26:24.891 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:24.891 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:26:24.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:26:25.062 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:25.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-20T10:26:25.150 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-20T10:26:25.268 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-20T10:26:25.269 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:25.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:26:25.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:26:25.436 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:25.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-04-20T10:26:25.524 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-20T10:26:25.606 INFO:teuthology.run_tasks:Running task ceph... 2024-04-20T10:26:25.690 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-20T10:26:25.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 777 /var/log/ceph 2024-04-20T10:26:25.695 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2024-04-20T10:26:25.782 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-20T10:26:25.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T10:26:25.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-20T10:26:25.871 INFO:tasks.ceph:Creating extra log directories... 2024-04-20T10:26:25.871 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T10:26:25.875 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-20T10:26:25.968 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-20T10:26:25.968 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-20T10:26:25.968 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-20T10:26:25.968 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:25.968 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T10:26:26.005 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:26.179 DEBUG:teuthology.orchestra.run.smithi047:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:26.238 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:26.238 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-20T10:26:26.295 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:26.525 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:26.619 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-20T10:26:26.619 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-20T10:26:26.619 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463', 'branch': 'wip-rishabh-testing-20240416.193735', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair}', 'email': None, 'first_in_suite': False, 'job_id': '7664463', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'mgrmodules': {'sequential': [{'print': 'Enabling mgr modules'}]}, 'name': 'rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rishabh-testing-20240416.193735'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'subvolume_version': 2, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-rishabh-testing-20240416.193735', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7954, 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'sleep_before_teardown': 0, 'subset': '364/512', 'suite': 'fs', 'suite_branch': 'wip-rishabh-testing-20240416.193735', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_a8c84f5997950fd7d008204839253291d52e6eda/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi047.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGtvGgOa4z63kz9+Cg0/az2X1LuKHDEPB4/vjsPPwWlwfYctZSn3+flXdwjk5TiwAa+oHTBslvdwFxZ0k70J8e0=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPz7C7ZIChXoqlSACubGmD165Bl6znw68xZaLIvEd6qgX1SR1vQe5NsrzYeEQUvJyakvWvBUIyCLRT803TXwkB4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'force file system read-only', 'bad backtrace', 'MDS in read-only mode', '\\(MDS_READ_ONLY\\)'], 'sha1': 'a8c84f5997950fd7d008204839253291d52e6eda', 'cluster': 'ceph'}}, {'sequential': ['mgrmodules']}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_auto_repair']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-20_09:50:15', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-20T10:26:26.620 DEBUG:teuthology.orchestra.run.smithi047:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T10:26:26.625 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-20T10:26:26.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T10:26:26.707 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2024-04-20T10:26:26.797 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:26.797 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:26:26.837 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:26:26.837 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_1 2024-04-20T10:26:26.924 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:26:26.924 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:26.924 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 636 Links: 1 2024-04-20T10:26:26.924 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:26.925 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:26.925 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 10:26:15.588841424 +0000 2024-04-20T10:26:26.925 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 10:23:52.195312534 +0000 2024-04-20T10:26:26.925 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 10:23:52.195312534 +0000 2024-04-20T10:26:26.925 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-20 10:23:52.195312534 +0000 2024-04-20T10:26:26.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:26:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T10:26:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T10:26:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000229932 s, 2.2 MB/s 2024-04-20T10:26:27.023 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:26:27.061 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_2 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 655 Links: 1 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 10:26:15.588841424 +0000 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 10:23:52.811299769 +0000 2024-04-20T10:26:27.141 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 10:23:52.811299769 +0000 2024-04-20T10:26:27.142 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-20 10:23:52.811299769 +0000 2024-04-20T10:26:27.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:26:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T10:26:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T10:26:27.241 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000185853 s, 2.8 MB/s 2024-04-20T10:26:27.244 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:26:27.285 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_3 2024-04-20T10:26:27.363 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:26:27.363 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:27.364 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 673 Links: 1 2024-04-20T10:26:27.364 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:27.364 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:27.364 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 10:26:15.589841403 +0000 2024-04-20T10:26:27.364 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 10:23:53.415287251 +0000 2024-04-20T10:26:27.364 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 10:23:53.415287251 +0000 2024-04-20T10:26:27.364 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-20 10:23:53.415287251 +0000 2024-04-20T10:26:27.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:26:27.463 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T10:26:27.463 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T10:26:27.463 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000187956 s, 2.7 MB/s 2024-04-20T10:26:27.467 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:26:27.508 DEBUG:teuthology.orchestra.run.smithi047:> stat /dev/vg_nvme/lv_4 2024-04-20T10:26:27.587 INFO:teuthology.orchestra.run.smithi047.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout:Device: 6h/6d Inode: 692 Links: 1 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout:Access: 2024-04-20 10:26:15.590841382 +0000 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout:Modify: 2024-04-20 10:23:54.051274072 +0000 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout:Change: 2024-04-20 10:23:54.051274072 +0000 2024-04-20T10:26:27.588 INFO:teuthology.orchestra.run.smithi047.stdout: Birth: 2024-04-20 10:23:54.051274072 +0000 2024-04-20T10:26:27.588 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:26:27.698 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records in 2024-04-20T10:26:27.698 INFO:teuthology.orchestra.run.smithi047.stderr:1+0 records out 2024-04-20T10:26:27.698 INFO:teuthology.orchestra.run.smithi047.stderr:512 bytes copied, 0.000256526 s, 2.0 MB/s 2024-04-20T10:26:27.701 DEBUG:teuthology.orchestra.run.smithi047:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:26:27.752 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:27.752 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:27.752 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-04-20T10:26:27.809 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-20T10:26:27.810 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 634 Links: 1 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 10:26:00.019200015 +0000 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 10:23:47.908862055 +0000 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 10:23:47.908862055 +0000 2024-04-20T10:26:27.863 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 10:23:47.908862055 +0000 2024-04-20T10:26:27.863 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-20T10:26:27.941 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T10:26:27.941 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T10:26:27.941 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.0004035 s, 1.3 MB/s 2024-04-20T10:26:27.944 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-20T10:26:27.999 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 10:26:00.020199999 +0000 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 10:23:48.524852465 +0000 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 10:23:48.524852465 +0000 2024-04-20T10:26:28.049 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 10:23:48.524852465 +0000 2024-04-20T10:26:28.049 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-20T10:26:28.120 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T10:26:28.120 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T10:26:28.120 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000341381 s, 1.5 MB/s 2024-04-20T10:26:28.123 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-20T10:26:28.176 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-04-20T10:26:28.226 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 10:26:00.021199984 +0000 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 10:23:49.138842906 +0000 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 10:23:49.138842906 +0000 2024-04-20T10:26:28.227 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 10:23:49.138842906 +0000 2024-04-20T10:26:28.227 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-20T10:26:28.307 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T10:26:28.307 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T10:26:28.307 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000342037 s, 1.5 MB/s 2024-04-20T10:26:28.311 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-20T10:26:28.368 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-04-20T10:26:28.417 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 688 Links: 1 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-04-20 10:26:00.021199984 +0000 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-04-20 10:23:49.735833611 +0000 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-04-20 10:23:49.735833611 +0000 2024-04-20T10:26:28.418 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-04-20 10:23:49.735833611 +0000 2024-04-20T10:26:28.418 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-20T10:26:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-04-20T10:26:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-04-20T10:26:28.496 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000348195 s, 1.5 MB/s 2024-04-20T10:26:28.500 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-20T10:26:28.557 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:28.557 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T10:26:28.557 INFO:tasks.ceph:Generating config... 2024-04-20T10:26:28.558 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-20T10:26:28.558 INFO:tasks.ceph:[client] debug client = 20 2024-04-20T10:26:28.558 INFO:tasks.ceph:[client] debug ms = 1 2024-04-20T10:26:28.558 INFO:tasks.ceph:[client] pid file = 2024-04-20T10:26:28.558 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-20T10:26:28.559 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mgr] debug client = 20 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-20T10:26:28.559 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-20T10:26:28.560 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-04-20T10:26:28.560 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-20T10:26:28.560 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-20T10:26:28.560 INFO:tasks.ceph:Setting up mon.a... 2024-04-20T10:26:28.561 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-20T10:26:28.668 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.keyring 2024-04-20T10:26:28.674 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-20T10:26:28.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:26:28.825 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.47'), ('mon.b', '172.21.15.175'), ('mon.c', '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]')] 2024-04-20T10:26:28.825 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-20T10:26:28.826 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:28.826 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T10:26:28.864 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.47 --add b 172.21.15.175 --addv c '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: generated fsid 775946e6-93c9-4843-aaf4-db758801eec1 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:setting min_mon_release = quincy 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:epoch 0 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:fsid 775946e6-93c9-4843-aaf4-db758801eec1 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:last_changed 2024-04-20T10:26:28.996312+0000 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:created 2024-04-20T10:26:28.996312+0000 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:min_mon_release 17 (quincy) 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:election_strategy: 1 2024-04-20T10:26:28.995 INFO:teuthology.orchestra.run.smithi047.stdout:0: [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] mon.a 2024-04-20T10:26:28.996 INFO:teuthology.orchestra.run.smithi047.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-04-20T10:26:28.996 INFO:teuthology.orchestra.run.smithi047.stdout:2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-04-20T10:26:28.996 INFO:teuthology.orchestra.run.smithi047.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-20T10:26:28.998 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-20T10:26:29.035 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 775946e6-93c9-4843-aaf4-db758801eec1... 2024-04-20T10:26:29.036 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T10:26:29.080 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-20T10:26:29.192 INFO:teuthology.orchestra.run.smithi047.stdout:[global] 2024-04-20T10:26:29.192 INFO:teuthology.orchestra.run.smithi047.stdout: chdir = "" 2024-04-20T10:26:29.192 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T10:26:29.192 INFO:teuthology.orchestra.run.smithi047.stdout: auth supported = cephx 2024-04-20T10:26:29.192 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: filestore xattr use omap = true 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: mon clock drift allowed = 1.000 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: osd crush chooseleaf type = 0 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: auth debug = true 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on old message = true 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: ms die on bug = true 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: mon pg warn max object skew = 0 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T10:26:29.193 INFO:teuthology.orchestra.run.smithi047.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default size = 2 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow primary affinity = true 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd allow pg remap = true 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on legacy crush tunables = false 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on crush straw calc version zero = false 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on no sortbitwise = false 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on osd down out interval zero = false 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon warn on too few osds = false 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: mon_allow_pool_size_one = true 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.194 INFO:teuthology.orchestra.run.smithi047.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: osd default data pool replay window = 5 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: mon allow pool delete = true 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: mon cluster log file level = debug 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: debug asserts on shutdown = true 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: mon health detail to clog = false 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: fsid = 775946e6-93c9-4843-aaf4-db758801eec1 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout:[osd] 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: osd journal size = 100 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.195 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub load threshold = 5.0 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd scrub max interval = 600 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd mclock profile = high_recovery_ops 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd recover clone overlap = true 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd recovery max chunk = 1048576 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug shutdown = true 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug op order = true 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug verify stray on activate = true 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug trim objects = true 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd open classes on start = true 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug pg log writeout = true 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: osd deep scrub update digest min age = 30 2024-04-20T10:26:29.196 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: osd map max advance = 10 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: journal zero on create = true 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: filestore ondisk finisher threads = 3 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: filestore apply finisher threads = 3 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: bdev debug aio = true 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: osd debug misdirected ops = true 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore block size = 96636764160 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: bluestore fsck on mount = True 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluefs = 20 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug bluestore = 20 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T10:26:29.197 INFO:teuthology.orchestra.run.smithi047.stdout: debug osd = 20 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: debug rocksdb = 10 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd full ratio = 0.9 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd nearfull ratio = 0.8 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: osd failsafe full ratio = 0.95 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: osd objectstore = bluestore 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr] 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: debug mgr = 20 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:26:29.198 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: mgr/telemetry/nag = false 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout:[mon] 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug mon = 20 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug paxos = 20 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: debug auth = 20 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: mon data avail warn = 5 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: mon mgr mkfs grace = 240 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd reporter subtree level = osd 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: mon osd prime pg temp = true 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:26:29.199 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: # 1m isn't quite enough 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon_down_mkfs_grace = 2m 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon_warn_on_filestore_osds = false 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon down mkfs grace = 300 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: mon op complaint time = 120 2024-04-20T10:26:29.200 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout:[client] 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: rgw cache enabled = true 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable ops log = true 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: rgw enable usage log = true 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: client mount timeout = 600 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: debug client = 20 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: pid file = "" 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.a] 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.b] 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout:[mon.c] 2024-04-20T10:26:29.201 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout:[mds] 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds = 20 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: debug mds balancer = 20 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: debug ms = 1 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug frag = True 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: mds debug scatterstat = True 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: mds op complaint time = 180 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: mds verify scatter = True 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: osd op complaint time = 180 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: rados mon op timeout = 900 2024-04-20T10:26:29.202 INFO:teuthology.orchestra.run.smithi047.stdout: rados osd op timeout = 900 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-04-20T10:26:29.224 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-04-20T10:26:29.225 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.47,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = 775946e6-93c9-4843-aaf4-db758801eec1 2024-04-20T10:26:29.226 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug trim objects = true 2024-04-20T10:26:29.227 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-04-20T10:26:29.228 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 20 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 20 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 10 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-04-20T10:26:29.229 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-04-20T10:26:29.230 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: mon down mkfs grace = 300 2024-04-20T10:26:29.231 INFO:teuthology.orchestra.run.smithi175.stdout: mon op complaint time = 120 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: client mount timeout = 600 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-04-20T10:26:29.232 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout:[mds] 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds = 20 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: debug mds balancer = 20 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug frag = True 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: mds debug scatterstat = True 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: mds op complaint time = 180 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: mds verify scatter = True 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: osd op complaint time = 180 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: rados mon op timeout = 900 2024-04-20T10:26:29.233 INFO:teuthology.orchestra.run.smithi175.stdout: rados osd op timeout = 900 2024-04-20T10:26:29.258 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-20T10:26:29.258 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-20T10:26:29.368 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-20T10:26:29.368 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:29.368 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-20T10:26:29.412 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:29.412 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-20T10:26:29.495 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:26:29.496 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:29.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T10:26:29.496 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:26:29.623 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:26:29.624 DEBUG:teuthology.orchestra.run.smithi047:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:29.704 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:26:29.704 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:29.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-20T10:26:29.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-20T10:26:29.813 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:26:29.813 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:29.908 INFO:tasks.ceph:Setting up mon nodes... 2024-04-20T10:26:29.909 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-20T10:26:29.909 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T10:26:30.060 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-20T10:26:30.067 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T10:26:30.230 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-20T10:26:30.241 INFO:tasks.ceph:Setting up mds nodes... 2024-04-20T10:26:30.241 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T10:26:30.407 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-20T10:26:30.415 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-20T10:26:30.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T10:26:30.654 INFO:teuthology.orchestra.run.smithi047.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-20T10:26:30.664 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-20T10:26:30.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T10:26:30.900 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-20T10:26:30.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-20T10:26:30.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T10:26:31.139 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-20T10:26:31.146 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-20T10:26:31.229 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-20T10:26:31.229 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-20T10:26:31.356 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-20T10:26:31.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-20T10:26:31.508 INFO:teuthology.orchestra.run.smithi047.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-20T10:26:31.544 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-20T10:26:31.666 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-20T10:26:31.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-20T10:26:31.833 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-20T10:26:31.870 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-20T10:26:31.870 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi047.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-20T10:26:31.870 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-20T10:26:31.954 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:31.954 INFO:tasks.ceph:role: osd.0 2024-04-20T10:26:31.954 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:26:31.954 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:32.026 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:32.061 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-20T10:26:32.067 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-20T10:26:32.067 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-20T10:26:32.174 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-20T10:26:32.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-20T10:26:32.321 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:32.321 INFO:tasks.ceph:role: osd.1 2024-04-20T10:26:32.321 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:26:32.321 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:32.394 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:32.395 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:32.395 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:32.428 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-20T10:26:32.433 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-20T10:26:32.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-20T10:26:32.536 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-20T10:26:32.608 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-20T10:26:32.674 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:32.674 INFO:tasks.ceph:role: osd.2 2024-04-20T10:26:32.674 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:26:32.674 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:32.744 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:32.777 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-20T10:26:32.783 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-20T10:26:32.783 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-20T10:26:32.890 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-20T10:26:32.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-20T10:26:33.055 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:33.055 INFO:tasks.ceph:role: osd.3 2024-04-20T10:26:33.055 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:26:33.056 DEBUG:teuthology.orchestra.run.smithi047:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:33.142 INFO:teuthology.orchestra.run.smithi047.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:33.176 INFO:teuthology.orchestra.run.smithi047.stdout:Discarding blocks...Done. 2024-04-20T10:26:33.181 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi047.front.sepia.ceph.com -o noatime 2024-04-20T10:26:33.181 DEBUG:teuthology.orchestra.run.smithi047:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-20T10:26:33.293 DEBUG:teuthology.orchestra.run.smithi047:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-20T10:26:33.372 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:33.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:33.555+0000 7f90eb58e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-20T10:26:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:33.556+0000 7f90eb58e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-20T10:26:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:33.556+0000 7f90eb58e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:26:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:33.556+0000 7f90eb58e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:26:33.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:33.556+0000 7f90eb58e640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-20T10:26:33.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:33.560+0000 7f90eb58e640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-20T10:26:36.569 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-20T10:26:36.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:36.794+0000 7eff3bc1d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-20T10:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:36.795+0000 7eff3bc1d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-20T10:26:36.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:36.795+0000 7eff3bc1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:26:36.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:36.795+0000 7eff3bc1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:26:36.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:36.795+0000 7eff3bc1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-20T10:26:36.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:36.797+0000 7eff3bc1d640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-20T10:26:39.750 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-20T10:26:39.833 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:39.968+0000 7f4c3dc07640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-20T10:26:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:39.969+0000 7f4c3dc07640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-20T10:26:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:39.969+0000 7f4c3dc07640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:26:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:39.969+0000 7f4c3dc07640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:26:39.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:39.969+0000 7f4c3dc07640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-20T10:26:39.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:39.971+0000 7f4c3dc07640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-20T10:26:42.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-20T10:26:43.038 DEBUG:teuthology.orchestra.run.smithi047:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:43.174+0000 7fefcde0f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-20T10:26:43.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:43.175+0000 7fefcde0f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-20T10:26:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:43.175+0000 7fefcde0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:26:43.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:43.175+0000 7fefcde0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:26:43.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:43.175+0000 7fefcde0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-20T10:26:43.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:26:43.177+0000 7fefcde0f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-20T10:26:46.158 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-20T10:26:46.249 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-20T10:26:46.339 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:46.339 INFO:tasks.ceph:role: osd.4 2024-04-20T10:26:46.339 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:26:46.339 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-20T10:26:46.429 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:46.429 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:46.429 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:46.429 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:46.429 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:46.429 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:46.430 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:46.430 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:46.430 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:46.430 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:46.846 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T10:26:46.853 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T10:26:46.853 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-20T10:26:47.022 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-20T10:26:47.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-20T10:26:47.186 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:47.186 INFO:tasks.ceph:role: osd.5 2024-04-20T10:26:47.186 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:26:47.186 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:47.259 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:47.260 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:47.260 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:47.612 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T10:26:47.616 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T10:26:47.616 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-20T10:26:47.723 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-20T10:26:47.814 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-20T10:26:47.883 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:47.883 INFO:tasks.ceph:role: osd.6 2024-04-20T10:26:47.883 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:26:47.883 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-20T10:26:47.959 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:47.959 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:47.959 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:47.959 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:47.960 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:47.960 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:47.960 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:47.960 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:47.960 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:47.960 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:48.348 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T10:26:48.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T10:26:48.354 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-20T10:26:48.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-20T10:26:48.551 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-20T10:26:48.615 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-20T10:26:48.615 INFO:tasks.ceph:role: osd.7 2024-04-20T10:26:48.615 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:26:48.615 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-20T10:26:48.691 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-20T10:26:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-20T10:26:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-20T10:26:48.691 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-20T10:26:48.692 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-20T10:26:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-04-20T10:26:48.692 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-20T10:26:48.692 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-20T10:26:48.692 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-20T10:26:48.692 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-20T10:26:49.110 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-04-20T10:26:49.116 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-04-20T10:26:49.116 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-20T10:26:49.273 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-20T10:26:49.364 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:49.504 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:49.505+0000 7f5db49b8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-20T10:26:49.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:49.505+0000 7f5db49b8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-20T10:26:49.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:49.505+0000 7f5db49b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:26:49.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:49.505+0000 7f5db49b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:26:49.505 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:49.505+0000 7f5db49b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-20T10:26:49.508 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:49.509+0000 7f5db49b8640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-20T10:26:52.751 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-20T10:26:52.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:52.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:52.963+0000 7f041280f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-20T10:26:52.963 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:52.963+0000 7f041280f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-20T10:26:52.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:52.964+0000 7f041280f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:26:52.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:52.964+0000 7f041280f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:26:52.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:52.964+0000 7f041280f640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-20T10:26:52.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:52.966+0000 7f041280f640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-20T10:26:55.954 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-20T10:26:56.036 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:56.166 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:56.166+0000 7fea73a11640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-20T10:26:56.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:56.167+0000 7fea73a11640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-20T10:26:56.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:56.167+0000 7fea73a11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:26:56.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:56.167+0000 7fea73a11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:26:56.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:56.167+0000 7fea73a11640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-20T10:26:56.169 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:56.170+0000 7fea73a11640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-20T10:26:56.621 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:56.629 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:26:59.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-20T10:26:59.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:26:59.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:59.349+0000 7f1ac5c15640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-20T10:26:59.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:59.349+0000 7f1ac5c15640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-20T10:26:59.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:59.349+0000 7f1ac5c15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:26:59.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:59.349+0000 7f1ac5c15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:26:59.349 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:59.349+0000 7f1ac5c15640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-20T10:26:59.351 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:26:59.352+0000 7f1ac5c15640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-20T10:27:02.328 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-20T10:27:02.414 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-20T10:27:02.414 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:02.415 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-20T10:27:02.508 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:02.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-20T10:27:02.599 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:02.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-20T10:27:02.688 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:02.688 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-20T10:27:02.775 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:02.775 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-20T10:27:02.864 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:02.865 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-20T10:27:02.953 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:02.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-20T10:27:03.044 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.044 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-20T10:27:03.130 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.130 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-20T10:27:03.212 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-20T10:27:03.287 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.287 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-20T10:27:03.369 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.369 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-20T10:27:03.442 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.442 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-20T10:27:03.514 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-20T10:27:03.643 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:03.643 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-20T10:27:03.704 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:03.704 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-20T10:27:03.761 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.761 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-20T10:27:03.818 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:03.818 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-20T10:27:03.867 INFO:tasks.ceph:Adding keys to all mons... 2024-04-20T10:27:03.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T10:27:03.874 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBWmCNm862fAxAA3IcbrK5u7lUlerIjzUqIwA== 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBWmCNmFJVUGBAAWWv9IYGbahiOT/gLniKBOw== 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBWmCNmSGsVJxAA9UQVq15Djoww3cb3RYpnvw== 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2024-04-20T10:27:03.950 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBZmCNmA30rIRAAVq4KbsC5TmRgR2fuF6Ioow== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBcmCNmTJtrLxAAJmSqkjORGtnWDI0g4to51A== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBfmCNmxQDLORAAVyQxgnBHyR13yRBAGF+zNQ== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBjmCNmL4FwChAA11Oo/q6Csl2I4KAhh7pPDg== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBWmCNmVaHQDRAA1jKTsPvvO24I4zZ996gBQg== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBWmCNmTUa9NRAAqv2igp+zzxGgHeyPdWTaMg== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBXmCNm6XxVCBAA2JRaNbycMKfG+H5MRYd/BQ== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBpmCNmc0UlHhAA1/xDr2QsraxlgUXGj5zKJA== 2024-04-20T10:27:03.951 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBsmCNmyZ92ORAAkP7I8i8eANOvmITsE4Sv8Q== 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBwmCNmag3+CRAAr+SJC24tUSlkRpbdRnZydA== 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBzmCNmF+LYFBAAXXFXRj5Har31kqZtljDnWg== 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBXmCNmrHlNFRAAY90gnx6m90huEzUUDh9rtw== 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout:[client.2] 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBXmCNmSmJdHhAAsLDDN4yZBlkSAAPckAd0rw== 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBXmCNmaFnFJxAAIjkRNWpCrcCA8aq28sHHqQ== 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout:[client.3] 2024-04-20T10:27:03.952 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQBXmCNm4Nm7MRAA8yPbfNcKnJjd0QiaBTOcMw== 2024-04-20T10:27:03.953 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.y] 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBWmCNm862fAxAA3IcbrK5u7lUlerIjzUqIwA== 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.a] 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBWmCNmFJVUGBAAWWv9IYGbahiOT/gLniKBOw== 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.c] 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBWmCNmSGsVJxAA9UQVq15Djoww3cb3RYpnvw== 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.0] 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBZmCNmA30rIRAAVq4KbsC5TmRgR2fuF6Ioow== 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.1] 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBcmCNmTJtrLxAAJmSqkjORGtnWDI0g4to51A== 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.2] 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBfmCNmxQDLORAAVyQxgnBHyR13yRBAGF+zNQ== 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.3] 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBjmCNmL4FwChAA11Oo/q6Csl2I4KAhh7pPDg== 2024-04-20T10:27:03.954 INFO:teuthology.orchestra.run.smithi047.stdout:[mgr.x] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBWmCNmVaHQDRAA1jKTsPvvO24I4zZ996gBQg== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.b] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBWmCNmTUa9NRAAqv2igp+zzxGgHeyPdWTaMg== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[mds.d] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBXmCNm6XxVCBAA2JRaNbycMKfG+H5MRYd/BQ== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.4] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBpmCNmc0UlHhAA1/xDr2QsraxlgUXGj5zKJA== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.5] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBsmCNmyZ92ORAAkP7I8i8eANOvmITsE4Sv8Q== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.6] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBwmCNmag3+CRAAr+SJC24tUSlkRpbdRnZydA== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[osd.7] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBzmCNmF+LYFBAAXXFXRj5Har31kqZtljDnWg== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBXmCNmrHlNFRAAY90gnx6m90huEzUUDh9rtw== 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout:[client.2] 2024-04-20T10:27:03.955 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBXmCNmSmJdHhAAsLDDN4yZBlkSAAPckAd0rw== 2024-04-20T10:27:03.956 INFO:teuthology.orchestra.run.smithi047.stdout:[client.1] 2024-04-20T10:27:03.956 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBXmCNmaFnFJxAAIjkRNWpCrcCA8aq28sHHqQ== 2024-04-20T10:27:03.956 INFO:teuthology.orchestra.run.smithi047.stdout:[client.3] 2024-04-20T10:27:03.956 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBXmCNm4Nm7MRAA8yPbfNcKnJjd0QiaBTOcMw== 2024-04-20T10:27:03.958 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:27:04.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:27:04.144 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:04.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:04.287 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:04.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:04.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.434 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.706 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.845 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:04.984 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:27:04.990 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-20T10:27:05.121 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:05.127 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:05.265 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:05.272 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-20T10:27:05.410 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.415 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.549 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.554 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.689 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.694 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.826 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-20T10:27:05.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:05.968 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:06.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:06.115 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:06.250 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:06.257 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:06.396 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:06.403 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-20T10:27:06.540 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-20T10:27:06.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-20T10:27:06.629 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:27:06.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-20T10:27:06.873 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-20T10:27:06.961 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:27:07.108 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-20T10:27:07.187 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-20T10:27:07.270 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-20T10:27:07.420 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-20T10:27:07.494 DEBUG:teuthology.orchestra.run.smithi047:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:27:07.501 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-20T10:27:07.553 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-20T10:27:07.553 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-20T10:27:07.553 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-20T10:27:07.558 INFO:tasks.ceph.mon.a:Started 2024-04-20T10:27:07.559 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-20T10:27:07.559 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-20T10:27:07.563 INFO:tasks.ceph.mon.b:Started 2024-04-20T10:27:07.563 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-20T10:27:07.563 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-20T10:27:07.567 INFO:tasks.ceph.mon.c:Started 2024-04-20T10:27:07.567 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-20T10:27:07.567 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-20T10:27:07.568 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-20T10:27:07.572 INFO:tasks.ceph.mgr.y:Started 2024-04-20T10:27:07.572 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-20T10:27:07.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-20T10:27:07.577 INFO:tasks.ceph.mgr.x:Started 2024-04-20T10:27:07.577 INFO:tasks.ceph:Setting crush tunables to default 2024-04-20T10:27:07.577 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph osd crush tunables default 2024-04-20T10:27:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.753+0000 7f9ff9000640 1 Processor -- start 2024-04-20T10:27:07.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.754+0000 7f9ff9000640 1 -- start start 2024-04-20T10:27:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff9000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9ff40ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff9000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 0x7f9ff40d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff9000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ff4141c60 con 0x7f9ff4156c20 2024-04-20T10:27:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff9000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9ff406c010 con 0x7f9ff40ad8e0 2024-04-20T10:27:07.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff9000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff415cf90 con 0x7f9ff415ca40 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9febe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9ff40ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff2800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 0x7f9ff40d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff1e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ff4156c20 0x7f9ff406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58958/0 (socket says 172.21.15.47:58958) 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.755+0000 7f9ff1e00640 1 -- 172.21.15.47:0/2082281869 learned_addr learned my addr 172.21.15.47:0/2082281869 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9ff2800640 1 -- 172.21.15.47:0/2082281869 >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 msgr2=0x7f9ff40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9ff2800640 1 -- 172.21.15.47:0/2082281869 >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 msgr2=0x7f9ff40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2082281869 >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 0x7f9ff40d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2082281869 >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 0x7f9ff40d0360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9febe00640 1 -- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 msgr2=0x7f9ff40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2024-04-20T10:27:07.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9febe00640 1 -- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 msgr2=0x7f9ff40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-20T10:27:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9febe00640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9ff40ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-20T10:27:07.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.756+0000 7f9febe00640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9ff40ad3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-20T10:27:07.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1553989470 0 0) 0x7f9ff4141c60 con 0x7f9ff4156c20 2024-04-20T10:27:07.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9fd8003520 con 0x7f9ff4156c20 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3226630679 0 0) 0x7f9fd8003520 con 0x7f9ff4156c20 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 msgr2=0x7f9ff40ad3a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9ff40ad3a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 msgr2=0x7f9ff40d0360 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 --2- 172.21.15.47:0/2082281869 >> v2:172.21.15.175:3300/0 conn(0x7f9ff40ad8e0 0x7f9ff40d0360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff415d270 con 0x7f9ff4156c20 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f9fe4001670 con 0x7f9ff4156c20 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 >> v1:172.21.15.47:6789/0 conn(0x7f9ff4156c20 legacy=0x7f9ff406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80039b0 0x7f9fd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9fd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff406c010 con 0x7f9fd80039b0 2024-04-20T10:27:07.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff415cf90 con 0x7f9fd8004d40 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.768+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9fd800ba70 con 0x7f9ff415ca40 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff1e00640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80039b0 0x7f9fd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9febe00640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9fd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff2800640 1 -- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 msgr2=0x7f9fd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9fd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff2800640 1 -- 172.21.15.47:0/2082281869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80039b0 msgr2=0x7f9fd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80039b0 0x7f9fd8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff2800640 1 -- 172.21.15.47:0/2082281869 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd8003520 con 0x7f9fd8004d40 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9febe00640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ff415ca40 0x7f9fd8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:07.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.769+0000 7f9ff1e00640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9fd80039b0 0x7f9fd8003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.771+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009610 tx=0x7f9fd8005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79ccd3bf05950ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.771+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec039070 con 0x7f9fd8004d40 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.771+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec02cc80 con 0x7f9fd8004d40 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.771+0000 7f9ff1400640 1 -- 172.21.15.47:0/2082281869 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec009bd0 con 0x7f9fd8004d40 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.772+0000 7f9ff9000640 1 -- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8004d40 msgr2=0x7f9fd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.772+0000 7f9ff9000640 1 --2- 172.21.15.47:0/2082281869 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fd8004d40 0x7f9fd80051c0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009610 tx=0x7f9fd8005830 comp rx=0 tx=0).stop 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.772+0000 7f9ff2800640 1 -- 172.21.15.47:0/2082281869 reap_dead start 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.772+0000 7f9ff9000640 1 -- 172.21.15.47:0/2082281869 shutdown_connections 2024-04-20T10:27:07.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.772+0000 7f9ff9000640 1 -- 172.21.15.47:0/2082281869 >> 172.21.15.47:0/2082281869 conn(0x7f9ff40b3350 msgr2=0x7f9ff40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:07.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.772+0000 7f9ff9000640 1 -- 172.21.15.47:0/2082281869 shutdown_connections 2024-04-20T10:27:07.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.772+0000 7f9ff9000640 1 -- 172.21.15.47:0/2082281869 wait complete. 2024-04-20T10:27:07.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.774+0000 7f9ff9000640 1 Processor -- start 2024-04-20T10:27:07.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.774+0000 7f9ff9000640 1 -- start start 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff9000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8004d40 0x7f9ff4152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff9000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4073a40 0x7f9ff41f2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff41e7510 0x7f9ff41e79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff9000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff40ad3a0 con 0x7f9ff41e7510 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff9000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ff415d830 con 0x7f9ff4073a40 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff9000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ff40ad520 con 0x7f9fd8004d40 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff41e7510 0x7f9ff41e79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff41e7510 0x7f9ff41e79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41020/0 (socket says 172.21.15.47:41020) 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9ff2800640 1 -- 172.21.15.47:0/2084278383 learned_addr learned my addr 172.21.15.47:0/2084278383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.775+0000 7f9febe00640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4073a40 0x7f9ff41f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff1e00640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8004d40 0x7f9ff4152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:07.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff2800640 1 -- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8004d40 msgr2=0x7f9ff4152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:07.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8004d40 0x7f9ff4152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff2800640 1 -- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4073a40 msgr2=0x7f9ff41f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:07.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4073a40 0x7f9ff41f2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff2800640 1 -- 172.21.15.47:0/2084278383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ff40ce730 con 0x7f9ff41e7510 2024-04-20T10:27:07.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff2800640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff41e7510 0x7f9ff41e79a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0073f0 tx=0x7f9fec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9fea000640 1 -- 172.21.15.47:0/2084278383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec039040 con 0x7f9ff41e7510 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9fea000640 1 -- 172.21.15.47:0/2084278383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9fec046070 con 0x7f9ff41e7510 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9fea000640 1 -- 172.21.15.47:0/2084278383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fe4007d70 con 0x7f9ff41e7510 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.776+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ff40cf4b0 con 0x7f9ff41e7510 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.777+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ff40cecb0 con 0x7f9ff41e7510 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.777+0000 7f9fea000640 1 -- 172.21.15.47:0/2084278383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9fec03f470 con 0x7f9ff41e7510 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.777+0000 7f9fea000640 1 -- 172.21.15.47:0/2084278383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9fec04b4b0 con 0x7f9ff41e7510 2024-04-20T10:27:07.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.778+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ff4068980 con 0x7f9ff41e7510 2024-04-20T10:27:07.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.781+0000 7f9fea000640 1 -- 172.21.15.47:0/2084278383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f9fec03f8a0 con 0x7f9ff41e7510 2024-04-20T10:27:07.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.853+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f9ff40cfd90 con 0x7f9ff41e7510 2024-04-20T10:27:07.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.856+0000 7f9fea000640 1 -- 172.21.15.47:0/2084278383 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f9fec03fa80 con 0x7f9ff41e7510 2024-04-20T10:27:07.856 INFO:teuthology.orchestra.run.smithi047.stderr:adjusted tunables profile to default 2024-04-20T10:27:07.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.857+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff41e7510 msgr2=0x7f9ff41e79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:07.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.857+0000 7f9ff9000640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff41e7510 0x7f9ff41e79a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fec0073f0 tx=0x7f9fec004ab0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.857+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 shutdown_connections 2024-04-20T10:27:07.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.857+0000 7f9ff9000640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ff41e7510 0x7f9ff41e79a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.857+0000 7f9ff9000640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ff4073a40 0x7f9ff41f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.857+0000 7f9ff9000640 1 --2- 172.21.15.47:0/2084278383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fd8004d40 0x7f9ff4152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:07.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.858+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 >> 172.21.15.47:0/2084278383 conn(0x7f9ff40b3350 msgr2=0x7f9ff415e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:07.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.858+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 shutdown_connections 2024-04-20T10:27:07.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:07.858+0000 7f9ff9000640 1 -- 172.21.15.47:0/2084278383 wait complete. 2024-04-20T10:27:07.875 INFO:tasks.ceph:check_enable_crimson: False 2024-04-20T10:27:07.876 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-20T10:27:07.876 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:07.876 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-20T10:27:07.953 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:07.953 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-20T10:27:08.070 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:08.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-20T10:27:08.071 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:08.071+0000 7fe674a21040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:27:08.105 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:08.105+0000 7f8bfcf53040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-20T10:27:08.145 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:08.145 DEBUG:teuthology.orchestra.run.smithi047:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-20T10:27:08.263 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:08.263+0000 7fe674a21040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:27:08.269 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:08.269 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-20T10:27:08.327 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:08.328+0000 7f8bfcf53040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:27:08.332 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:08.332+0000 7fe674a21040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:27:08.353 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:08.353 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-20T10:27:08.395 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:08.395+0000 7f8bfcf53040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-20T10:27:08.421 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:08.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-20T10:27:08.542 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:27:08.543 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-20T10:27:08.612 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 42adcc33-e207-4065-962d-c429043222a8 0 2024-04-20T10:27:08.662 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:08.662+0000 7fe674a21040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:27:08.718 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:08.718+0000 7fe674a21040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:27:08.726 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:08.726+0000 7f8bfcf53040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-20T10:27:08.787 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:08.787+0000 7f8bfcf53040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-20T10:27:08.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.832+0000 7ff777200640 1 Processor -- start 2024-04-20T10:27:08.832 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.832+0000 7ff777200640 1 -- start start 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff777200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7780d1010 0x7ff7780b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff777200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7780d46f0 0x7ff7780d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff777200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7780d15b0 con 0x7ff77815e8c0 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff777200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff7780d5080 con 0x7ff7780d1010 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff777200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff7780b6170 con 0x7ff7780d46f0 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff776800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7780d46f0 0x7ff7780d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff776800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7780d46f0 0x7ff7780d4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47338/0 (socket says 172.21.15.175:47338) 2024-04-20T10:27:08.834 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff776800640 1 -- 172.21.15.175:0/4125246640 learned_addr learned my addr 172.21.15.175:0/4125246640 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:08.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff776800640 1 -- 172.21.15.175:0/4125246640 >> v2:172.21.15.47:3300/0 conn(0x7ff7780d1010 msgr2=0x7ff7780b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:08.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff776800640 1 --2- 172.21.15.175:0/4125246640 >> v2:172.21.15.47:3300/0 conn(0x7ff7780d1010 0x7ff7780b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.834+0000 7ff776800640 1 -- 172.21.15.175:0/4125246640 >> v1:172.21.15.47:6789/0 conn(0x7ff77815e8c0 legacy=0x7ff77806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:08.835 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.835+0000 7ff776800640 1 -- 172.21.15.175:0/4125246640 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7780d1890 con 0x7ff7780d46f0 2024-04-20T10:27:08.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.836+0000 7ff776800640 1 --2- 172.21.15.175:0/4125246640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d46f0 0x7ff7780d4b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff77808dfa0 tx=0x7ff77002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4a252481f06c663b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:08.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.836+0000 7ff774a00640 1 -- 172.21.15.175:0/4125246640 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff77002ca80 con 0x7ff7780d46f0 2024-04-20T10:27:08.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff774a00640 1 -- 172.21.15.175:0/4125246640 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff77002cc20 con 0x7ff7780d46f0 2024-04-20T10:27:08.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff774a00640 1 -- 172.21.15.175:0/4125246640 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7700343f0 con 0x7ff7780d46f0 2024-04-20T10:27:08.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 -- 172.21.15.175:0/4125246640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d46f0 msgr2=0x7ff7780d4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:08.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 --2- 172.21.15.175:0/4125246640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d46f0 0x7ff7780d4b40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff77808dfa0 tx=0x7ff77002bee0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 -- 172.21.15.175:0/4125246640 shutdown_connections 2024-04-20T10:27:08.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 --2- 172.21.15.175:0/4125246640 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d46f0 0x7ff7780d4b40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 --2- 172.21.15.175:0/4125246640 >> v2:172.21.15.47:3300/0 conn(0x7ff7780d1010 0x7ff7780b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.837 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 -- 172.21.15.175:0/4125246640 >> 172.21.15.175:0/4125246640 conn(0x7ff77809d6f0 msgr2=0x7ff7780b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:08.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 -- 172.21.15.175:0/4125246640 shutdown_connections 2024-04-20T10:27:08.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.837+0000 7ff777200640 1 -- 172.21.15.175:0/4125246640 wait complete. 2024-04-20T10:27:08.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.839+0000 7ff777200640 1 Processor -- start 2024-04-20T10:27:08.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.839+0000 7ff777200640 1 -- start start 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.840+0000 7ff777200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d1010 0x7ff77814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.840+0000 7ff777200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7780d46f0 0x7ff7780ca850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.840+0000 7ff777200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77815e8c0 0x7ff77806c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.840+0000 7ff777200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780b6b40 con 0x7ff7780d46f0 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.840+0000 7ff777200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7780b5e00 con 0x7ff7780d1010 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.840+0000 7ff777200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7780b6cc0 con 0x7ff77815e8c0 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.840+0000 7ff775e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d1010 0x7ff77814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d1010 0x7ff77814e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47354/0 (socket says 172.21.15.175:47354) 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 -- 172.21.15.175:0/4265946051 learned_addr learned my addr 172.21.15.175:0/4265946051 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff776800640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77815e8c0 0x7ff77806c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775400640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7780d46f0 0x7ff7780ca850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 -- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77815e8c0 msgr2=0x7ff77806c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:08.841 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77815e8c0 0x7ff77806c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 -- 172.21.15.175:0/4265946051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7780d46f0 msgr2=0x7ff7780ca850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:08.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7780d46f0 0x7ff7780ca850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 -- 172.21.15.175:0/4265946051 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77814d590 con 0x7ff7780d1010 2024-04-20T10:27:08.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff776800640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77815e8c0 0x7ff77806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.841+0000 7ff775e00640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d1010 0x7ff77814e540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff76800b3e0 tx=0x7ff76800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.842+0000 7ff76e000640 1 -- 172.21.15.175:0/4265946051 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768015020 con 0x7ff7780d1010 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.842+0000 7ff76e000640 1 -- 172.21.15.175:0/4265946051 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff768011850 con 0x7ff7780d1010 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.842+0000 7ff76e000640 1 -- 172.21.15.175:0/4265946051 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff768011c40 con 0x7ff7780d1010 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.842+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff77806c7c0 con 0x7ff7780d1010 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.842+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77806f0b0 con 0x7ff7780d1010 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.842+0000 7ff76e000640 1 -- 172.21.15.175:0/4265946051 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff768018020 con 0x7ff7780d1010 2024-04-20T10:27:08.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.842+0000 7ff76e000640 1 -- 172.21.15.175:0/4265946051 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7ff768019770 con 0x7ff7780d1010 2024-04-20T10:27:08.844 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.843+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff778075b30 con 0x7ff7780d1010 2024-04-20T10:27:08.845 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.846+0000 7ff76e000640 1 -- 172.21.15.175:0/4265946051 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7ff76801f020 con 0x7ff7780d1010 2024-04-20T10:27:08.925 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.925+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "42adcc33-e207-4065-962d-c429043222a8", "id": 0} v 0) -- 0x7ff7780b2000 con 0x7ff7780d1010 2024-04-20T10:27:08.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.933+0000 7ff76e000640 1 -- 172.21.15.175:0/4265946051 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "42adcc33-e207-4065-962d-c429043222a8", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7ff768011de0 con 0x7ff7780d1010 2024-04-20T10:27:08.933 INFO:teuthology.orchestra.run.smithi175.stdout:0 2024-04-20T10:27:08.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.934+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d1010 msgr2=0x7ff77814e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:08.934 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.934+0000 7ff777200640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d1010 0x7ff77814e540 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff76800b3e0 tx=0x7ff76800b8a0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.935+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 shutdown_connections 2024-04-20T10:27:08.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.935+0000 7ff777200640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77815e8c0 0x7ff77806c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.935+0000 7ff777200640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7780d46f0 0x7ff7780ca850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.935+0000 7ff777200640 1 --2- 172.21.15.175:0/4265946051 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7780d1010 0x7ff77814e540 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:08.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.935+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 >> 172.21.15.175:0/4265946051 conn(0x7ff77809d6f0 msgr2=0x7ff7780b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:08.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.935+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 shutdown_connections 2024-04-20T10:27:08.935 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:08.935+0000 7ff777200640 1 -- 172.21.15.175:0/4265946051 wait complete. 2024-04-20T10:27:08.952 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new e74aaa9e-5db8-4f77-8235-5a57636e3b53 1 2024-04-20T10:27:09.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.132+0000 7f8414e00640 1 Processor -- start 2024-04-20T10:27:09.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.132+0000 7f8414e00640 1 -- start start 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f8414e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84100d3b20 0x7f84100b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f8414e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f841006c010 0x7f84100ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f8414e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f84100d0980 con 0x7f8410161a50 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f8414e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f84100d0c60 con 0x7f841006c010 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f8414e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8410162ae0 con 0x7f84100d3b20 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f840fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f841006c010 0x7f84100ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f840ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84100d3b20 0x7f84100b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f840fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f841006c010 0x7f84100ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47356/0 (socket says 172.21.15.175:47356) 2024-04-20T10:27:09.134 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.134+0000 7f840fe00640 1 -- 172.21.15.175:0/1934746063 learned_addr learned my addr 172.21.15.175:0/1934746063 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.135+0000 7f840fe00640 1 -- 172.21.15.175:0/1934746063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84100d3b20 msgr2=0x7f84100b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.135+0000 7f840fe00640 1 --2- 172.21.15.175:0/1934746063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84100d3b20 0x7f84100b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.135+0000 7f840fe00640 1 -- 172.21.15.175:0/1934746063 >> v1:172.21.15.47:6789/0 conn(0x7f8410161a50 legacy=0x7f841006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.135+0000 7f840fe00640 1 -- 172.21.15.175:0/1934746063 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8410162490 con 0x7f841006c010 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.136+0000 7f840fe00640 1 --2- 172.21.15.175:0/1934746063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841006c010 0x7f84100ceaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8408009e50 tx=0x7f840802bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae6bedb325d80ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.136+0000 7f840e000640 1 -- 172.21.15.175:0/1934746063 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f840802cb00 con 0x7f841006c010 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.136+0000 7f840e000640 1 -- 172.21.15.175:0/1934746063 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840802cca0 con 0x7f841006c010 2024-04-20T10:27:09.136 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.136+0000 7f840e000640 1 -- 172.21.15.175:0/1934746063 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408034610 con 0x7f841006c010 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 -- 172.21.15.175:0/1934746063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841006c010 msgr2=0x7f84100ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 --2- 172.21.15.175:0/1934746063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841006c010 0x7f84100ceaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8408009e50 tx=0x7f840802bf60 comp rx=0 tx=0).stop 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 -- 172.21.15.175:0/1934746063 shutdown_connections 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 --2- 172.21.15.175:0/1934746063 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841006c010 0x7f84100ceaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 --2- 172.21.15.175:0/1934746063 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f84100d3b20 0x7f84100b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 -- 172.21.15.175:0/1934746063 >> 172.21.15.175:0/1934746063 conn(0x7f84100d0ed0 msgr2=0x7f84100d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 -- 172.21.15.175:0/1934746063 shutdown_connections 2024-04-20T10:27:09.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.137+0000 7f8414e00640 1 -- 172.21.15.175:0/1934746063 wait complete. 2024-04-20T10:27:09.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.139+0000 7f8414e00640 1 Processor -- start 2024-04-20T10:27:09.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.139+0000 7f8414e00640 1 -- start start 2024-04-20T10:27:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f8414e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841006c010 0x7f84100ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f8414e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8410161a50 0x7f84100aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f8414e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841015c860 0x7f84100b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f8414e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f84100d0600 con 0x7f8410161a50 2024-04-20T10:27:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f8414e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84100d0480 con 0x7f841015c860 2024-04-20T10:27:09.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f8414e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8410161e30 con 0x7f841006c010 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f840f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841006c010 0x7f84100ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f840fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841015c860 0x7f84100b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f840fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841015c860 0x7f84100b4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47358/0 (socket says 172.21.15.175:47358) 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f840f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841006c010 0x7f84100ca300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52174/0 (socket says 172.21.15.175:52174) 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f840fe00640 1 -- 172.21.15.175:0/3694128310 learned_addr learned my addr 172.21.15.175:0/3694128310 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.140+0000 7f840ea00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8410161a50 0x7f84100aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f840fe00640 1 -- 172.21.15.175:0/3694128310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841006c010 msgr2=0x7f84100ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f840fe00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841006c010 0x7f84100ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f840fe00640 1 -- 172.21.15.175:0/3694128310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8410161a50 msgr2=0x7f84100aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f840fe00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8410161a50 0x7f84100aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.141 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f840fe00640 1 -- 172.21.15.175:0/3694128310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84100ab650 con 0x7f841015c860 2024-04-20T10:27:09.142 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f840fe00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841015c860 0x7f84100b4f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8408009f70 tx=0x7f8408004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f8407400640 1 -- 172.21.15.175:0/3694128310 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408043020 con 0x7f841015c860 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f8407400640 1 -- 172.21.15.175:0/3694128310 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f840802ce50 con 0x7f841015c860 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84100ab930 con 0x7f841015c860 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f8407400640 1 -- 172.21.15.175:0/3694128310 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8408009510 con 0x7f841015c860 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.141+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84100b1190 con 0x7f841015c860 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.142+0000 7f8407400640 1 -- 172.21.15.175:0/3694128310 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8408046020 con 0x7f841015c860 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.142+0000 7f8407400640 1 -- 172.21.15.175:0/3694128310 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f84080487b0 con 0x7f841015c860 2024-04-20T10:27:09.143 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.142+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8410075b30 con 0x7f841015c860 2024-04-20T10:27:09.146 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.147+0000 7f8407400640 1 -- 172.21.15.175:0/3694128310 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f840804d030 con 0x7f841015c860 2024-04-20T10:27:09.225 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.225+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e74aaa9e-5db8-4f77-8235-5a57636e3b53", "id": 1} v 0) -- 0x7f84100accc0 con 0x7f841015c860 2024-04-20T10:27:09.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.232+0000 7f8407400640 1 -- 172.21.15.175:0/3694128310 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e74aaa9e-5db8-4f77-8235-5a57636e3b53", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f840803ee10 con 0x7f841015c860 2024-04-20T10:27:09.232 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841015c860 msgr2=0x7f84100b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841015c860 0x7f84100b4f10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8408009f70 tx=0x7f8408004a40 comp rx=0 tx=0).stop 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 shutdown_connections 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f841015c860 0x7f84100b4f10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8410161a50 0x7f84100aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 --2- 172.21.15.175:0/3694128310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f841006c010 0x7f84100ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 >> 172.21.15.175:0/3694128310 conn(0x7f84100d0ed0 msgr2=0x7f84100d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 shutdown_connections 2024-04-20T10:27:09.234 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.234+0000 7f8414e00640 1 -- 172.21.15.175:0/3694128310 wait complete. 2024-04-20T10:27:09.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 509d1c2b-3974-4e6e-bf6c-2b3861e20c72 2 2024-04-20T10:27:09.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.429+0000 7f0a09400640 1 Processor -- start 2024-04-20T10:27:09.429 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.429+0000 7f0a09400640 1 -- start start 2024-04-20T10:27:09.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.430+0000 7f0a09400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0a04158a00 0x7f0a04162c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.430+0000 7f0a09400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a040b6290 con 0x7f0a04158de0 2024-04-20T10:27:09.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.430+0000 7f0a09400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0a0415f150 con 0x7f0a04073370 2024-04-20T10:27:09.430 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.430+0000 7f0a09400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0a0415f430 con 0x7f0a04158a00 2024-04-20T10:27:09.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.431+0000 7f0a03200640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f0a04073370 0x7f0a040ce160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.175:60316/0 (socket says 172.21.15.175:60316) 2024-04-20T10:27:09.431 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.431+0000 7f0a03200640 1 -- 172.21.15.175:0/477786311 learned_addr learned my addr 172.21.15.175:0/477786311 (peer_addr_for_me v1:172.21.15.175:0/0) 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.431+0000 7f0a02800640 1 --2- 172.21.15.175:0/477786311 >> v2:172.21.15.47:3300/0 conn(0x7f0a04158a00 0x7f0a04162c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.431+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2337290397 0 0) 0x7f0a0415f150 con 0x7f0a04073370 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.431+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f09e8003520 con 0x7f0a04073370 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.431+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2069838061 0 0) 0x7f0a040b6290 con 0x7f0a04158de0 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.431+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0a0415f150 con 0x7f0a04158de0 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1343253755 0 0) 0x7f09e8003520 con 0x7f0a04073370 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 >> v2:172.21.15.47:3300/0 conn(0x7f0a04158a00 msgr2=0x7f0a04162c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 --2- 172.21.15.175:0/477786311 >> v2:172.21.15.47:3300/0 conn(0x7f0a04158a00 0x7f0a04162c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 >> v1:172.21.15.47:6789/0 conn(0x7f0a04158de0 legacy=0x7f0a04072c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0415f710 con 0x7f0a04073370 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f09fc0019f0 con 0x7f0a04073370 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 >> v1:172.21.15.175:6789/0 conn(0x7f0a04073370 legacy=0x7f0a040ce160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8003b40 0x7f09e8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004800 0x7f09e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09e80051c0 0x7f09e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0415f430 con 0x7f09e80051c0 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09e800c040 con 0x7f09e8004800 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f09e800c240 con 0x7f09e8003b40 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a02800640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8003b40 0x7f09e8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a01e00640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004800 0x7f09e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a03200640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09e80051c0 0x7f09e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a02800640 1 -- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004800 msgr2=0x7f09e8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a02800640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8004800 0x7f09e8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.434 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a02800640 1 -- 172.21.15.175:0/477786311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09e80051c0 msgr2=0x7f09e800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:09.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a02800640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09e80051c0 0x7f09e800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a02800640 1 -- 172.21.15.175:0/477786311 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e8003520 con 0x7f09e8003b40 2024-04-20T10:27:09.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a03200640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f09e80051c0 0x7f09e800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:09.435 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.432+0000 7f0a03200640 1 -- 172.21.15.175:0/477786311 reap_dead start 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.434+0000 7f0a02800640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8003b40 0x7f09e8003f40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009ac0 tx=0x7f09e8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85419899960efd1a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.434+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4039070 con 0x7f09e8003b40 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.434+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f402c6b0 con 0x7f09e8003b40 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.434+0000 7f0a01400640 1 -- 172.21.15.175:0/477786311 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4009690 con 0x7f09e8003b40 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.434+0000 7f0a09400640 1 -- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8003b40 msgr2=0x7f09e8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.434+0000 7f0a09400640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8003b40 0x7f09e8003f40 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009ac0 tx=0x7f09e8005830 comp rx=0 tx=0).stop 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.435+0000 7f0a09400640 1 -- 172.21.15.175:0/477786311 shutdown_connections 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.435+0000 7f0a09400640 1 --2- 172.21.15.175:0/477786311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09e8003b40 0x7f09e8003f40 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.436 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.435+0000 7f0a09400640 1 -- 172.21.15.175:0/477786311 >> 172.21.15.175:0/477786311 conn(0x7f0a040b31c0 msgr2=0x7f0a040b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:09.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.435+0000 7f0a09400640 1 -- 172.21.15.175:0/477786311 shutdown_connections 2024-04-20T10:27:09.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.435+0000 7f0a09400640 1 -- 172.21.15.175:0/477786311 wait complete. 2024-04-20T10:27:09.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.436+0000 7f0a09400640 1 Processor -- start 2024-04-20T10:27:09.437 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.437+0000 7f0a09400640 1 -- start start 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.437+0000 7f0a09400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003b40 0x7f0a0414a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.437+0000 7f0a09400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0406f210 0x7f0a0414ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.437+0000 7f0a09400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a041f7e20 0x7f0a041f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.437+0000 7f0a09400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0415ed50 con 0x7f0a041f7e20 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.437+0000 7f0a09400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0415ebd0 con 0x7f09e8003b40 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.437+0000 7f0a09400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a0415eed0 con 0x7f0a0406f210 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a02800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003b40 0x7f0a0414a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0406f210 0x7f0a0414ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0406f210 0x7f0a0414ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52188/0 (socket says 172.21.15.175:52188) 2024-04-20T10:27:09.439 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 -- 172.21.15.175:0/422443074 learned_addr learned my addr 172.21.15.175:0/422443074 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a03200640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a041f7e20 0x7f0a041f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 -- 172.21.15.175:0/422443074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003b40 msgr2=0x7f0a0414a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003b40 0x7f0a0414a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 -- 172.21.15.175:0/422443074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a041f7e20 msgr2=0x7f0a041f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a041f7e20 0x7f0a041f82b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 -- 172.21.15.175:0/422443074 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0406f6a0 con 0x7f0a0406f210 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.438+0000 7f0a01e00640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0406f210 0x7f0a0414ad80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f09f000f770 tx=0x7f09f000fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.439+0000 7f09faa00640 1 -- 172.21.15.175:0/422443074 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f00050d0 con 0x7f0a0406f210 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.439+0000 7f09faa00640 1 -- 172.21.15.175:0/422443074 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f09f0018070 con 0x7f0a0406f210 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.439+0000 7f09faa00640 1 -- 172.21.15.175:0/422443074 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f000d730 con 0x7f0a0406f210 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.439+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0406c860 con 0x7f0a0406f210 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.439+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0406ce20 con 0x7f0a0406f210 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.439+0000 7f09faa00640 1 -- 172.21.15.175:0/422443074 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f09f000dae0 con 0x7f0a0406f210 2024-04-20T10:27:09.440 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.439+0000 7f09faa00640 1 -- 172.21.15.175:0/422443074 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f09f000ddd0 con 0x7f0a0406f210 2024-04-20T10:27:09.441 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.440+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a04069090 con 0x7f0a0406f210 2024-04-20T10:27:09.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.443+0000 7f09faa00640 1 -- 172.21.15.175:0/422443074 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f09f00035f0 con 0x7f0a0406f210 2024-04-20T10:27:09.548 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.548+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "509d1c2b-3974-4e6e-bf6c-2b3861e20c72", "id": 2} v 0) -- 0x7f0a04162740 con 0x7f0a0406f210 2024-04-20T10:27:09.556 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.556+0000 7f09faa00640 1 -- 172.21.15.175:0/422443074 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "509d1c2b-3974-4e6e-bf6c-2b3861e20c72", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f09f0003890 con 0x7f0a0406f210 2024-04-20T10:27:09.556 INFO:teuthology.orchestra.run.smithi175.stdout:2 2024-04-20T10:27:09.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.557+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0406f210 msgr2=0x7f0a0414ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.557+0000 7f0a09400640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0406f210 0x7f0a0414ad80 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f09f000f770 tx=0x7f09f000fc30 comp rx=0 tx=0).stop 2024-04-20T10:27:09.557 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.557+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 shutdown_connections 2024-04-20T10:27:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.557+0000 7f0a09400640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0a041f7e20 0x7f0a041f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.557+0000 7f0a09400640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a0406f210 0x7f0a0414ad80 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.557+0000 7f0a09400640 1 --2- 172.21.15.175:0/422443074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09e8003b40 0x7f0a0414a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.557+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 >> 172.21.15.175:0/422443074 conn(0x7f0a040b31c0 msgr2=0x7f0a040b35a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.558+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 shutdown_connections 2024-04-20T10:27:09.558 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.558+0000 7f0a09400640 1 -- 172.21.15.175:0/422443074 wait complete. 2024-04-20T10:27:09.574 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 5e63812e-7c05-4655-8d10-89aaccfc408c 3 2024-04-20T10:27:09.735 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:09.735+0000 7f8bfcf53040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:27:09.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.750+0000 7f5905c00640 1 Processor -- start 2024-04-20T10:27:09.750 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.750+0000 7f5905c00640 1 -- start start 2024-04-20T10:27:09.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f5905c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f590006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.752 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f5905c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f59000d1010 0x7f59000b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f5905c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59000d46f0 0x7f59000d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f5905c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f59000b6170 con 0x7f59000d46f0 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f5905c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f59000d15b0 con 0x7f59000d1010 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f5905c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59000d1890 con 0x7f590015e8c0 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f590006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f58ff400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f59000d1010 0x7f59000b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f58ffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f590006ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52204/0 (socket says 172.21.15.175:52204) 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f58ffe00640 1 -- 172.21.15.175:0/3464291312 learned_addr learned my addr 172.21.15.175:0/3464291312 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.752+0000 7f5905200640 1 --2- 172.21.15.175:0/3464291312 >> v2:172.21.15.47:3300/0 conn(0x7f59000d46f0 0x7f59000d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.753+0000 7f58ffe00640 1 -- 172.21.15.175:0/3464291312 >> v2:172.21.15.175:3300/0 conn(0x7f59000d1010 msgr2=0x7f59000b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.753+0000 7f58ffe00640 1 --2- 172.21.15.175:0/3464291312 >> v2:172.21.15.175:3300/0 conn(0x7f59000d1010 0x7f59000b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.753+0000 7f58ffe00640 1 -- 172.21.15.175:0/3464291312 >> v2:172.21.15.47:3300/0 conn(0x7f59000d46f0 msgr2=0x7f59000d4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:09.753 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.753+0000 7f58ffe00640 1 --2- 172.21.15.175:0/3464291312 >> v2:172.21.15.47:3300/0 conn(0x7f59000d46f0 0x7f59000d4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.753+0000 7f58ffe00640 1 -- 172.21.15.175:0/3464291312 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59000d1b70 con 0x7f590015e8c0 2024-04-20T10:27:09.754 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.753+0000 7f58ff400640 1 --2- 172.21.15.175:0/3464291312 >> v2:172.21.15.175:3300/0 conn(0x7f59000d1010 0x7f59000b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.754+0000 7f58ffe00640 1 --2- 172.21.15.175:0/3464291312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f590006ec40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f590008c8e0 tx=0x7f58f002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1246fe619f7fc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.754+0000 7f58fea00640 1 -- 172.21.15.175:0/3464291312 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0035020 con 0x7f590015e8c0 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.754+0000 7f58fea00640 1 -- 172.21.15.175:0/3464291312 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f0031870 con 0x7f590015e8c0 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.754+0000 7f58fea00640 1 -- 172.21.15.175:0/3464291312 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f0031c40 con 0x7f590015e8c0 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 -- 172.21.15.175:0/3464291312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 msgr2=0x7f590006ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 --2- 172.21.15.175:0/3464291312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f590006ec40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f590008c8e0 tx=0x7f58f002b910 comp rx=0 tx=0).stop 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 -- 172.21.15.175:0/3464291312 shutdown_connections 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 --2- 172.21.15.175:0/3464291312 >> v2:172.21.15.47:3300/0 conn(0x7f59000d46f0 0x7f59000d4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 --2- 172.21.15.175:0/3464291312 >> v2:172.21.15.175:3300/0 conn(0x7f59000d1010 0x7f59000b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.755 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 --2- 172.21.15.175:0/3464291312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f590006ec40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 -- 172.21.15.175:0/3464291312 >> 172.21.15.175:0/3464291312 conn(0x7f590009d6f0 msgr2=0x7f59000b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:09.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 -- 172.21.15.175:0/3464291312 shutdown_connections 2024-04-20T10:27:09.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.755+0000 7f5905c00640 1 -- 172.21.15.175:0/3464291312 wait complete. 2024-04-20T10:27:09.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.757+0000 7f5905c00640 1 Processor -- start 2024-04-20T10:27:09.756 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.757+0000 7f5905c00640 1 -- start start 2024-04-20T10:27:09.759 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f5905c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000d1010 0x7f59001615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f5905c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000d46f0 0x7f59000c9e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f5905c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f59000cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f5905c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f59000b6b40 con 0x7f59000d1010 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f5905c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f590006f180 con 0x7f59000d46f0 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f5905c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59000b6cc0 con 0x7f590015e8c0 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f58ff400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000d46f0 0x7f59000c9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f5905200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f59000cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f58ff400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000d46f0 0x7f59000c9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47398/0 (socket says 172.21.15.175:47398) 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f58ff400640 1 -- 172.21.15.175:0/4158326792 learned_addr learned my addr 172.21.15.175:0/4158326792 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.758+0000 7f58ffe00640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000d1010 0x7f59001615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905200640 1 -- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000d46f0 msgr2=0x7f59000c9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905200640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000d46f0 0x7f59000c9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905200640 1 -- 172.21.15.175:0/4158326792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000d1010 msgr2=0x7f59001615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.760 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905200640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000d1010 0x7f59001615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905200640 1 -- 172.21.15.175:0/4158326792 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f590015fbe0 con 0x7f590015e8c0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f58ff400640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000d46f0 0x7f59000c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905200640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f59000cd560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58f8007d90 tx=0x7f58f800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f58f7e00640 1 -- 172.21.15.175:0/4158326792 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f800edd0 con 0x7f590015e8c0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f58f7e00640 1 -- 172.21.15.175:0/4158326792 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58f8003990 con 0x7f590015e8c0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f58f7e00640 1 -- 172.21.15.175:0/4158326792 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f8003d80 con 0x7f590015e8c0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f590015fec0 con 0x7f590015e8c0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.759+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5900160480 con 0x7f590015e8c0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.760+0000 7f58f7e00640 1 -- 172.21.15.175:0/4158326792 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f58f801b020 con 0x7f590015e8c0 2024-04-20T10:27:09.761 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.760+0000 7f58f7e00640 1 -- 172.21.15.175:0/4158326792 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f58f8014420 con 0x7f590015e8c0 2024-04-20T10:27:09.762 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:09.759+0000 7fe674a21040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-20T10:27:09.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.760+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5900075b30 con 0x7f590015e8c0 2024-04-20T10:27:09.762 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.763+0000 7f58f7e00640 1 -- 172.21.15.175:0/4158326792 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f58f8014a60 con 0x7f590015e8c0 2024-04-20T10:27:09.843 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.843+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5e63812e-7c05-4655-8d10-89aaccfc408c", "id": 3} v 0) -- 0x7f59000b1c20 con 0x7f590015e8c0 2024-04-20T10:27:09.846 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:09.846+0000 7f8bfcf53040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:27:09.849 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.850+0000 7f58f7e00640 1 -- 172.21.15.175:0/4158326792 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5e63812e-7c05-4655-8d10-89aaccfc408c", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f58f8014d00 con 0x7f590015e8c0 2024-04-20T10:27:09.849 INFO:teuthology.orchestra.run.smithi175.stdout:3 2024-04-20T10:27:09.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 msgr2=0x7f59000cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:09.850 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f59000cd560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f58f8007d90 tx=0x7f58f800e4b0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 shutdown_connections 2024-04-20T10:27:09.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f590015e8c0 0x7f59000cd560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59000d46f0 0x7f59000c9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 --2- 172.21.15.175:0/4158326792 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59000d1010 0x7f59001615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:09.851 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 >> 172.21.15.175:0/4158326792 conn(0x7f590009d6f0 msgr2=0x7f59000d4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:09.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 shutdown_connections 2024-04-20T10:27:09.852 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:09.851+0000 7f5905c00640 1 -- 172.21.15.175:0/4158326792 wait complete. 2024-04-20T10:27:09.868 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new ead50013-a94f-44b0-9051-4e01a5858c4e 4 2024-04-20T10:27:09.879 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:09.879+0000 7fe674a21040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-20T10:27:09.901 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:09.902+0000 7f8bfcf53040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:27:09.936 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:09.937+0000 7fe674a21040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-20T10:27:10.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.038+0000 7fe19ee00640 1 Processor -- start 2024-04-20T10:27:10.038 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.038+0000 7fe19ee00640 1 -- start start 2024-04-20T10:27:10.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19ee00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1980d3b20 0x7fe1980b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806c010 0x7fe1980ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19ee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1980d0980 con 0x7fe198161a50 2024-04-20T10:27:10.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19ee00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe1980d0c60 con 0x7fe1980d3b20 2024-04-20T10:27:10.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe198162ae0 con 0x7fe19806c010 2024-04-20T10:27:10.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806c010 0x7fe1980ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.040 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19d000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe1980d3b20 0x7fe1980b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806c010 0x7fe1980ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52224/0 (socket says 172.21.15.175:52224) 2024-04-20T10:27:10.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.040+0000 7fe19e400640 1 -- 172.21.15.175:0/735546475 learned_addr learned my addr 172.21.15.175:0/735546475 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.041+0000 7fe19e400640 1 -- 172.21.15.175:0/735546475 >> v2:172.21.15.175:3300/0 conn(0x7fe1980d3b20 msgr2=0x7fe1980b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:10.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.041+0000 7fe19e400640 1 --2- 172.21.15.175:0/735546475 >> v2:172.21.15.175:3300/0 conn(0x7fe1980d3b20 0x7fe1980b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.041+0000 7fe19e400640 1 -- 172.21.15.175:0/735546475 >> v1:172.21.15.47:6789/0 conn(0x7fe198161a50 legacy=0x7fe19806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.041+0000 7fe19e400640 1 -- 172.21.15.175:0/735546475 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe198162490 con 0x7fe19806c010 2024-04-20T10:27:10.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.041+0000 7fe19d000640 1 --2- 172.21.15.175:0/735546475 >> v2:172.21.15.175:3300/0 conn(0x7fe1980d3b20 0x7fe1980b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.042+0000 7fe19e400640 1 --2- 172.21.15.175:0/735546475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806c010 0x7fe1980ceaa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe190009e50 tx=0x7fe19002bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7dbf08451a75bdd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.042+0000 7fe197e00640 1 -- 172.21.15.175:0/735546475 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19002cb00 con 0x7fe19806c010 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.042+0000 7fe197e00640 1 -- 172.21.15.175:0/735546475 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe19002cca0 con 0x7fe19806c010 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.042+0000 7fe197e00640 1 -- 172.21.15.175:0/735546475 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190034610 con 0x7fe19806c010 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 -- 172.21.15.175:0/735546475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806c010 msgr2=0x7fe1980ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 --2- 172.21.15.175:0/735546475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806c010 0x7fe1980ceaa0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe190009e50 tx=0x7fe19002bf60 comp rx=0 tx=0).stop 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 -- 172.21.15.175:0/735546475 shutdown_connections 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 --2- 172.21.15.175:0/735546475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806c010 0x7fe1980ceaa0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 --2- 172.21.15.175:0/735546475 >> v2:172.21.15.175:3300/0 conn(0x7fe1980d3b20 0x7fe1980b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 -- 172.21.15.175:0/735546475 >> 172.21.15.175:0/735546475 conn(0x7fe1980d0ed0 msgr2=0x7fe1980d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:10.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 -- 172.21.15.175:0/735546475 shutdown_connections 2024-04-20T10:27:10.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.043+0000 7fe19ee00640 1 -- 172.21.15.175:0/735546475 wait complete. 2024-04-20T10:27:10.044 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.045+0000 7fe19ee00640 1 Processor -- start 2024-04-20T10:27:10.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.045+0000 7fe19ee00640 1 -- start start 2024-04-20T10:27:10.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe19806c010 0x7fe1981537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198161a50 0x7fe1980c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1980c9000 0x7fe1980ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe1980d0600 con 0x7fe198161a50 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1980d0480 con 0x7fe19806c010 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe198161e30 con 0x7fe1980c9000 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe19806c010 0x7fe1981537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1980c9000 0x7fe1980ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1980c9000 0x7fe1980ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52236/0 (socket says 172.21.15.175:52236) 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19e400640 1 -- 172.21.15.175:0/3887316933 learned_addr learned my addr 172.21.15.175:0/3887316933 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.046+0000 7fe19d000640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198161a50 0x7fe1980c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19da00640 1 -- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1980c9000 msgr2=0x7fe1980ad250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19da00640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1980c9000 0x7fe1980ad250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19da00640 1 -- 172.21.15.175:0/3887316933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198161a50 msgr2=0x7fe1980c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19da00640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198161a50 0x7fe1980c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19da00640 1 -- 172.21.15.175:0/3887316933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1980c9630 con 0x7fe19806c010 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19e400640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1980c9000 0x7fe1980ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:10.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19da00640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe19806c010 0x7fe1981537b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c390 tx=0x7fe18c00c850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe195600640 1 -- 172.21.15.175:0/3887316933 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c017070 con 0x7fe19806c010 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe195600640 1 -- 172.21.15.175:0/3887316933 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe18c0052d0 con 0x7fe19806c010 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe195600640 1 -- 172.21.15.175:0/3887316933 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c003830 con 0x7fe19806c010 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1980c9910 con 0x7fe19806c010 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.047+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe198160f90 con 0x7fe19806c010 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.048+0000 7fe195600640 1 -- 172.21.15.175:0/3887316933 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe18c01a070 con 0x7fe19806c010 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.048+0000 7fe195600640 1 -- 172.21.15.175:0/3887316933 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fe18c011590 con 0x7fe19806c010 2024-04-20T10:27:10.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.048+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe198075b30 con 0x7fe19806c010 2024-04-20T10:27:10.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.051+0000 7fe195600640 1 -- 172.21.15.175:0/3887316933 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7fe18c021070 con 0x7fe19806c010 2024-04-20T10:27:10.101 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:10.101+0000 7f8bfcf53040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:27:10.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.130+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ead50013-a94f-44b0-9051-4e01a5858c4e", "id": 4} v 0) -- 0x7fe1980d3310 con 0x7fe19806c010 2024-04-20T10:27:10.137 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.138+0000 7fe195600640 1 -- 172.21.15.175:0/3887316933 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ead50013-a94f-44b0-9051-4e01a5858c4e", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7fe18c011ae0 con 0x7fe19806c010 2024-04-20T10:27:10.137 INFO:teuthology.orchestra.run.smithi175.stdout:4 2024-04-20T10:27:10.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe19806c010 msgr2=0x7fe1981537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe19806c010 0x7fe1981537b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c390 tx=0x7fe18c00c850 comp rx=0 tx=0).stop 2024-04-20T10:27:10.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 shutdown_connections 2024-04-20T10:27:10.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1980c9000 0x7fe1980ad250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe198161a50 0x7fe1980c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 --2- 172.21.15.175:0/3887316933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe19806c010 0x7fe1981537b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 >> 172.21.15.175:0/3887316933 conn(0x7fe1980d0ed0 msgr2=0x7fe1980b7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:10.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 shutdown_connections 2024-04-20T10:27:10.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.139+0000 7fe19ee00640 1 -- 172.21.15.175:0/3887316933 wait complete. 2024-04-20T10:27:10.154 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:10.153+0000 7fe674a21040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-20T10:27:10.155 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 5335fadb-0129-4fc5-a293-1c3a317097b3 5 2024-04-20T10:27:10.203 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:10.198+0000 7f8bfcf53040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:27:10.226 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:10.226+0000 7fe674a21040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-20T10:27:10.263 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:10.263+0000 7f8bfcf53040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:27:10.294 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:10.294+0000 7fe674a21040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-20T10:27:10.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.378+0000 7f6e53200640 1 Processor -- start 2024-04-20T10:27:10.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.378+0000 7f6e53200640 1 -- start start 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.379+0000 7f6e53200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e5406c010 0x7f6e540b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e53200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e540d0980 con 0x7f6e54161a50 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e53200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e54162490 con 0x7f6e540d3b20 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e53200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e540d0c60 con 0x7f6e5406c010 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e52800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e5406c010 0x7f6e540b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e52800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e5406c010 0x7f6e540b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47418/0 (socket says 172.21.15.175:47418) 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e52800640 1 -- 172.21.15.175:0/1981059661 learned_addr learned my addr 172.21.15.175:0/1981059661 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e52800640 1 -- 172.21.15.175:0/1981059661 >> v1:172.21.15.175:6789/0 conn(0x7f6e540d3b20 legacy=0x7f6e540ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e52800640 1 -- 172.21.15.175:0/1981059661 >> v1:172.21.15.47:6789/0 conn(0x7f6e54161a50 legacy=0x7f6e5406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.380+0000 7f6e52800640 1 -- 172.21.15.175:0/1981059661 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e54162770 con 0x7f6e5406c010 2024-04-20T10:27:10.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.382+0000 7f6e52800640 1 --2- 172.21.15.175:0/1981059661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5406c010 0x7f6e540b3180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e80 tx=0x7f6e4c02bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b996e2dadbb66fdd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.382+0000 7f6e50a00640 1 -- 172.21.15.175:0/1981059661 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c02cb90 con 0x7f6e5406c010 2024-04-20T10:27:10.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.382+0000 7f6e50a00640 1 -- 172.21.15.175:0/1981059661 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e4c02cd30 con 0x7f6e5406c010 2024-04-20T10:27:10.382 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.382+0000 7f6e50a00640 1 -- 172.21.15.175:0/1981059661 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c034610 con 0x7f6e5406c010 2024-04-20T10:27:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.383+0000 7f6e53200640 1 -- 172.21.15.175:0/1981059661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5406c010 msgr2=0x7f6e540b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.383+0000 7f6e53200640 1 --2- 172.21.15.175:0/1981059661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5406c010 0x7f6e540b3180 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e80 tx=0x7f6e4c02bff0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.383+0000 7f6e53200640 1 -- 172.21.15.175:0/1981059661 shutdown_connections 2024-04-20T10:27:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.383+0000 7f6e53200640 1 --2- 172.21.15.175:0/1981059661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5406c010 0x7f6e540b3180 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.383+0000 7f6e53200640 1 -- 172.21.15.175:0/1981059661 >> 172.21.15.175:0/1981059661 conn(0x7f6e540d0ed0 msgr2=0x7f6e540d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.383+0000 7f6e53200640 1 -- 172.21.15.175:0/1981059661 shutdown_connections 2024-04-20T10:27:10.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.383+0000 7f6e53200640 1 -- 172.21.15.175:0/1981059661 wait complete. 2024-04-20T10:27:10.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.385+0000 7f6e53200640 1 Processor -- start 2024-04-20T10:27:10.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.385+0000 7f6e53200640 1 -- start start 2024-04-20T10:27:10.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e53200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5406c010 0x7f6e540b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e53200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54161a50 0x7f6e540cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e53200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e540ccd50 0x7f6e540c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e53200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e54162ae0 con 0x7f6e540ccd50 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e53200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e540d0690 con 0x7f6e54161a50 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e53200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e54161e30 con 0x7f6e5406c010 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e51400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54161a50 0x7f6e540cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e51e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5406c010 0x7f6e540b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e51400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54161a50 0x7f6e540cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47424/0 (socket says 172.21.15.175:47424) 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e51400640 1 -- 172.21.15.175:0/2810221036 learned_addr learned my addr 172.21.15.175:0/2810221036 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.386+0000 7f6e52800640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e540ccd50 0x7f6e540c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e51400640 1 -- 172.21.15.175:0/2810221036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5406c010 msgr2=0x7f6e540b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e51400640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5406c010 0x7f6e540b26f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e51400640 1 -- 172.21.15.175:0/2810221036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e540ccd50 msgr2=0x7f6e540c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e51400640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e540ccd50 0x7f6e540c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e51400640 1 -- 172.21.15.175:0/2810221036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e540cd2d0 con 0x7f6e54161a50 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e51400640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54161a50 0x7f6e540cc810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c400 tx=0x7f6e4400c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e4a000640 1 -- 172.21.15.175:0/2810221036 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44017070 con 0x7f6e54161a50 2024-04-20T10:27:10.388 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e4a000640 1 -- 172.21.15.175:0/2810221036 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e440052d0 con 0x7f6e54161a50 2024-04-20T10:27:10.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e4a000640 1 -- 172.21.15.175:0/2810221036 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e44003830 con 0x7f6e54161a50 2024-04-20T10:27:10.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e540cd5b0 con 0x7f6e54161a50 2024-04-20T10:27:10.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e540cdb70 con 0x7f6e54161a50 2024-04-20T10:27:10.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.387+0000 7f6e4a000640 1 -- 172.21.15.175:0/2810221036 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e4401a070 con 0x7f6e54161a50 2024-04-20T10:27:10.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.388+0000 7f6e4a000640 1 -- 172.21.15.175:0/2810221036 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f6e44011590 con 0x7f6e54161a50 2024-04-20T10:27:10.389 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.388+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e540b5d20 con 0x7f6e54161a50 2024-04-20T10:27:10.390 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.391+0000 7f6e4a000640 1 -- 172.21.15.175:0/2810221036 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f6e540b5d20 con 0x7f6e54161a50 2024-04-20T10:27:10.398 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:10.399+0000 7f8bfcf53040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:27:10.430 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:10.430+0000 7fe674a21040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-20T10:27:10.469 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.470+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5335fadb-0129-4fc5-a293-1c3a317097b3", "id": 5} v 0) -- 0x7f6e54075b30 con 0x7f6e54161a50 2024-04-20T10:27:10.476 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.476+0000 7f6e4a000640 1 -- 172.21.15.175:0/2810221036 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5335fadb-0129-4fc5-a293-1c3a317097b3", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f6e44021020 con 0x7f6e54161a50 2024-04-20T10:27:10.476 INFO:teuthology.orchestra.run.smithi175.stdout:5 2024-04-20T10:27:10.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.477+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54161a50 msgr2=0x7f6e540cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.477+0000 7f6e53200640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54161a50 0x7f6e540cc810 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400c400 tx=0x7f6e4400c8c0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.477 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.477+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 shutdown_connections 2024-04-20T10:27:10.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.477+0000 7f6e53200640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e540ccd50 0x7f6e540c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.477+0000 7f6e53200640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e54161a50 0x7f6e540cc810 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.477+0000 7f6e53200640 1 --2- 172.21.15.175:0/2810221036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5406c010 0x7f6e540b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.478+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 >> 172.21.15.175:0/2810221036 conn(0x7f6e540d0ed0 msgr2=0x7f6e540b7f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:10.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.478+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 shutdown_connections 2024-04-20T10:27:10.478 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.478+0000 7f6e53200640 1 -- 172.21.15.175:0/2810221036 wait complete. 2024-04-20T10:27:10.494 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 869d63f5-c8d5-485c-a632-d0405aef964d 6 2024-04-20T10:27:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.675+0000 7f3957000640 1 Processor -- start 2024-04-20T10:27:10.675 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.675+0000 7f3957000640 1 -- start start 2024-04-20T10:27:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3957000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f39500d4270 0x7f39500d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3957000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39500d4650 0x7f39500b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3957000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f39500d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3957000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f39500d40c0 con 0x7f39500d4270 2024-04-20T10:27:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3957000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f39500d3820 con 0x7f39500d4650 2024-04-20T10:27:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3957000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f39500d3b00 con 0x7f39500b17d0 2024-04-20T10:27:10.678 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3947400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39500d4650 0x7f39500b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3955000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f39500d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3955000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f39500d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52256/0 (socket says 172.21.15.175:52256) 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3947400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f39500d4650 0x7f39500b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47428/0 (socket says 172.21.15.175:47428) 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.677+0000 7f3955000640 1 -- 172.21.15.175:0/2098845617 learned_addr learned my addr 172.21.15.175:0/2098845617 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.678+0000 7f394fe00640 1 --2- 172.21.15.175:0/2098845617 >> v2:172.21.15.47:3300/0 conn(0x7f39500d4270 0x7f39500d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.678+0000 7f3955000640 1 -- 172.21.15.175:0/2098845617 >> v2:172.21.15.175:3300/0 conn(0x7f39500d4650 msgr2=0x7f39500b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.678+0000 7f3955000640 1 --2- 172.21.15.175:0/2098845617 >> v2:172.21.15.175:3300/0 conn(0x7f39500d4650 0x7f39500b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.678+0000 7f3955000640 1 -- 172.21.15.175:0/2098845617 >> v2:172.21.15.47:3300/0 conn(0x7f39500d4270 msgr2=0x7f39500d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.678+0000 7f3955000640 1 --2- 172.21.15.175:0/2098845617 >> v2:172.21.15.47:3300/0 conn(0x7f39500d4270 0x7f39500d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.678+0000 7f3955000640 1 -- 172.21.15.175:0/2098845617 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39500d3de0 con 0x7f39500b17d0 2024-04-20T10:27:10.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.678+0000 7f3947400640 1 --2- 172.21.15.175:0/2098845617 >> v2:172.21.15.175:3300/0 conn(0x7f39500d4650 0x7f39500b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.679+0000 7f3955000640 1 --2- 172.21.15.175:0/2098845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f39500d0380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39500b8060 tx=0x7f394802bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af96adbadc77aa5b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.679+0000 7f394f400640 1 -- 172.21.15.175:0/2098845617 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394802ca80 con 0x7f39500b17d0 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.679+0000 7f394f400640 1 -- 172.21.15.175:0/2098845617 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394802cc20 con 0x7f39500b17d0 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.679+0000 7f394f400640 1 -- 172.21.15.175:0/2098845617 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39480343f0 con 0x7f39500b17d0 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 -- 172.21.15.175:0/2098845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 msgr2=0x7f39500d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 --2- 172.21.15.175:0/2098845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f39500d0380 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f39500b8060 tx=0x7f394802bee0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 -- 172.21.15.175:0/2098845617 shutdown_connections 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 --2- 172.21.15.175:0/2098845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f39500d0380 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 --2- 172.21.15.175:0/2098845617 >> v2:172.21.15.175:3300/0 conn(0x7f39500d4650 0x7f39500b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 --2- 172.21.15.175:0/2098845617 >> v2:172.21.15.47:3300/0 conn(0x7f39500d4270 0x7f39500d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 -- 172.21.15.175:0/2098845617 >> 172.21.15.175:0/2098845617 conn(0x7f39500b99c0 msgr2=0x7f39500b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 -- 172.21.15.175:0/2098845617 shutdown_connections 2024-04-20T10:27:10.681 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.680+0000 7f3957000640 1 -- 172.21.15.175:0/2098845617 wait complete. 2024-04-20T10:27:10.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.682+0000 7f3957000640 1 Processor -- start 2024-04-20T10:27:10.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.682+0000 7f3957000640 1 -- start start 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3957000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f395014ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3957000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39500d4270 0x7f395014e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3957000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39500d4650 0x7f3950150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3957000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f39500d4ae0 con 0x7f39500d4270 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3957000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39500d53b0 con 0x7f39500d4650 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3957000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f39500d4c60 con 0x7f39500b17d0 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f394fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f395014ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3955000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39500d4650 0x7f3950150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f394fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f395014ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:52264/0 (socket says 172.21.15.175:52264) 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f394fe00640 1 -- 172.21.15.175:0/198521557 learned_addr learned my addr 172.21.15.175:0/198521557 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.683+0000 7f3955000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39500d4650 0x7f3950150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47432/0 (socket says 172.21.15.175:47432) 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.684+0000 7f3947400640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39500d4270 0x7f395014e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.684 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.684+0000 7f3955000640 1 -- 172.21.15.175:0/198521557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 msgr2=0x7f395014ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.684+0000 7f3955000640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f395014ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.684+0000 7f3955000640 1 -- 172.21.15.175:0/198521557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39500d4270 msgr2=0x7f395014e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.684+0000 7f3955000640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39500d4270 0x7f395014e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.684+0000 7f3955000640 1 -- 172.21.15.175:0/198521557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395006d8c0 con 0x7f39500d4650 2024-04-20T10:27:10.685 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.684+0000 7f3955000640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39500d4650 0x7f3950150aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3948032000 tx=0x7f3948004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.685+0000 7f394cc00640 1 -- 172.21.15.175:0/198521557 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948043020 con 0x7f39500d4650 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.685+0000 7f394cc00640 1 -- 172.21.15.175:0/198521557 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f394803e450 con 0x7f39500d4650 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.685+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395006dba0 con 0x7f39500d4650 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.685+0000 7f394cc00640 1 -- 172.21.15.175:0/198521557 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948009510 con 0x7f39500d4650 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.685+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f395014e670 con 0x7f39500d4650 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.685+0000 7f394cc00640 1 -- 172.21.15.175:0/198521557 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3948046020 con 0x7f39500d4650 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.685+0000 7f394cc00640 1 -- 172.21.15.175:0/198521557 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f394802ca90 con 0x7f39500d4650 2024-04-20T10:27:10.686 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.686+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3950075c20 con 0x7f39500d4650 2024-04-20T10:27:10.688 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.688+0000 7f394cc00640 1 -- 172.21.15.175:0/198521557 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f3950075c20 con 0x7f39500d4650 2024-04-20T10:27:10.767 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.767+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "869d63f5-c8d5-485c-a632-d0405aef964d", "id": 6} v 0) -- 0x7f3950068980 con 0x7f39500d4650 2024-04-20T10:27:10.773 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.774+0000 7f394cc00640 1 -- 172.21.15.175:0/198521557 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "869d63f5-c8d5-485c-a632-d0405aef964d", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f394804d030 con 0x7f39500d4650 2024-04-20T10:27:10.773 INFO:teuthology.orchestra.run.smithi175.stdout:6 2024-04-20T10:27:10.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.775+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39500d4650 msgr2=0x7f3950150aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.775+0000 7f3957000640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39500d4650 0x7f3950150aa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f3948032000 tx=0x7f3948004a40 comp rx=0 tx=0).stop 2024-04-20T10:27:10.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.775+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 shutdown_connections 2024-04-20T10:27:10.775 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.775+0000 7f3957000640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39500d4650 0x7f3950150aa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.775+0000 7f3957000640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f39500d4270 0x7f395014e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.775+0000 7f3957000640 1 --2- 172.21.15.175:0/198521557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39500b17d0 0x7f395014ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.775+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 >> 172.21.15.175:0/198521557 conn(0x7f39500b99c0 msgr2=0x7f39500b6050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:10.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.776+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 shutdown_connections 2024-04-20T10:27:10.776 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.776+0000 7f3957000640 1 -- 172.21.15.175:0/198521557 wait complete. 2024-04-20T10:27:10.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 4bcce85d-9d35-460a-ae67-821f60715c81 7 2024-04-20T10:27:10.880 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:10.880+0000 7f8bfcf53040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:27:10.921 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:10.921+0000 7fe674a21040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-20T10:27:10.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.964+0000 7f1c93200640 1 Processor -- start 2024-04-20T10:27:10.964 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.965+0000 7f1c93200640 1 -- start start 2024-04-20T10:27:10.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c93200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1c9415e8c0 0x7f1c9406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.966 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c93200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1c940d1010 0x7f1c940b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c93200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c940d15b0 con 0x7f1c940d46f0 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c93200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1c940b6170 con 0x7f1c940d1010 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c93200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1c940d1890 con 0x7f1c9415e8c0 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c91e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1c9415e8c0 0x7f1c9406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c91e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1c9415e8c0 0x7f1c9406ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47444/0 (socket says 172.21.15.175:47444) 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c91e00640 1 -- 172.21.15.175:0/2741855596 learned_addr learned my addr 172.21.15.175:0/2741855596 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c91400640 1 --2- 172.21.15.175:0/2741855596 >> v2:172.21.15.47:3300/0 conn(0x7f1c940d1010 0x7f1c940b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.966+0000 7f1c91e00640 1 -- 172.21.15.175:0/2741855596 >> v2:172.21.15.47:3300/0 conn(0x7f1c940d1010 msgr2=0x7f1c940b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.967+0000 7f1c91e00640 1 --2- 172.21.15.175:0/2741855596 >> v2:172.21.15.47:3300/0 conn(0x7f1c940d1010 0x7f1c940b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.967+0000 7f1c91e00640 1 -- 172.21.15.175:0/2741855596 >> v1:172.21.15.175:6789/0 conn(0x7f1c940d46f0 legacy=0x7f1c940cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.967+0000 7f1c91e00640 1 -- 172.21.15.175:0/2741855596 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c940d1b70 con 0x7f1c9415e8c0 2024-04-20T10:27:10.967 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.967+0000 7f1c91400640 1 --2- 172.21.15.175:0/2741855596 >> v2:172.21.15.47:3300/0 conn(0x7f1c940d1010 0x7f1c940b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:10.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.968+0000 7f1c91e00640 1 --2- 172.21.15.175:0/2741855596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9415e8c0 0x7f1c9406ec40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c9408e650 tx=0x7f1c8402b9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c87e52f618fb3b41 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.968+0000 7f1c90a00640 1 -- 172.21.15.175:0/2741855596 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84034070 con 0x7f1c9415e8c0 2024-04-20T10:27:10.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.968+0000 7f1c90a00640 1 -- 172.21.15.175:0/2741855596 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c84030880 con 0x7f1c9415e8c0 2024-04-20T10:27:10.968 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.968+0000 7f1c90a00640 1 -- 172.21.15.175:0/2741855596 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84030c50 con 0x7f1c9415e8c0 2024-04-20T10:27:10.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 -- 172.21.15.175:0/2741855596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9415e8c0 msgr2=0x7f1c9406ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 --2- 172.21.15.175:0/2741855596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9415e8c0 0x7f1c9406ec40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1c9408e650 tx=0x7f1c8402b9c0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 -- 172.21.15.175:0/2741855596 shutdown_connections 2024-04-20T10:27:10.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 --2- 172.21.15.175:0/2741855596 >> v2:172.21.15.47:3300/0 conn(0x7f1c940d1010 0x7f1c940b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 --2- 172.21.15.175:0/2741855596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c9415e8c0 0x7f1c9406ec40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.969 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 -- 172.21.15.175:0/2741855596 >> 172.21.15.175:0/2741855596 conn(0x7f1c9409d6f0 msgr2=0x7f1c940b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:10.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 -- 172.21.15.175:0/2741855596 shutdown_connections 2024-04-20T10:27:10.970 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.969+0000 7f1c93200640 1 -- 172.21.15.175:0/2741855596 wait complete. 2024-04-20T10:27:10.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.972+0000 7f1c93200640 1 Processor -- start 2024-04-20T10:27:10.971 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.972+0000 7f1c93200640 1 -- start start 2024-04-20T10:27:10.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c93200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c940d1010 0x7f1c94161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c93200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c940d46f0 0x7f1c941601c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c93200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9415e8c0 0x7f1c940b4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c93200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1c940b6b40 con 0x7f1c9415e8c0 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c93200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c9406f180 con 0x7f1c940d1010 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c93200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c940b6cc0 con 0x7f1c940d46f0 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c940d1010 0x7f1c94161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.973+0000 7f1c91400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c940d46f0 0x7f1c941601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c940d1010 0x7f1c94161770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:47452/0 (socket says 172.21.15.175:47452) 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 -- 172.21.15.175:0/3781759286 learned_addr learned my addr 172.21.15.175:0/3781759286 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c92800640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9415e8c0 0x7f1c940b4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 -- 172.21.15.175:0/3781759286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c940d46f0 msgr2=0x7f1c941601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c940d46f0 0x7f1c941601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 -- 172.21.15.175:0/3781759286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9415e8c0 msgr2=0x7f1c940b4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9415e8c0 0x7f1c940b4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 -- 172.21.15.175:0/3781759286 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c941f7630 con 0x7f1c940d1010 2024-04-20T10:27:10.975 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c92800640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9415e8c0 0x7f1c940b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.974+0000 7f1c91e00640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c940d1010 0x7f1c94161770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c84000c00 tx=0x7f1c840046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.975+0000 7f1c8a000640 1 -- 172.21.15.175:0/3781759286 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c84034040 con 0x7f1c940d1010 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.975+0000 7f1c8a000640 1 -- 172.21.15.175:0/3781759286 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c840433f0 con 0x7f1c940d1010 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.975+0000 7f1c8a000640 1 -- 172.21.15.175:0/3781759286 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c840437e0 con 0x7f1c940d1010 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.975+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c941f7910 con 0x7f1c940d1010 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.975+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c941f7ed0 con 0x7f1c940d1010 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.975+0000 7f1c8a000640 1 -- 172.21.15.175:0/3781759286 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1c84037070 con 0x7f1c940d1010 2024-04-20T10:27:10.976 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.975+0000 7f1c8a000640 1 -- 172.21.15.175:0/3781759286 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7f1c84030740 con 0x7f1c940d1010 2024-04-20T10:27:10.977 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.976+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c94075b30 con 0x7f1c940d1010 2024-04-20T10:27:10.979 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:10.980+0000 7f1c8a000640 1 -- 172.21.15.175:0/3781759286 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f1c8403e070 con 0x7f1c940d1010 2024-04-20T10:27:11.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.058+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4bcce85d-9d35-460a-ae67-821f60715c81", "id": 7} v 0) -- 0x7f1c94074dd0 con 0x7f1c940d1010 2024-04-20T10:27:11.066 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.066+0000 7f1c8a000640 1 -- 172.21.15.175:0/3781759286 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4bcce85d-9d35-460a-ae67-821f60715c81", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7f1c84043980 con 0x7f1c940d1010 2024-04-20T10:27:11.066 INFO:teuthology.orchestra.run.smithi175.stdout:7 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c940d1010 msgr2=0x7f1c94161770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c940d1010 0x7f1c94161770 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1c84000c00 tx=0x7f1c840046c0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 shutdown_connections 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c9415e8c0 0x7f1c940b4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c940d46f0 0x7f1c941601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 --2- 172.21.15.175:0/3781759286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c940d1010 0x7f1c94161770 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 >> 172.21.15.175:0/3781759286 conn(0x7f1c9409d6f0 msgr2=0x7f1c940d1f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 shutdown_connections 2024-04-20T10:27:11.068 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:27:11.068+0000 7f1c93200640 1 -- 172.21.15.175:0/3781759286 wait complete. 2024-04-20T10:27:11.084 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-20T10:27:11.084 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-20T10:27:11.091 INFO:tasks.ceph.osd.0:Started 2024-04-20T10:27:11.092 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-20T10:27:11.092 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-20T10:27:11.101 INFO:tasks.ceph.osd.1:Started 2024-04-20T10:27:11.101 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-20T10:27:11.101 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-20T10:27:11.106 INFO:tasks.ceph.osd.2:Started 2024-04-20T10:27:11.106 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-20T10:27:11.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-20T10:27:11.111 INFO:tasks.ceph.osd.3:Started 2024-04-20T10:27:11.111 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-20T10:27:11.112 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-20T10:27:11.117 INFO:tasks.ceph.osd.4:Started 2024-04-20T10:27:11.117 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-20T10:27:11.118 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-20T10:27:11.123 INFO:tasks.ceph.osd.5:Started 2024-04-20T10:27:11.123 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-20T10:27:11.123 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-20T10:27:11.127 INFO:tasks.ceph.osd.6:Started 2024-04-20T10:27:11.128 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-20T10:27:11.128 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-20T10:27:11.133 INFO:tasks.ceph.osd.7:Started 2024-04-20T10:27:11.134 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:11.207 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:11.207+0000 7f8bfcf53040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:27:11.279 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:11.279+0000 7fe674a21040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-20T10:27:11.292 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:11.292+0000 7f8bfcf53040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:27:11.348 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:11.347+0000 7fe674a21040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-20T10:27:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.362+0000 7feb7be00640 1 Processor -- start 2024-04-20T10:27:11.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.362+0000 7feb7be00640 1 -- start start 2024-04-20T10:27:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.363+0000 7feb7be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feb7c06cde0 0x7feb7c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.363+0000 7feb7be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0b3330 0x7feb7c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.363+0000 7feb7be00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb7c06c190 con 0x7feb7c161d50 2024-04-20T10:27:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.363+0000 7feb7be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7feb7c162130 con 0x7feb7c06cde0 2024-04-20T10:27:11.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.363+0000 7feb7be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c06c470 con 0x7feb7c0b3330 2024-04-20T10:27:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.363+0000 7feb7aa00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feb7c161d50 0x7feb7c06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58970/0 (socket says 172.21.15.47:58970) 2024-04-20T10:27:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.363+0000 7feb7aa00640 1 -- 172.21.15.47:0/3964074543 learned_addr learned my addr 172.21.15.47:0/3964074543 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:11.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb7a000640 1 --2- 172.21.15.47:0/3964074543 >> v2:172.21.15.175:3300/0 conn(0x7feb7c06cde0 0x7feb7c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb7b400640 1 --2- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0b3330 0x7feb7c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb79600640 1 -- 172.21.15.47:0/3964074543 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 985667581 0 0) 0x7feb7c06c190 con 0x7feb7c161d50 2024-04-20T10:27:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb79600640 1 -- 172.21.15.47:0/3964074543 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7feb60003520 con 0x7feb7c161d50 2024-04-20T10:27:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb7a000640 1 -- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0b3330 msgr2=0x7feb7c0b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb7a000640 1 --2- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0b3330 0x7feb7c0b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb7a000640 1 -- 172.21.15.47:0/3964074543 >> v1:172.21.15.47:6789/0 conn(0x7feb7c161d50 legacy=0x7feb7c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:11.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.364+0000 7feb7a000640 1 -- 172.21.15.47:0/3964074543 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c0d2e00 con 0x7feb7c06cde0 2024-04-20T10:27:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.365+0000 7feb7a000640 1 --2- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c06cde0 0x7feb7c0d38e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009d90 tx=0x7feb6c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21df4a6f0da2f721 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.365+0000 7feb79600640 1 -- 172.21.15.47:0/3964074543 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c039070 con 0x7feb7c06cde0 2024-04-20T10:27:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.365+0000 7feb79600640 1 -- 172.21.15.47:0/3964074543 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb6c02cc50 con 0x7feb7c06cde0 2024-04-20T10:27:11.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.365+0000 7feb79600640 1 -- 172.21.15.47:0/3964074543 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb6c034440 con 0x7feb7c06cde0 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 -- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c06cde0 msgr2=0x7feb7c0d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 --2- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c06cde0 0x7feb7c0d38e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009d90 tx=0x7feb6c02bea0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 -- 172.21.15.47:0/3964074543 shutdown_connections 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 --2- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c0b3330 0x7feb7c0b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 --2- 172.21.15.47:0/3964074543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c06cde0 0x7feb7c0d38e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 -- 172.21.15.47:0/3964074543 >> 172.21.15.47:0/3964074543 conn(0x7feb7c0d4450 msgr2=0x7feb7c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 -- 172.21.15.47:0/3964074543 shutdown_connections 2024-04-20T10:27:11.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.366+0000 7feb7be00640 1 -- 172.21.15.47:0/3964074543 wait complete. 2024-04-20T10:27:11.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.368+0000 7feb7be00640 1 Processor -- start 2024-04-20T10:27:11.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.368+0000 7feb7be00640 1 -- start start 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7c06cde0 0x7feb7c0aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c0b3330 0x7feb7c0ab330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c161d50 0x7feb7c0ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c0b7660 con 0x7feb7c06cde0 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c0d33c0 con 0x7feb7c0b3330 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c0b77e0 con 0x7feb7c161d50 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7c06cde0 0x7feb7c0aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7c06cde0 0x7feb7c0aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41086/0 (socket says 172.21.15.47:41086) 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7aa00640 1 -- 172.21.15.47:0/2231124405 learned_addr learned my addr 172.21.15.47:0/2231124405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7b400640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c161d50 0x7feb7c0ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.369+0000 7feb7a000640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c0b3330 0x7feb7c0ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb7aa00640 1 -- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c161d50 msgr2=0x7feb7c0ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:11.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb7aa00640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c161d50 0x7feb7c0ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb7aa00640 1 -- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c0b3330 msgr2=0x7feb7c0ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb7aa00640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c0b3330 0x7feb7c0ab330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb7aa00640 1 -- 172.21.15.47:0/2231124405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c0c8610 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb7aa00640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7c06cde0 0x7feb7c0aadf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feb640010e0 tx=0x7feb6400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb72a00640 1 -- 172.21.15.47:0/2231124405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64017070 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb72a00640 1 -- 172.21.15.47:0/2231124405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb64005d00 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb72a00640 1 -- 172.21.15.47:0/2231124405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb64007310 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.370+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c0c88b0 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.371+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c0c8e70 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.371+0000 7feb72a00640 1 -- 172.21.15.47:0/2231124405 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7feb6401a070 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.371+0000 7feb72a00640 1 -- 172.21.15.47:0/2231124405 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7feb640074b0 con 0x7feb7c06cde0 2024-04-20T10:27:11.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.372+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c075b60 con 0x7feb7c06cde0 2024-04-20T10:27:11.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.374+0000 7feb72a00640 1 -- 172.21.15.47:0/2231124405 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7feb64021020 con 0x7feb7c06cde0 2024-04-20T10:27:11.429 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:11.430+0000 7f8bfcf53040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:27:11.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.441+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb7c0cb2e0 con 0x7feb7c06cde0 2024-04-20T10:27:11.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.442+0000 7feb72a00640 1 -- 172.21.15.47:0/2231124405 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7feb6401c440 con 0x7feb7c06cde0 2024-04-20T10:27:11.442 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:11.442 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":10,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:11.060578+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7c06cde0 msgr2=0x7feb7c0aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7c06cde0 0x7feb7c0aadf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feb640010e0 tx=0x7feb6400c890 comp rx=0 tx=0).stop 2024-04-20T10:27:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 shutdown_connections 2024-04-20T10:27:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb7c161d50 0x7feb7c0ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb7c0b3330 0x7feb7c0ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 --2- 172.21.15.47:0/2231124405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7c06cde0 0x7feb7c0aadf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 >> 172.21.15.47:0/2231124405 conn(0x7feb7c0d4450 msgr2=0x7feb7c15fe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 shutdown_connections 2024-04-20T10:27:11.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:11.443+0000 7feb7be00640 1 -- 172.21.15.47:0/2231124405 wait complete. 2024-04-20T10:27:11.459 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-20T10:27:11.459 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-20T10:27:11.491 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:11.491+0000 7fe674a21040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-20T10:27:11.752 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:11.752+0000 7f8bfcf53040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:27:11.805 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T10:27:11.805+0000 7f461da2c640 -1 Falling back to public interface 2024-04-20T10:27:11.806 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T10:27:11.806+0000 7f32ca20c640 -1 Falling back to public interface 2024-04-20T10:27:11.808 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T10:27:11.808+0000 7f461bc00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6806/0: (98) Address already in use 2024-04-20T10:27:11.809 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-20T10:27:11.809+0000 7f30e8a0c640 -1 Falling back to public interface 2024-04-20T10:27:11.810 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:11.810+0000 7f8bfcf53040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:27:11.810 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T10:27:11.810+0000 7fbffbd14640 -1 Falling back to public interface 2024-04-20T10:27:11.814 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T10:27:11.815+0000 7f8299dff640 -1 Falling back to public interface 2024-04-20T10:27:11.816 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T10:27:11.816+0000 7f2e3cc0c640 -1 Falling back to public interface 2024-04-20T10:27:11.831 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T10:27:11.831+0000 7fb4fb00f640 -1 Falling back to public interface 2024-04-20T10:27:11.836 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T10:27:11.837+0000 7f39ff030640 -1 Falling back to public interface 2024-04-20T10:27:11.842 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:11.842+0000 7fe674a21040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-20T10:27:11.901 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:11.901+0000 7fe674a21040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-20T10:27:11.997 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:11.997+0000 7f8bfcf53040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:27:12.094 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:12.093+0000 7fe674a21040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-20T10:27:12.165 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:12.166+0000 7f8bfcf53040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:27:12.223 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:12.224+0000 7f8bfcf53040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:27:12.265 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:12.272 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:12.270+0000 7fe674a21040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-20T10:27:12.302 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:12.303+0000 7f8bfcf53040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:27:12.332 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:12.332+0000 7fe674a21040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-20T10:27:12.354 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:12.354+0000 7f8bfcf53040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:27:12.420 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:12.420+0000 7fe674a21040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-20T10:27:12.427 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.427+0000 7f540da00640 1 Processor -- start 2024-04-20T10:27:12.428 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.428+0000 7f540da00640 1 -- start start 2024-04-20T10:27:12.429 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.429+0000 7f540da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f540815e930 0x7f54080d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:12.429 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.429+0000 7f540da00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54080d0e30 con 0x7f54080d09a0 2024-04-20T10:27:12.429 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.429+0000 7f540da00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f54080d1b30 con 0x7f54080d4a60 2024-04-20T10:27:12.430 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.429+0000 7f540da00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f54080d27b0 con 0x7f540815e930 2024-04-20T10:27:12.430 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.429+0000 7f5407400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f54080d09a0 0x7f54080d2c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:58978/0 (socket says 172.21.15.47:58978) 2024-04-20T10:27:12.430 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.429+0000 7f5407400640 1 -- 172.21.15.47:0/3110889075 learned_addr learned my addr 172.21.15.47:0/3110889075 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:12.430 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.430+0000 7f5406000640 1 --2- 172.21.15.47:0/3110889075 >> v2:172.21.15.175:3300/0 conn(0x7f540815e930 0x7f54080d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:12.430 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.430+0000 7f5405600640 1 -- 172.21.15.47:0/3110889075 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1194198364 0 0) 0x7f53f8001110 con 0x7f54080d09a0 2024-04-20T10:27:12.431 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.430+0000 7f5405600640 1 -- 172.21.15.47:0/3110889075 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53ec003520 con 0x7f54080d09a0 2024-04-20T10:27:12.431 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.430+0000 7f5406000640 1 -- 172.21.15.47:0/3110889075 >> v1:172.21.15.175:6789/0 conn(0x7f54080d4a60 legacy=0x7f54080d4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.431 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.430+0000 7f5406000640 1 -- 172.21.15.47:0/3110889075 >> v1:172.21.15.47:6789/0 conn(0x7f54080d09a0 legacy=0x7f54080d2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.431 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.430+0000 7f5406000640 1 -- 172.21.15.47:0/3110889075 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54080d1e10 con 0x7f540815e930 2024-04-20T10:27:12.431 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.431+0000 7f5406000640 1 --2- 172.21.15.47:0/3110889075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540815e930 0x7f54080d1450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097a0 tx=0x7f53f002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3fee825d87e0757 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:12.432 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.432+0000 7f5405600640 1 -- 172.21.15.47:0/3110889075 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f0034070 con 0x7f540815e930 2024-04-20T10:27:12.432 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.432+0000 7f5405600640 1 -- 172.21.15.47:0/3110889075 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f0030610 con 0x7f540815e930 2024-04-20T10:27:12.432 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.432+0000 7f5405600640 1 -- 172.21.15.47:0/3110889075 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f00309c0 con 0x7f540815e930 2024-04-20T10:27:12.433 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.432+0000 7f540da00640 1 -- 172.21.15.47:0/3110889075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540815e930 msgr2=0x7f54080d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.433 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.432+0000 7f540da00640 1 --2- 172.21.15.47:0/3110889075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540815e930 0x7f54080d1450 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f53f00097a0 tx=0x7f53f002b960 comp rx=0 tx=0).stop 2024-04-20T10:27:12.433 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.432+0000 7f540da00640 1 -- 172.21.15.47:0/3110889075 shutdown_connections 2024-04-20T10:27:12.433 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.433+0000 7f540da00640 1 --2- 172.21.15.47:0/3110889075 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f540815e930 0x7f54080d1450 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.433 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.433+0000 7f540da00640 1 -- 172.21.15.47:0/3110889075 >> 172.21.15.47:0/3110889075 conn(0x7f54080b9730 msgr2=0x7f54080b9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:12.433 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.433+0000 7f540da00640 1 -- 172.21.15.47:0/3110889075 shutdown_connections 2024-04-20T10:27:12.433 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.433+0000 7f540da00640 1 -- 172.21.15.47:0/3110889075 wait complete. 2024-04-20T10:27:12.434 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.434+0000 7f540da00640 1 Processor -- start 2024-04-20T10:27:12.434 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.434+0000 7f540da00640 1 -- start start 2024-04-20T10:27:12.436 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.435+0000 7f540da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54080d09a0 0x7f5408158c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:12.436 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.435+0000 7f540da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54080d4a60 0x7f54080c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:12.436 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.435+0000 7f540da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540815e930 0x7f540806cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:12.436 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.435+0000 7f540da00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f54080d3350 con 0x7f540815e930 2024-04-20T10:27:12.436 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.435+0000 7f540da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54080d2440 con 0x7f54080d09a0 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.435+0000 7f540da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54080d34d0 con 0x7f54080d4a60 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540815e930 0x7f540806cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540815e930 0x7f540806cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41098/0 (socket says 172.21.15.47:41098) 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 -- 172.21.15.47:0/3353441582 learned_addr learned my addr 172.21.15.47:0/3353441582 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5406a00640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54080d09a0 0x7f5408158c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5406000640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54080d4a60 0x7f54080c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 -- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54080d4a60 msgr2=0x7f54080c8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54080d4a60 0x7f54080c8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 -- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54080d09a0 msgr2=0x7f5408158c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54080d09a0 0x7f5408158c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 -- 172.21.15.47:0/3353441582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540806bab0 con 0x7f540815e930 2024-04-20T10:27:12.437 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.436+0000 7f5407400640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540815e930 0x7f540806cb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53f800c730 tx=0x7f53f800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:12.438 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.437+0000 7f53fea00640 1 -- 172.21.15.47:0/3353441582 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8017070 con 0x7f540815e930 2024-04-20T10:27:12.438 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.437+0000 7f53fea00640 1 -- 172.21.15.47:0/3353441582 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53f80058b0 con 0x7f540815e930 2024-04-20T10:27:12.439 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.437+0000 7f53fea00640 1 -- 172.21.15.47:0/3353441582 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8005ca0 con 0x7f540815e930 2024-04-20T10:27:12.439 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.437+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540806bd90 con 0x7f540815e930 2024-04-20T10:27:12.439 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.437+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f540806c350 con 0x7f540815e930 2024-04-20T10:27:12.439 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.437+0000 7f53fea00640 1 -- 172.21.15.47:0/3353441582 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f53f801a070 con 0x7f540815e930 2024-04-20T10:27:12.439 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.437+0000 7f53fea00640 1 -- 172.21.15.47:0/3353441582 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f53f800aa50 con 0x7f540815e930 2024-04-20T10:27:12.439 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.438+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5408069430 con 0x7f540815e930 2024-04-20T10:27:12.441 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.441+0000 7f53fea00640 1 -- 172.21.15.47:0/3353441582 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73276 (secure 0 0 0) 0x7f53f8021020 con 0x7f540815e930 2024-04-20T10:27:12.488 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:12.488+0000 7fe674a21040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-20T10:27:12.517 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.518+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f54080d2f00 con 0x7f540815e930 2024-04-20T10:27:12.518 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.518+0000 7f53fea00640 1 -- 172.21.15.47:0/3353441582 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f54080d0e30 con 0x7f540815e930 2024-04-20T10:27:12.519 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-20T10:27:12.519 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":10,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:11.060578+0000","last_up_change":"0.000000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540815e930 msgr2=0x7f540806cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540815e930 0x7f540806cb50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f53f800c730 tx=0x7f53f800cbf0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 shutdown_connections 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f540815e930 0x7f540806cb50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54080d4a60 0x7f54080c8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 --2- 172.21.15.47:0/3353441582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54080d09a0 0x7f5408158c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 >> 172.21.15.47:0/3353441582 conn(0x7f54080b9730 msgr2=0x7f54080cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:12.520 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 shutdown_connections 2024-04-20T10:27:12.521 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:12.520+0000 7f540da00640 1 -- 172.21.15.47:0/3353441582 wait complete. 2024-04-20T10:27:12.533 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-20T10:27:12.896 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:12.896+0000 7f8bfcf53040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:27:13.030 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:13.030+0000 7f8bfcf53040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:27:13.080 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:13.080+0000 7fe674a21040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-20T10:27:13.220 INFO:tasks.ceph.mgr.y.smithi047.stderr:2024-04-20T10:27:13.220+0000 7fe674a21040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-20T10:27:14.073 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:14.074+0000 7f8b9e000640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-20T10:27:14.077 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-04-20T10:27:14.078+0000 7f8b9e000640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-20T10:27:15.009 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T10:27:15.009+0000 7fb4fb00f640 -1 osd.4 0 log_to_monitors true 2024-04-20T10:27:15.193 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T10:27:15.194+0000 7fbffbd14640 -1 osd.5 0 log_to_monitors true 2024-04-20T10:27:15.404 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T10:27:15.404+0000 7f39ff030640 -1 osd.6 0 log_to_monitors true 2024-04-20T10:27:15.407 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T10:27:15.407+0000 7f32ca20c640 -1 osd.1 0 log_to_monitors true 2024-04-20T10:27:15.427 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T10:27:15.427+0000 7f461da2c640 -1 osd.0 0 log_to_monitors true 2024-04-20T10:27:15.438 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T10:27:15.438+0000 7f8299dff640 -1 osd.7 0 log_to_monitors true 2024-04-20T10:27:15.671 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T10:27:15.671+0000 7f2e3cc0c640 -1 osd.2 0 log_to_monitors true 2024-04-20T10:27:15.672 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-20T10:27:15.673+0000 7f30e8a0c640 -1 osd.3 0 log_to_monitors true 2024-04-20T10:27:17.170 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T10:27:17.170+0000 7fb4f5800640 -1 osd.4 0 waiting for initial osdmap 2024-04-20T10:27:17.226 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-04-20T10:27:17.225+0000 7fb4eb800640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:18.796 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T10:27:18.796+0000 7f2e38400640 -1 osd.2 0 waiting for initial osdmap 2024-04-20T10:27:18.803 INFO:tasks.ceph.osd.2.smithi047.stderr:2024-04-20T10:27:18.802+0000 7f2e2da00640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:18.859 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T10:27:18.858+0000 7f32c4c00640 -1 osd.1 0 waiting for initial osdmap 2024-04-20T10:27:18.867 INFO:tasks.ceph.osd.1.smithi047.stderr:2024-04-20T10:27:18.867+0000 7f32bac00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:18.867 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-20T10:27:18.867+0000 7f30e3200640 -1 osd.3 0 waiting for initial osdmap 2024-04-20T10:27:18.870 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T10:27:18.870+0000 7f4618000640 -1 osd.0 0 waiting for initial osdmap 2024-04-20T10:27:18.876 INFO:tasks.ceph.osd.0.smithi047.stderr:2024-04-20T10:27:18.876+0000 7f460e000640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:18.879 INFO:tasks.ceph.osd.3.smithi047.stderr:2024-04-20T10:27:18.879+0000 7f30d9200640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:18.932 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T10:27:18.932+0000 7fbff7600640 -1 osd.5 0 waiting for initial osdmap 2024-04-20T10:27:18.942 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-04-20T10:27:18.942+0000 7fbfecc00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:18.959 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T10:27:18.959+0000 7f8294c00640 -1 osd.7 0 waiting for initial osdmap 2024-04-20T10:27:18.969 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-04-20T10:27:18.968+0000 7f828ac00640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:18.993 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T10:27:18.993+0000 7f39f9400640 -1 osd.6 0 waiting for initial osdmap 2024-04-20T10:27:19.003 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-04-20T10:27:19.003+0000 7f39ef400640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-20T10:27:19.338 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:19.532 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.532+0000 7ffac9600640 1 Processor -- start 2024-04-20T10:27:19.532 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.532+0000 7ffac9600640 1 -- start start 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.533+0000 7ffac9600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffac415e900 0x7ffac40d1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.533+0000 7ffac9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac40d3c50 0x7ffac40d4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac9600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffac40d4f80 0x7ffac40d2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac9600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ffac40b6f60 con 0x7ffac415e900 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac9600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ffac415ece0 con 0x7ffac40d4f80 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffac40b61a0 con 0x7ffac40d3c50 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac2a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffac415e900 0x7ffac40d1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac2a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffac415e900 0x7ffac40d1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58414/0 (socket says 172.21.15.47:58414) 2024-04-20T10:27:19.534 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac2a00640 1 -- 172.21.15.47:0/3250329000 learned_addr learned my addr 172.21.15.47:0/3250329000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac2000640 1 --2- 172.21.15.47:0/3250329000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac40d3c50 0x7ffac40d4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac3400640 1 --2- 172.21.15.47:0/3250329000 >> v2:172.21.15.175:3300/0 conn(0x7ffac40d4f80 0x7ffac40d2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac2a00640 1 -- 172.21.15.47:0/3250329000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac40d3c50 msgr2=0x7ffac40d4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.534+0000 7ffac2a00640 1 --2- 172.21.15.47:0/3250329000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac40d3c50 0x7ffac40d4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.535+0000 7ffac2a00640 1 -- 172.21.15.47:0/3250329000 >> v2:172.21.15.175:3300/0 conn(0x7ffac40d4f80 msgr2=0x7ffac40d2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.535+0000 7ffac2a00640 1 --2- 172.21.15.47:0/3250329000 >> v2:172.21.15.175:3300/0 conn(0x7ffac40d4f80 0x7ffac40d2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.535+0000 7ffac2a00640 1 -- 172.21.15.47:0/3250329000 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac40b6480 con 0x7ffac415e900 2024-04-20T10:27:19.535 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.535+0000 7ffac2000640 1 --2- 172.21.15.47:0/3250329000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac40d3c50 0x7ffac40d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:19.536 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.536+0000 7ffac2a00640 1 --2- 172.21.15.47:0/3250329000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac415e900 0x7ffac40d1720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffac40cb5d0 tx=0x7ffab402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfed36747da8598 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:19.536 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.536+0000 7ffac1600640 1 -- 172.21.15.47:0/3250329000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab402ca80 con 0x7ffac415e900 2024-04-20T10:27:19.536 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.536+0000 7ffac1600640 1 -- 172.21.15.47:0/3250329000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab402cc20 con 0x7ffac415e900 2024-04-20T10:27:19.536 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.536+0000 7ffac1600640 1 -- 172.21.15.47:0/3250329000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab40343f0 con 0x7ffac415e900 2024-04-20T10:27:19.537 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 -- 172.21.15.47:0/3250329000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac415e900 msgr2=0x7ffac40d1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:19.537 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 --2- 172.21.15.47:0/3250329000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac415e900 0x7ffac40d1720 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffac40cb5d0 tx=0x7ffab402bee0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.537 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 -- 172.21.15.47:0/3250329000 shutdown_connections 2024-04-20T10:27:19.537 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 --2- 172.21.15.47:0/3250329000 >> v2:172.21.15.175:3300/0 conn(0x7ffac40d4f80 0x7ffac40d2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.538 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 --2- 172.21.15.47:0/3250329000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac40d3c50 0x7ffac40d4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.538 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 --2- 172.21.15.47:0/3250329000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac415e900 0x7ffac40d1720 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.538 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 -- 172.21.15.47:0/3250329000 >> 172.21.15.47:0/3250329000 conn(0x7ffac40b9720 msgr2=0x7ffac40b9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:19.538 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 -- 172.21.15.47:0/3250329000 shutdown_connections 2024-04-20T10:27:19.538 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.537+0000 7ffac9600640 1 -- 172.21.15.47:0/3250329000 wait complete. 2024-04-20T10:27:19.539 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.539+0000 7ffac9600640 1 Processor -- start 2024-04-20T10:27:19.539 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.539+0000 7ffac9600640 1 -- start start 2024-04-20T10:27:19.540 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.540+0000 7ffac9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac40d3c50 0x7ffac40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:19.540 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.540+0000 7ffac9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac40d4f80 0x7ffac414f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:19.540 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.540+0000 7ffac9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac415e900 0x7ffac406d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.540+0000 7ffac9600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffac40d1c60 con 0x7ffac40d3c50 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.540+0000 7ffac9600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffac40b6a40 con 0x7ffac40d4f80 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.540+0000 7ffac9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffac40d1de0 con 0x7ffac415e900 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac40d3c50 0x7ffac40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac40d3c50 0x7ffac40ce190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58418/0 (socket says 172.21.15.47:58418) 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 -- 172.21.15.47:0/4285567557 learned_addr learned my addr 172.21.15.47:0/4285567557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2000640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac40d4f80 0x7ffac414f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac3400640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac415e900 0x7ffac406d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 -- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac415e900 msgr2=0x7ffac406d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac415e900 0x7ffac406d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 -- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac40d4f80 msgr2=0x7ffac414f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:19.542 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac40d4f80 0x7ffac414f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 -- 172.21.15.47:0/4285567557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffac4153140 con 0x7ffac40d3c50 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac3400640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac415e900 0x7ffac406d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2000640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac40d4f80 0x7ffac414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.541+0000 7ffac2a00640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac40d3c50 0x7ffac40ce190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffab4000f80 tx=0x7ffab4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.542+0000 7ffabaa00640 1 -- 172.21.15.47:0/4285567557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4043020 con 0x7ffac40d3c50 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.542+0000 7ffabaa00640 1 -- 172.21.15.47:0/4285567557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffab403e450 con 0x7ffac40d3c50 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.542+0000 7ffabaa00640 1 -- 172.21.15.47:0/4285567557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffab4009510 con 0x7ffac40d3c50 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.542+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffac40cd3d0 con 0x7ffac40d3c50 2024-04-20T10:27:19.543 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.542+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffac40cd990 con 0x7ffac40d3c50 2024-04-20T10:27:19.544 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.543+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffac4069430 con 0x7ffac40d3c50 2024-04-20T10:27:19.544 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.544+0000 7ffabaa00640 1 -- 172.21.15.47:0/4285567557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 4) ==== 90984+0+0 (secure 0 0 0) 0x7ffab4046020 con 0x7ffac40d3c50 2024-04-20T10:27:19.544 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.545+0000 7ffabaa00640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffaa006c860 0x7ffaa006ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:19.545 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.545+0000 7ffabaa00640 1 -- 172.21.15.47:0/4285567557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7ffab40ad750 con 0x7ffac40d3c50 2024-04-20T10:27:19.545 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.545+0000 7ffac2000640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffaa006c860 0x7ffaa006ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:19.546 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.546+0000 7ffac2000640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffaa006c860 0x7ffaa006ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006fd0 tx=0x7ffaac008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:19.550 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.550+0000 7ffabaa00640 1 -- 172.21.15.47:0/4285567557 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffab404d030 con 0x7ffac40d3c50 2024-04-20T10:27:19.747 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.746+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffac406ee30 con 0x7ffac40d3c50 2024-04-20T10:27:19.748 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.748+0000 7ffabaa00640 1 -- 172.21.15.47:0/4285567557 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7567 (secure 0 0 0) 0x7ffab407c8c0 con 0x7ffac40d3c50 2024-04-20T10:27:19.748 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-20T10:27:19.748 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":14,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:18.179244+0000","last_up_change":"2024-04-20T10:27:18.179244+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:19.751 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffaa006c860 msgr2=0x7ffaa006ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:19.751 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffaa006c860 0x7ffaa006ed20 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006fd0 tx=0x7ffaac008040 comp rx=0 tx=0).stop 2024-04-20T10:27:19.751 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac40d3c50 msgr2=0x7ffac40ce190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:19.751 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac40d3c50 0x7ffac40ce190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ffab4000f80 tx=0x7ffab4004a40 comp rx=0 tx=0).stop 2024-04-20T10:27:19.751 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 shutdown_connections 2024-04-20T10:27:19.751 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffac415e900 0x7ffac406d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.751 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffac40d4f80 0x7ffac414f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.752 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffaa006c860 0x7ffaa006ed20 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.752 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 --2- 172.21.15.47:0/4285567557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffac40d3c50 0x7ffac40ce190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:19.752 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 >> 172.21.15.47:0/4285567557 conn(0x7ffac40b9720 msgr2=0x7ffac40cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:19.752 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 shutdown_connections 2024-04-20T10:27:19.752 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:19.751+0000 7ffac9600640 1 -- 172.21.15.47:0/4285567557 wait complete. 2024-04-20T10:27:19.764 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-04-20T10:27:26.570 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:26.769 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.769+0000 7f2df5c00640 1 Processor -- start 2024-04-20T10:27:26.769 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.769+0000 7f2df5c00640 1 -- start start 2024-04-20T10:27:26.771 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.770+0000 7f2df5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 0x7f2df0160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.771 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.770+0000 7f2df5c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2df00af560 0x7f2df015ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.771 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.770+0000 7f2df5c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2df015d790 con 0x7f2df01569b0 2024-04-20T10:27:26.771 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.770+0000 7f2df5c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2df015da70 con 0x7f2df00af560 2024-04-20T10:27:26.771 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.770+0000 7f2df5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2df01601f0 con 0x7f2df0156d90 2024-04-20T10:27:26.771 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.771+0000 7f2defe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2df01569b0 0x7f2df00aee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38794/0 (socket says 172.21.15.47:38794) 2024-04-20T10:27:26.771 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.771+0000 7f2defe00640 1 -- 172.21.15.47:0/1056433780 learned_addr learned my addr 172.21.15.47:0/1056433780 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:26.773 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.771+0000 7f2def400640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 0x7f2df0160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:26.773 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.771+0000 7f2df5200640 1 --2- 172.21.15.47:0/1056433780 >> v2:172.21.15.175:3300/0 conn(0x7f2df00af560 0x7f2df015ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:26.773 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.771+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1157137753 0 0) 0x7f2df015d790 con 0x7f2df01569b0 2024-04-20T10:27:26.773 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.771+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2dd0003520 con 0x7f2df01569b0 2024-04-20T10:27:26.773 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.771+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1253350046 0 0) 0x7f2dd0003520 con 0x7f2df01569b0 2024-04-20T10:27:26.773 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 msgr2=0x7f2df0160cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:26.773 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 0x7f2df0160cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 >> v2:172.21.15.175:3300/0 conn(0x7f2df00af560 msgr2=0x7f2df015ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 --2- 172.21.15.47:0/1056433780 >> v2:172.21.15.175:3300/0 conn(0x7f2df00af560 0x7f2df015ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01604d0 con 0x7f2df01569b0 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f2de0001670 con 0x7f2df01569b0 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 >> v1:172.21.15.47:6789/0 conn(0x7f2df01569b0 legacy=0x7f2df00aee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd00039b0 0x7f2dd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd0004800 0x7f2dd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 0x7f2dd00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2df015da70 con 0x7f2dd0004800 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01601f0 con 0x7f2dd00039b0 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dd000fdd0 con 0x7f2df0156d90 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2def400640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd0004800 0x7f2dd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2df5200640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 0x7f2dd00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:26.774 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2defe00640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd00039b0 0x7f2dd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:26.775 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2def400640 1 -- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 msgr2=0x7f2dd00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:26.775 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2def400640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2df0156d90 0x7f2dd00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:26.775 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2def400640 1 -- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd00039b0 msgr2=0x7f2dd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:26.775 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2def400640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd00039b0 0x7f2dd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:26.775 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.772+0000 7f2def400640 1 -- 172.21.15.47:0/1056433780 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd0003520 con 0x7f2dd0004800 2024-04-20T10:27:26.775 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.773+0000 7f2defe00640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dd00039b0 0x7f2dd0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:26.776 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.774+0000 7f2def400640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd0004800 0x7f2dd0004c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009af0 tx=0x7f2dd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48846308b52759a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:26.776 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.774+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8039070 con 0x7f2dd0004800 2024-04-20T10:27:26.776 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.774+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd802c6b0 con 0x7f2dd0004800 2024-04-20T10:27:26.776 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.774+0000 7f2deea00640 1 -- 172.21.15.47:0/1056433780 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8009690 con 0x7f2dd0004800 2024-04-20T10:27:26.776 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.775+0000 7f2df5c00640 1 -- 172.21.15.47:0/1056433780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd0004800 msgr2=0x7f2dd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:26.776 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.775+0000 7f2df5c00640 1 --2- 172.21.15.47:0/1056433780 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dd0004800 0x7f2dd0004c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8009af0 tx=0x7f2dd0005830 comp rx=0 tx=0).stop 2024-04-20T10:27:26.776 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.775+0000 7f2df5200640 1 -- 172.21.15.47:0/1056433780 reap_dead start 2024-04-20T10:27:26.777 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.775+0000 7f2df5c00640 1 -- 172.21.15.47:0/1056433780 shutdown_connections 2024-04-20T10:27:26.777 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.775+0000 7f2df5c00640 1 -- 172.21.15.47:0/1056433780 >> 172.21.15.47:0/1056433780 conn(0x7f2df00b3170 msgr2=0x7f2df00b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:26.777 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.775+0000 7f2df5c00640 1 -- 172.21.15.47:0/1056433780 shutdown_connections 2024-04-20T10:27:26.777 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.775+0000 7f2df5c00640 1 -- 172.21.15.47:0/1056433780 wait complete. 2024-04-20T10:27:26.777 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.777+0000 7f2df5c00640 1 Processor -- start 2024-04-20T10:27:26.777 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.777+0000 7f2df5c00640 1 -- start start 2024-04-20T10:27:26.779 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2df5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd0004800 0x7f2df01ef710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2df5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2df0158d80 0x7f2df0157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2df5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df0159200 0x7f2df01576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2df5c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0160e50 con 0x7f2df0158d80 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2df5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0160cd0 con 0x7f2df0159200 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2df5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2df0160fd0 con 0x7f2dd0004800 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2def400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2df0158d80 0x7f2df0157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2def400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2df0158d80 0x7f2df0157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58398/0 (socket says 172.21.15.47:58398) 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.778+0000 7f2def400640 1 -- 172.21.15.47:0/913495723 learned_addr learned my addr 172.21.15.47:0/913495723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2def400640 1 -- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd0004800 msgr2=0x7f2df01ef710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2def400640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd0004800 0x7f2df01ef710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2def400640 1 -- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df0159200 msgr2=0x7f2df01576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2def400640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df0159200 0x7f2df01576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2def400640 1 -- 172.21.15.47:0/913495723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01efc50 con 0x7f2df0158d80 2024-04-20T10:27:26.780 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2def400640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2df0158d80 0x7f2df0157170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8037040 tx=0x7f2dd8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:26.781 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2de7e00640 1 -- 172.21.15.47:0/913495723 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8039050 con 0x7f2df0158d80 2024-04-20T10:27:26.781 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2de7e00640 1 -- 172.21.15.47:0/913495723 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dd8045070 con 0x7f2df0158d80 2024-04-20T10:27:26.781 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2de7e00640 1 -- 172.21.15.47:0/913495723 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd8003850 con 0x7f2df0158d80 2024-04-20T10:27:26.781 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.779+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df0157ce0 con 0x7f2df0158d80 2024-04-20T10:27:26.781 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.780+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01582a0 con 0x7f2df0158d80 2024-04-20T10:27:26.781 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.781+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df00692b0 con 0x7f2df0158d80 2024-04-20T10:27:26.781 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.782+0000 7f2de7e00640 1 -- 172.21.15.47:0/913495723 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dd80039f0 con 0x7f2df0158d80 2024-04-20T10:27:26.782 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.782+0000 7f2de7e00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2dcc06c8f0 0x7f2dcc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:26.782 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.783+0000 7f2de7e00640 1 -- 172.21.15.47:0/913495723 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f2dd80ad360 con 0x7f2df0158d80 2024-04-20T10:27:26.783 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.783+0000 7f2defe00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2dcc06c8f0 0x7f2dcc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:26.784 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.784+0000 7f2defe00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2dcc06c8f0 0x7f2dcc06edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2de0000ff0 tx=0x7f2de0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:26.788 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:26.788+0000 7f2de7e00640 1 -- 172.21.15.47:0/913495723 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2dd804e020 con 0x7f2df0158d80 2024-04-20T10:27:26.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:27:26.793 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:27:27.011 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.011+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2df006d070 con 0x7f2df0158d80 2024-04-20T10:27:27.013 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.013+0000 7f2de7e00640 1 -- 172.21.15.47:0/913495723 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7f2df01efc50 con 0x7f2df0158d80 2024-04-20T10:27:27.013 INFO:teuthology.misc.health.smithi047.stdout: 2024-04-20T10:27:27.014 INFO:teuthology.misc.health.smithi047.stdout:{"epoch":19,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:23.825756+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:27.016 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2dcc06c8f0 msgr2=0x7f2dcc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.016 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2dcc06c8f0 0x7f2dcc06edb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2de0000ff0 tx=0x7f2de0009040 comp rx=0 tx=0).stop 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2df0158d80 msgr2=0x7f2df0157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2df0158d80 0x7f2df0157170 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2dd8037040 tx=0x7f2dd8004730 comp rx=0 tx=0).stop 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 shutdown_connections 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2df0159200 0x7f2df01576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2dcc06c8f0 0x7f2dcc06edb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2df0158d80 0x7f2df0157170 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 --2- 172.21.15.47:0/913495723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dd0004800 0x7f2df01ef710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 >> 172.21.15.47:0/913495723 conn(0x7f2df00b3170 msgr2=0x7f2df006e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 shutdown_connections 2024-04-20T10:27:27.017 INFO:teuthology.misc.health.smithi047.stderr:2024-04-20T10:27:27.016+0000 7f2df5c00640 1 -- 172.21.15.47:0/913495723 wait complete. 2024-04-20T10:27:27.032 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-20T10:27:27.032 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-20T10:27:27.032 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T10:27:27.032 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T10:27:27.039 INFO:tasks.ceph.mds.a:Started 2024-04-20T10:27:27.039 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T10:27:27.039 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T10:27:27.046 INFO:tasks.ceph.mds.c:Started 2024-04-20T10:27:27.046 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:27:27.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:27:27.054 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:27:27.054 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T10:27:27.054 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T10:27:27.061 INFO:tasks.ceph.mds.d:Started 2024-04-20T10:27:27.062 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-20T10:27:27.062 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:27.189 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-20T10:27:27.192 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-20T10:27:27.215 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T10:27:27.215 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T10:27:27.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.244+0000 7f0566e00640 1 Processor -- start 2024-04-20T10:27:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.245+0000 7f0566e00640 1 -- start start 2024-04-20T10:27:27.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0566e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0560161d50 0x7f05600d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0566e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f05600b3330 0x7f05600b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0566e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f056006c190 con 0x7f056006c6d0 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0566e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0560162130 con 0x7f0560161d50 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0566e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f056006c470 con 0x7f05600b3330 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0565a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0560161d50 0x7f05600d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0565a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0560161d50 0x7f05600d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58432/0 (socket says 172.21.15.47:58432) 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0565a00640 1 -- 172.21.15.47:0/3371383782 learned_addr learned my addr 172.21.15.47:0/3371383782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.246+0000 7f0565000640 1 --2- 172.21.15.47:0/3371383782 >> v2:172.21.15.175:3300/0 conn(0x7f05600b3330 0x7f05600b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.247+0000 7f0565a00640 1 -- 172.21.15.47:0/3371383782 >> v2:172.21.15.175:3300/0 conn(0x7f05600b3330 msgr2=0x7f05600b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.247+0000 7f0565a00640 1 --2- 172.21.15.47:0/3371383782 >> v2:172.21.15.175:3300/0 conn(0x7f05600b3330 0x7f05600b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.247+0000 7f0565a00640 1 -- 172.21.15.47:0/3371383782 >> v1:172.21.15.175:6789/0 conn(0x7f056006c6d0 legacy=0x7f05600cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.247+0000 7f0565a00640 1 -- 172.21.15.47:0/3371383782 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600d2e00 con 0x7f0560161d50 2024-04-20T10:27:27.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.247+0000 7f055fe00640 1 -- 172.21.15.47:0/3371383782 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4262055607 0 0) 0x7f056006c190 con 0x7f056006c6d0 2024-04-20T10:27:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.248+0000 7f0565a00640 1 --2- 172.21.15.47:0/3371383782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0560161d50 0x7f05600d38e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05600d1f00 tx=0x7f055402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb28a285e4ec59e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.248+0000 7f055fe00640 1 -- 172.21.15.47:0/3371383782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554034070 con 0x7f0560161d50 2024-04-20T10:27:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.248+0000 7f055fe00640 1 -- 172.21.15.47:0/3371383782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0554030860 con 0x7f0560161d50 2024-04-20T10:27:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.248+0000 7f055fe00640 1 -- 172.21.15.47:0/3371383782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554030c30 con 0x7f0560161d50 2024-04-20T10:27:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 -- 172.21.15.47:0/3371383782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0560161d50 msgr2=0x7f05600d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 --2- 172.21.15.47:0/3371383782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0560161d50 0x7f05600d38e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f05600d1f00 tx=0x7f055402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 -- 172.21.15.47:0/3371383782 shutdown_connections 2024-04-20T10:27:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 --2- 172.21.15.47:0/3371383782 >> v2:172.21.15.175:3300/0 conn(0x7f05600b3330 0x7f05600b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 --2- 172.21.15.47:0/3371383782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0560161d50 0x7f05600d38e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 -- 172.21.15.47:0/3371383782 >> 172.21.15.47:0/3371383782 conn(0x7f05600d4450 msgr2=0x7f05600d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 -- 172.21.15.47:0/3371383782 shutdown_connections 2024-04-20T10:27:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.249+0000 7f0566e00640 1 -- 172.21.15.47:0/3371383782 wait complete. 2024-04-20T10:27:27.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.251+0000 7f0566e00640 1 Processor -- start 2024-04-20T10:27:27.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.251+0000 7f0566e00640 1 -- start start 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0566e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f056006c6d0 0x7f05600c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0566e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05600b3330 0x7f0560158aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0566e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0560161d50 0x7f05600c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0566e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f056006cb20 con 0x7f05600b3330 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0566e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05600d33c0 con 0x7f0560161d50 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0566e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f056006cca0 con 0x7f056006c6d0 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0565000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05600b3330 0x7f0560158aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0565000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05600b3330 0x7f0560158aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58434/0 (socket says 172.21.15.47:58434) 2024-04-20T10:27:27.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0565000640 1 -- 172.21.15.47:0/1137223857 learned_addr learned my addr 172.21.15.47:0/1137223857 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0565a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f056006c6d0 0x7f05600c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0566400640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0560161d50 0x7f05600c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0565000640 1 -- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f056006c6d0 msgr2=0x7f05600c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.252+0000 7f0565000640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f056006c6d0 0x7f05600c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f0565000640 1 -- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0560161d50 msgr2=0x7f05600c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f0565000640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0560161d50 0x7f05600c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f0565000640 1 -- 172.21.15.47:0/1137223857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05600c8eb0 con 0x7f05600b3330 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f0566400640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0560161d50 0x7f05600c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f0565000640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05600b3330 0x7f0560158aa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9e0 tx=0x7f055000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f055d600640 1 -- 172.21.15.47:0/1137223857 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550017070 con 0x7f05600b3330 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f055d600640 1 -- 172.21.15.47:0/1137223857 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f055000c9d0 con 0x7f05600b3330 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05600c99f0 con 0x7f05600b3330 2024-04-20T10:27:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f055d600640 1 -- 172.21.15.47:0/1137223857 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055000cdc0 con 0x7f05600b3330 2024-04-20T10:27:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.253+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05600c9430 con 0x7f05600b3330 2024-04-20T10:27:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.254+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0560075b60 con 0x7f05600b3330 2024-04-20T10:27:27.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.255+0000 7f055d600640 1 -- 172.21.15.47:0/1137223857 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f055001a070 con 0x7f05600b3330 2024-04-20T10:27:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.257+0000 7f055d600640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f054406c920 0x7f054406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.257+0000 7f055d600640 1 -- 172.21.15.47:0/1137223857 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f055008b7d0 con 0x7f05600b3330 2024-04-20T10:27:27.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.257+0000 7f0565a00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f054406c920 0x7f054406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.258+0000 7f0565a00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f054406c920 0x7f054406ede0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0554036000 tx=0x7f055403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.262+0000 7f055d600640 1 -- 172.21.15.47:0/1137223857 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0550059860 con 0x7f05600b3330 2024-04-20T10:27:27.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.477+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f05600b57a0 con 0x7f05600b3330 2024-04-20T10:27:27.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.478+0000 7f055d600640 1 -- 172.21.15.47:0/1137223857 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7f055005db60 con 0x7f05600b3330 2024-04-20T10:27:27.479 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:27.479 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:23.825756+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.481+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f054406c920 msgr2=0x7f054406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.481+0000 7f0566e00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f054406c920 0x7f054406ede0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0554036000 tx=0x7f055403b040 comp rx=0 tx=0).stop 2024-04-20T10:27:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.481+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05600b3330 msgr2=0x7f0560158aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.481+0000 7f0566e00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05600b3330 0x7f0560158aa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f055000e9e0 tx=0x7f055000eea0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 shutdown_connections 2024-04-20T10:27:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0560161d50 0x7f05600c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f054406c920 0x7f054406ede0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f05600b3330 0x7f0560158aa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 --2- 172.21.15.47:0/1137223857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f056006c6d0 0x7f05600c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 >> 172.21.15.47:0/1137223857 conn(0x7f05600d4450 msgr2=0x7f05600d2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 shutdown_connections 2024-04-20T10:27:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.482+0000 7f0566e00640 1 -- 172.21.15.47:0/1137223857 wait complete. 2024-04-20T10:27:27.498 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:27:27.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:27:27.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.690+0000 7f045be00640 1 Processor -- start 2024-04-20T10:27:27.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.690+0000 7f045be00640 1 -- start start 2024-04-20T10:27:27.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f045c15a130 0x7f045c073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c06f870 0x7f045c0739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f045c15cc20 con 0x7f045c071a60 2024-04-20T10:27:27.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045be00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f045c15d7f0 con 0x7f045c15a130 2024-04-20T10:27:27.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f045c15cec0 con 0x7f045c06f870 2024-04-20T10:27:27.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f045c15a130 0x7f045c073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f045c15a130 0x7f045c073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58436/0 (socket says 172.21.15.47:58436) 2024-04-20T10:27:27.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.692+0000 7f045aa00640 1 -- 172.21.15.47:0/3012019565 learned_addr learned my addr 172.21.15.47:0/3012019565 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.693+0000 7f045b400640 1 --2- 172.21.15.47:0/3012019565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c06f870 0x7f045c0739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.693+0000 7f045aa00640 1 -- 172.21.15.47:0/3012019565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c06f870 msgr2=0x7f045c0739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.693+0000 7f045aa00640 1 --2- 172.21.15.47:0/3012019565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c06f870 0x7f045c0739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.693+0000 7f045aa00640 1 -- 172.21.15.47:0/3012019565 >> v1:172.21.15.175:6789/0 conn(0x7f045c071a60 legacy=0x7f045c06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.693+0000 7f045aa00640 1 -- 172.21.15.47:0/3012019565 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c15d1a0 con 0x7f045c15a130 2024-04-20T10:27:27.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.694+0000 7f045aa00640 1 --2- 172.21.15.47:0/3012019565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c073470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f044c009e50 tx=0x7f044c02bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c24042118865654d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.694+0000 7f045a000640 1 -- 172.21.15.47:0/3012019565 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c02c950 con 0x7f045c15a130 2024-04-20T10:27:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.694+0000 7f045a000640 1 -- 172.21.15.47:0/3012019565 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f044c02caf0 con 0x7f045c15a130 2024-04-20T10:27:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.694+0000 7f045a000640 1 -- 172.21.15.47:0/3012019565 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c0343f0 con 0x7f045c15a130 2024-04-20T10:27:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 -- 172.21.15.47:0/3012019565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 msgr2=0x7f045c073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 --2- 172.21.15.47:0/3012019565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c073470 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f044c009e50 tx=0x7f044c02bdb0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 -- 172.21.15.47:0/3012019565 shutdown_connections 2024-04-20T10:27:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 --2- 172.21.15.47:0/3012019565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c06f870 0x7f045c0739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 --2- 172.21.15.47:0/3012019565 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c073470 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 -- 172.21.15.47:0/3012019565 >> 172.21.15.47:0/3012019565 conn(0x7f045c0cffa0 msgr2=0x7f045c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 -- 172.21.15.47:0/3012019565 shutdown_connections 2024-04-20T10:27:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.695+0000 7f045be00640 1 -- 172.21.15.47:0/3012019565 wait complete. 2024-04-20T10:27:27.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.697+0000 7f045be00640 1 Processor -- start 2024-04-20T10:27:27.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.697+0000 7f045be00640 1 -- start start 2024-04-20T10:27:27.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f045c06f870 0x7f045c158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c071a60 0x7f045c0c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f045c06ba80 con 0x7f045c15a130 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f045c06b900 con 0x7f045c06f870 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f045c06bc00 con 0x7f045c071a60 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c154ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58448/0 (socket says 172.21.15.47:58448) 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045b400640 1 -- 172.21.15.47:0/2820753287 learned_addr learned my addr 172.21.15.47:0/2820753287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045aa00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f045c06f870 0x7f045c158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f045b400640 1 -- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c071a60 msgr2=0x7f045c0c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.698+0000 7f0452000640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c071a60 0x7f045c0c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045b400640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c071a60 0x7f045c0c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045b400640 1 -- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f045c06f870 msgr2=0x7f045c158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045b400640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f045c06f870 0x7f045c158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045b400640 1 -- 172.21.15.47:0/2820753287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c0b3100 con 0x7f045c15a130 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045aa00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f045c06f870 0x7f045c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f0452000640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c071a60 0x7f045c0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045b400640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c154ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f045400c7f0 tx=0x7f045400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f0453400640 1 -- 172.21.15.47:0/2820753287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454015070 con 0x7f045c15a130 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f0453400640 1 -- 172.21.15.47:0/2820753287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0454004220 con 0x7f045c15a130 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f0453400640 1 -- 172.21.15.47:0/2820753287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0454007620 con 0x7f045c15a130 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c1f79c0 con 0x7f045c15a130 2024-04-20T10:27:27.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.699+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1f7f80 con 0x7f045c15a130 2024-04-20T10:27:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.700+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f045c075b30 con 0x7f045c15a130 2024-04-20T10:27:27.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.701+0000 7f0453400640 1 -- 172.21.15.47:0/2820753287 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04540079d0 con 0x7f045c15a130 2024-04-20T10:27:27.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.702+0000 7f0453400640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0438075100 0x7f04380775c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:27.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.702+0000 7f0453400640 1 -- 172.21.15.47:0/2820753287 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f045408b8d0 con 0x7f045c15a130 2024-04-20T10:27:27.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.703+0000 7f045aa00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0438075100 0x7f04380775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:27.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.704+0000 7f045aa00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0438075100 0x7f04380775c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f044c035000 tx=0x7f044c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:27.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.707+0000 7f0453400640 1 -- 172.21.15.47:0/2820753287 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f045405a040 con 0x7f045c15a130 2024-04-20T10:27:27.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.913+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f045c06d010 con 0x7f045c15a130 2024-04-20T10:27:27.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.914+0000 7f0453400640 1 -- 172.21.15.47:0/2820753287 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f0454059940 con 0x7f045c15a130 2024-04-20T10:27:27.914 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:27:27.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0438075100 msgr2=0x7f04380775c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0438075100 0x7f04380775c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f044c035000 tx=0x7f044c030000 comp rx=0 tx=0).stop 2024-04-20T10:27:27.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 msgr2=0x7f045c154ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:27.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c154ab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f045400c7f0 tx=0x7f045400ccb0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 shutdown_connections 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0438075100 0x7f04380775c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f045c15a130 0x7f045c154ab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f045c071a60 0x7f045c0c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 --2- 172.21.15.47:0/2820753287 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f045c06f870 0x7f045c158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 >> 172.21.15.47:0/2820753287 conn(0x7f045c0cffa0 msgr2=0x7f045c0cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.916+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 shutdown_connections 2024-04-20T10:27:27.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:27.917+0000 7f045be00640 1 -- 172.21.15.47:0/2820753287 wait complete. 2024-04-20T10:27:27.935 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:27:28.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.130+0000 7f9043200640 1 Processor -- start 2024-04-20T10:27:28.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.131+0000 7f9043200640 1 -- start start 2024-04-20T10:27:28.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.132+0000 7f9043200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9044161d50 0x7f90440d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.132+0000 7f9043200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90440b3330 0x7f90440b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.132+0000 7f9043200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f904406c190 con 0x7f904406c6d0 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.132+0000 7f9043200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9044162130 con 0x7f90440b3330 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.132+0000 7f9043200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f904406c470 con 0x7f9044161d50 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.132+0000 7f9041400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90440b3330 0x7f90440b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.132+0000 7f9041400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f90440b3330 0x7f90440b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58450/0 (socket says 172.21.15.47:58450) 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.133+0000 7f9041400640 1 -- 172.21.15.47:0/2993587312 learned_addr learned my addr 172.21.15.47:0/2993587312 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:28.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.133+0000 7f9041400640 1 -- 172.21.15.47:0/2993587312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9044161d50 msgr2=0x7f90440d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.133+0000 7f9041e00640 1 --2- 172.21.15.47:0/2993587312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9044161d50 0x7f90440d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.133+0000 7f9041400640 1 --2- 172.21.15.47:0/2993587312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9044161d50 0x7f90440d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.133+0000 7f9041400640 1 -- 172.21.15.47:0/2993587312 >> v1:172.21.15.175:6789/0 conn(0x7f904406c6d0 legacy=0x7f90440cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.133+0000 7f9041400640 1 -- 172.21.15.47:0/2993587312 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90440d2e00 con 0x7f90440b3330 2024-04-20T10:27:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.133+0000 7f9041e00640 1 --2- 172.21.15.47:0/2993587312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9044161d50 0x7f90440d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:28.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.134+0000 7f9041400640 1 --2- 172.21.15.47:0/2993587312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90440b3330 0x7f90440b3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9030009870 tx=0x7f903002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735b78a4272e89fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.134+0000 7f9040a00640 1 -- 172.21.15.47:0/2993587312 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030035020 con 0x7f90440b3330 2024-04-20T10:27:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9040a00640 1 -- 172.21.15.47:0/2993587312 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030031640 con 0x7f90440b3330 2024-04-20T10:27:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9040a00640 1 -- 172.21.15.47:0/2993587312 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903002d040 con 0x7f90440b3330 2024-04-20T10:27:28.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 -- 172.21.15.47:0/2993587312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90440b3330 msgr2=0x7f90440b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 --2- 172.21.15.47:0/2993587312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90440b3330 0x7f90440b3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9030009870 tx=0x7f903002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 -- 172.21.15.47:0/2993587312 shutdown_connections 2024-04-20T10:27:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 --2- 172.21.15.47:0/2993587312 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f90440b3330 0x7f90440b3880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 --2- 172.21.15.47:0/2993587312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9044161d50 0x7f90440d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 -- 172.21.15.47:0/2993587312 >> 172.21.15.47:0/2993587312 conn(0x7f90440d4450 msgr2=0x7f90440d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 -- 172.21.15.47:0/2993587312 shutdown_connections 2024-04-20T10:27:28.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.135+0000 7f9043200640 1 -- 172.21.15.47:0/2993587312 wait complete. 2024-04-20T10:27:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.137+0000 7f9043200640 1 Processor -- start 2024-04-20T10:27:28.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.137+0000 7f9043200640 1 -- start start 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.138+0000 7f9043200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904406c6d0 0x7f90441f4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.138+0000 7f9043200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440b3330 0x7f90440b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.138+0000 7f9043200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9044161d50 0x7f90440b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.138+0000 7f9043200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f904406cb60 con 0x7f904406c6d0 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.138+0000 7f9043200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90440d33c0 con 0x7f9044161d50 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.138+0000 7f9043200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f904406cce0 con 0x7f90440b3330 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.138+0000 7f9041e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904406c6d0 0x7f90441f4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904406c6d0 0x7f90441f4f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58452/0 (socket says 172.21.15.47:58452) 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 -- 172.21.15.47:0/291859122 learned_addr learned my addr 172.21.15.47:0/291859122 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041400640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440b3330 0x7f90440b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 -- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440b3330 msgr2=0x7f90440b8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440b3330 0x7f90440b8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9042800640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9044161d50 0x7f90440b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 -- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9044161d50 msgr2=0x7f90440b9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9044161d50 0x7f90440b9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 -- 172.21.15.47:0/291859122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90440cd260 con 0x7f904406c6d0 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9042800640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9044161d50 0x7f90440b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.139+0000 7f9041e00640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904406c6d0 0x7f90441f4f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f903400c820 tx=0x7f903400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.140+0000 7f903a000640 1 -- 172.21.15.47:0/291859122 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034015070 con 0x7f904406c6d0 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.140+0000 7f903a000640 1 -- 172.21.15.47:0/291859122 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9034007a30 con 0x7f904406c6d0 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.140+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90440cd540 con 0x7f904406c6d0 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.140+0000 7f903a000640 1 -- 172.21.15.47:0/291859122 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034007de0 con 0x7f904406c6d0 2024-04-20T10:27:28.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.140+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90440cd820 con 0x7f904406c6d0 2024-04-20T10:27:28.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.141+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9044075b60 con 0x7f904406c6d0 2024-04-20T10:27:28.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.142+0000 7f903a000640 1 -- 172.21.15.47:0/291859122 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9034018070 con 0x7f904406c6d0 2024-04-20T10:27:28.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.143+0000 7f903a000640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f902406c8f0 0x7f902406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.143+0000 7f903a000640 1 -- 172.21.15.47:0/291859122 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f903408b6b0 con 0x7f904406c6d0 2024-04-20T10:27:28.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.144+0000 7f9041400640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f902406c8f0 0x7f902406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.145+0000 7f9041400640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f902406c8f0 0x7f902406edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9030036000 tx=0x7f9030008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:28.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.148+0000 7f903a000640 1 -- 172.21.15.47:0/291859122 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9034059e20 con 0x7f904406c6d0 2024-04-20T10:27:28.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.395+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f904406d870 con 0x7f904406c6d0 2024-04-20T10:27:28.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.396+0000 7f903a000640 1 -- 172.21.15.47:0/291859122 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3870 (secure 0 0 0) 0x7f9034059720 con 0x7f904406c6d0 2024-04-20T10:27:28.397 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:28.397 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-20T10:27:28.397 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 3 2024-04-20T10:27:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f902406c8f0 msgr2=0x7f902406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f902406c8f0 0x7f902406edb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f9030036000 tx=0x7f9030008030 comp rx=0 tx=0).stop 2024-04-20T10:27:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904406c6d0 msgr2=0x7f90441f4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904406c6d0 0x7f90441f4f00 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f903400c820 tx=0x7f903400cce0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 shutdown_connections 2024-04-20T10:27:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9044161d50 0x7f90440b9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440b3330 0x7f90440b8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f902406c8f0 0x7f902406edb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 --2- 172.21.15.47:0/291859122 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f904406c6d0 0x7f90441f4f00 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 >> 172.21.15.47:0/291859122 conn(0x7f90440d4450 msgr2=0x7f904415e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 shutdown_connections 2024-04-20T10:27:28.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.399+0000 7f9043200640 1 -- 172.21.15.47:0/291859122 wait complete. 2024-04-20T10:27:28.418 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:28.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.625+0000 7fa972c00640 1 Processor -- start 2024-04-20T10:27:28.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.626+0000 7fa972c00640 1 -- start start 2024-04-20T10:27:28.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.627+0000 7fa972c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.627+0000 7fa972c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa96c0b3330 0x7fa96c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.627+0000 7fa972c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa96c06c190 con 0x7fa96c161d50 2024-04-20T10:27:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.627+0000 7fa972c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa96c162130 con 0x7fa96c0b3330 2024-04-20T10:27:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.627+0000 7fa972c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c06c470 con 0x7fa96c06cde0 2024-04-20T10:27:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.628+0000 7fa970e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.628+0000 7fa972200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa96c0b3330 0x7fa96c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.628+0000 7fa972200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa96c0b3330 0x7fa96c0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:34260/0 (socket says 172.21.15.47:34260) 2024-04-20T10:27:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.628+0000 7fa972200640 1 -- 172.21.15.47:0/4035657462 learned_addr learned my addr 172.21.15.47:0/4035657462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:28.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.629+0000 7fa972200640 1 -- 172.21.15.47:0/4035657462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 msgr2=0x7fa96c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:28.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.629+0000 7fa972200640 1 --2- 172.21.15.47:0/4035657462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.629+0000 7fa972200640 1 -- 172.21.15.47:0/4035657462 >> v1:172.21.15.175:6789/0 conn(0x7fa96c161d50 legacy=0x7fa96c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.629+0000 7fa972200640 1 -- 172.21.15.47:0/4035657462 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0d2e00 con 0x7fa96c0b3330 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.631+0000 7fa972200640 1 --2- 172.21.15.47:0/4035657462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c0b3880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa964009e50 tx=0x7fa96402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5dd7fe30a7b7c11 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.631+0000 7fa96be00640 1 -- 172.21.15.47:0/4035657462 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa96402cb00 con 0x7fa96c0b3330 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.631+0000 7fa96be00640 1 -- 172.21.15.47:0/4035657462 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96402cca0 con 0x7fa96c0b3330 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.631+0000 7fa96be00640 1 -- 172.21.15.47:0/4035657462 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964034610 con 0x7fa96c0b3330 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 -- 172.21.15.47:0/4035657462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 msgr2=0x7fa96c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 --2- 172.21.15.47:0/4035657462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c0b3880 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa964009e50 tx=0x7fa96402bf60 comp rx=0 tx=0).stop 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 -- 172.21.15.47:0/4035657462 shutdown_connections 2024-04-20T10:27:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 --2- 172.21.15.47:0/4035657462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c0b3880 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 --2- 172.21.15.47:0/4035657462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 -- 172.21.15.47:0/4035657462 >> 172.21.15.47:0/4035657462 conn(0x7fa96c0d4450 msgr2=0x7fa96c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 -- 172.21.15.47:0/4035657462 shutdown_connections 2024-04-20T10:27:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.632+0000 7fa972c00640 1 -- 172.21.15.47:0/4035657462 wait complete. 2024-04-20T10:27:28.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.634+0000 7fa972c00640 1 Processor -- start 2024-04-20T10:27:28.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.634+0000 7fa972c00640 1 -- start start 2024-04-20T10:27:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c161d50 0x7fa96c14e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c0b7660 con 0x7fa96c161d50 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c0d33c0 con 0x7fa96c0b3330 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c0b77e0 con 0x7fa96c06cde0 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c161d50 0x7fa96c14e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c161d50 0x7fa96c14e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58466/0 (socket says 172.21.15.47:58466) 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa972200640 1 -- 172.21.15.47:0/3765859345 learned_addr learned my addr 172.21.15.47:0/3765859345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa970e00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.635+0000 7fa971800640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972200640 1 -- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 msgr2=0x7fa96c0cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972200640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972200640 1 -- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 msgr2=0x7fa96c14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972200640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c14f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972200640 1 -- 172.21.15.47:0/3765859345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0cc180 con 0x7fa96c161d50 2024-04-20T10:27:28.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa970e00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:28.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa971800640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:28.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972200640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c161d50 0x7fa96c14e8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa964009f70 tx=0x7fa964004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa969600640 1 -- 172.21.15.47:0/3765859345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964043020 con 0x7fa96c161d50 2024-04-20T10:27:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa969600640 1 -- 172.21.15.47:0/3765859345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa96402ce50 con 0x7fa96c161d50 2024-04-20T10:27:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c0cc460 con 0x7fa96c161d50 2024-04-20T10:27:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa969600640 1 -- 172.21.15.47:0/3765859345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964009510 con 0x7fa96c161d50 2024-04-20T10:27:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.636+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c0cca20 con 0x7fa96c161d50 2024-04-20T10:27:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.637+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa96c075b60 con 0x7fa96c161d50 2024-04-20T10:27:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.639+0000 7fa969600640 1 -- 172.21.15.47:0/3765859345 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa964046020 con 0x7fa96c161d50 2024-04-20T10:27:28.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.639+0000 7fa969600640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa94c06c8f0 0x7fa94c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:28.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.640+0000 7fa969600640 1 -- 172.21.15.47:0/3765859345 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7fa9640ade30 con 0x7fa96c161d50 2024-04-20T10:27:28.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.640+0000 7fa971800640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa94c06c8f0 0x7fa94c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.641+0000 7fa971800640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa94c06c8f0 0x7fa94c06edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0015e0 tx=0x7fa95c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.644+0000 7fa969600640 1 -- 172.21.15.47:0/3765859345 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa96404d030 con 0x7fa96c161d50 2024-04-20T10:27:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.843+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa96c0cb2e0 con 0x7fa96c161d50 2024-04-20T10:27:28.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.845+0000 7fa969600640 1 -- 172.21.15.47:0/3765859345 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12896 (secure 0 0 0) 0x7fa96407c5a0 con 0x7fa96c161d50 2024-04-20T10:27:28.845 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:28.845 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:23.825756+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa94c06c8f0 msgr2=0x7fa94c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa94c06c8f0 0x7fa94c06edb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa95c0015e0 tx=0x7fa95c009040 comp rx=0 tx=0).stop 2024-04-20T10:27:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c161d50 msgr2=0x7fa96c14e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c161d50 0x7fa96c14e8b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa964009f70 tx=0x7fa964004a40 comp rx=0 tx=0).stop 2024-04-20T10:27:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 shutdown_connections 2024-04-20T10:27:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa94c06c8f0 0x7fa94c06edb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa96c161d50 0x7fa96c14e8b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa96c0b3330 0x7fa96c14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 --2- 172.21.15.47:0/3765859345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa96c06cde0 0x7fa96c0cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 >> 172.21.15.47:0/3765859345 conn(0x7fa96c0d4450 msgr2=0x7fa96c160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 shutdown_connections 2024-04-20T10:27:28.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:28.848+0000 7fa972c00640 1 -- 172.21.15.47:0/3765859345 wait complete. 2024-04-20T10:27:28.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:27:28.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:27:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.078+0000 7f9d19600640 1 Processor -- start 2024-04-20T10:27:29.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.078+0000 7f9d19600640 1 -- start start 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d19600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d14073b80 0x7f9d1406f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d19600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9d14071a60 0x7f9d1406f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d19600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9d1415cc20 con 0x7f9d1415a130 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d19600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1415d7f0 con 0x7f9d14073b80 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d19600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9d1406fbe0 con 0x7f9d14071a60 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d0be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d14073b80 0x7f9d1406f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d0be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9d14073b80 0x7f9d1406f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58482/0 (socket says 172.21.15.47:58482) 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d0be00640 1 -- 172.21.15.47:0/155450551 learned_addr learned my addr 172.21.15.47:0/155450551 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:29.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d13e00640 1 --1- 172.21.15.47:0/155450551 >> v1:172.21.15.47:6789/0 conn(0x7f9d1415a130 0x7f9d14073470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38822/0 (socket says 172.21.15.47:38822) 2024-04-20T10:27:29.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d18c00640 1 --2- 172.21.15.47:0/155450551 >> v2:172.21.15.175:3300/0 conn(0x7f9d14071a60 0x7f9d1406f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d0be00640 1 -- 172.21.15.47:0/155450551 >> v2:172.21.15.175:3300/0 conn(0x7f9d14071a60 msgr2=0x7f9d1406f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d0be00640 1 --2- 172.21.15.47:0/155450551 >> v2:172.21.15.175:3300/0 conn(0x7f9d14071a60 0x7f9d1406f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d0be00640 1 -- 172.21.15.47:0/155450551 >> v1:172.21.15.47:6789/0 conn(0x7f9d1415a130 legacy=0x7f9d14073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d0be00640 1 -- 172.21.15.47:0/155450551 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1415cec0 con 0x7f9d14073b80 2024-04-20T10:27:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.080+0000 7f9d18c00640 1 --2- 172.21.15.47:0/155450551 >> v2:172.21.15.175:3300/0 conn(0x7f9d14071a60 0x7f9d1406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:29.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.082+0000 7f9d0be00640 1 --2- 172.21.15.47:0/155450551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14073b80 0x7f9d1406f160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d140d0f90 tx=0x7f9cfc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cbb702ff8ccd4c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.082+0000 7f9d13400640 1 -- 172.21.15.47:0/155450551 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc035020 con 0x7f9d14073b80 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.082+0000 7f9d13400640 1 -- 172.21.15.47:0/155450551 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cfc031870 con 0x7f9d14073b80 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.082+0000 7f9d13400640 1 -- 172.21.15.47:0/155450551 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc031c40 con 0x7f9d14073b80 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.082+0000 7f9d19600640 1 -- 172.21.15.47:0/155450551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14073b80 msgr2=0x7f9d1406f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.082+0000 7f9d19600640 1 --2- 172.21.15.47:0/155450551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14073b80 0x7f9d1406f160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9d140d0f90 tx=0x7f9cfc02b910 comp rx=0 tx=0).stop 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.083+0000 7f9d19600640 1 -- 172.21.15.47:0/155450551 shutdown_connections 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.083+0000 7f9d19600640 1 --2- 172.21.15.47:0/155450551 >> v2:172.21.15.175:3300/0 conn(0x7f9d14071a60 0x7f9d1406f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.083+0000 7f9d19600640 1 --2- 172.21.15.47:0/155450551 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14073b80 0x7f9d1406f160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.083+0000 7f9d19600640 1 -- 172.21.15.47:0/155450551 >> 172.21.15.47:0/155450551 conn(0x7f9d140cffa0 msgr2=0x7f9d140d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.083+0000 7f9d19600640 1 -- 172.21.15.47:0/155450551 shutdown_connections 2024-04-20T10:27:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.083+0000 7f9d19600640 1 -- 172.21.15.47:0/155450551 wait complete. 2024-04-20T10:27:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.085+0000 7f9d19600640 1 Processor -- start 2024-04-20T10:27:29.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.085+0000 7f9d19600640 1 -- start start 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d19600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14071a60 0x7f9d140b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d19600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14073b80 0x7f9d1409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d19600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d1415a130 0x7f9d140c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d19600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1406b900 con 0x7f9d14071a60 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d19600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1415d480 con 0x7f9d14073b80 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d19600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1406ba80 con 0x7f9d1415a130 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14071a60 0x7f9d140b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14071a60 0x7f9d140b2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58486/0 (socket says 172.21.15.47:58486) 2024-04-20T10:27:29.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 -- 172.21.15.47:0/3974737242 learned_addr learned my addr 172.21.15.47:0/3974737242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d0be00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14073b80 0x7f9d1409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 -- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d1415a130 msgr2=0x7f9d140c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d18c00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d1415a130 0x7f9d140c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d1415a130 0x7f9d140c9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 -- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14073b80 msgr2=0x7f9d1409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14073b80 0x7f9d1409a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d13e00640 1 -- 172.21.15.47:0/3974737242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d140ca690 con 0x7f9d14071a60 2024-04-20T10:27:29.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.086+0000 7f9d0be00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14073b80 0x7f9d1409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.087+0000 7f9d13e00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14071a60 0x7f9d140b2d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c3c0 tx=0x7f9d0400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.087+0000 7f9d10c00640 1 -- 172.21.15.47:0/3974737242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d040074e0 con 0x7f9d14071a60 2024-04-20T10:27:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.087+0000 7f9d10c00640 1 -- 172.21.15.47:0/3974737242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d04007680 con 0x7f9d14071a60 2024-04-20T10:27:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.087+0000 7f9d10c00640 1 -- 172.21.15.47:0/3974737242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d04004a00 con 0x7f9d14071a60 2024-04-20T10:27:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.087+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1409b160 con 0x7f9d14071a60 2024-04-20T10:27:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.087+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1409b720 con 0x7f9d14071a60 2024-04-20T10:27:29.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.088+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d14075b30 con 0x7f9d14071a60 2024-04-20T10:27:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.089+0000 7f9d10c00640 1 -- 172.21.15.47:0/3974737242 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9d0401a020 con 0x7f9d14071a60 2024-04-20T10:27:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.090+0000 7f9d10c00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cf006c8f0 0x7f9cf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.091+0000 7f9d10c00640 1 -- 172.21.15.47:0/3974737242 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f9d0408b0d0 con 0x7f9d14071a60 2024-04-20T10:27:29.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.091+0000 7f9d0be00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cf006c8f0 0x7f9cf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.092+0000 7f9d0be00640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cf006c8f0 0x7f9cf006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc036000 tx=0x7f9cfc03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.095+0000 7f9d10c00640 1 -- 172.21.15.47:0/3974737242 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9d04059720 con 0x7f9d14071a60 2024-04-20T10:27:29.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.297+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9d1406d010 con 0x7f9d14071a60 2024-04-20T10:27:29.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.297+0000 7f9d10c00640 1 -- 172.21.15.47:0/3974737242 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) ==== 93+0+10 (secure 0 0 0) 0x7f9d04059020 con 0x7f9d14071a60 2024-04-20T10:27:29.298 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.299+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cf006c8f0 msgr2=0x7f9cf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.299+0000 7f9d19600640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cf006c8f0 0x7f9cf006edb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc036000 tx=0x7f9cfc03c040 comp rx=0 tx=0).stop 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.299+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14071a60 msgr2=0x7f9d140b2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14071a60 0x7f9d140b2d60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d0400c3c0 tx=0x7f9d0400c880 comp rx=0 tx=0).stop 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 shutdown_connections 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d1415a130 0x7f9d140c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d14073b80 0x7f9d1409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cf006c8f0 0x7f9cf006edb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 --2- 172.21.15.47:0/3974737242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9d14071a60 0x7f9d140b2d60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 >> 172.21.15.47:0/3974737242 conn(0x7f9d140cffa0 msgr2=0x7f9d1415f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:29.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 shutdown_connections 2024-04-20T10:27:29.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.300+0000 7f9d19600640 1 -- 172.21.15.47:0/3974737242 wait complete. 2024-04-20T10:27:29.317 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T10:27:29.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.522+0000 7f3808e00640 1 Processor -- start 2024-04-20T10:27:29.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.522+0000 7f3808e00640 1 -- start start 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.524+0000 7f3808e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f38040b3330 0x7f38040d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.524+0000 7f3808e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380406c190 con 0x7f380406cde0 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.524+0000 7f3808e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380406c470 con 0x7f3804161d50 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.524+0000 7f3808e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3804162130 con 0x7f38040b3330 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.524+0000 7f37faa00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f380406cde0 0x7f38040b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38836/0 (socket says 172.21.15.47:38836) 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.524+0000 7f37faa00640 1 -- 172.21.15.47:0/901698381 learned_addr learned my addr 172.21.15.47:0/901698381 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3803e00640 1 --2- 172.21.15.47:0/901698381 >> v2:172.21.15.175:3300/0 conn(0x7f38040b3330 0x7f38040d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3312312954 0 0) 0x7f380406c190 con 0x7f380406cde0 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f37e8003520 con 0x7f380406cde0 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3091700022 0 0) 0x7f37e8003520 con 0x7f380406cde0 2024-04-20T10:27:29.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 >> v2:172.21.15.175:3300/0 conn(0x7f38040b3330 msgr2=0x7f38040d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 --2- 172.21.15.47:0/901698381 >> v2:172.21.15.175:3300/0 conn(0x7f38040b3330 0x7f38040d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 >> v1:172.21.15.175:6789/0 conn(0x7f3804161d50 legacy=0x7f380406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38040d2e00 con 0x7f380406cde0 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f37ec001670 con 0x7f380406cde0 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 >> v1:172.21.15.47:6789/0 conn(0x7f380406cde0 legacy=0x7f38040b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37e80039b0 0x7f37e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e80043b0 0x7f37e8004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e8005080 0x7f37e800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f37e800bfb0 con 0x7f37e8005080 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37e800c160 con 0x7f37e80043b0 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.525+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37e800c360 con 0x7f37e80039b0 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f3803e00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e8005080 0x7f37e800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f3803400640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37e80039b0 0x7f37e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f37faa00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e80043b0 0x7f37e8004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f3803e00640 1 -- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37e80039b0 msgr2=0x7f37e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f3803e00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37e80039b0 0x7f37e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f3803e00640 1 -- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e80043b0 msgr2=0x7f37e8004880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f3803e00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e80043b0 0x7f37e8004880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.526+0000 7f3803e00640 1 -- 172.21.15.47:0/901698381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e8003520 con 0x7f37e8005080 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.527+0000 7f3803e00640 1 -- 172.21.15.47:0/901698381 reap_dead start 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.527+0000 7f3803e00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e8005080 0x7f37e800b9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009fc0 tx=0x7f37e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813de59f08670d13 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.527+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc009c10 con 0x7f37e8005080 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37fc02cca0 con 0x7f37e8005080 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3802a00640 1 -- 172.21.15.47:0/901698381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc003bd0 con 0x7f37e8005080 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3808e00640 1 -- 172.21.15.47:0/901698381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e8005080 msgr2=0x7f37e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3808e00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e8005080 0x7f37e800b9c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009fc0 tx=0x7f37e8005830 comp rx=0 tx=0).stop 2024-04-20T10:27:29.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3808e00640 1 -- 172.21.15.47:0/901698381 shutdown_connections 2024-04-20T10:27:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3808e00640 1 --2- 172.21.15.47:0/901698381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37e8005080 0x7f37e800b9c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3808e00640 1 -- 172.21.15.47:0/901698381 >> 172.21.15.47:0/901698381 conn(0x7f38040d4450 msgr2=0x7f38040d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3808e00640 1 -- 172.21.15.47:0/901698381 shutdown_connections 2024-04-20T10:27:29.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.528+0000 7f3808e00640 1 -- 172.21.15.47:0/901698381 wait complete. 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.530+0000 7f3808e00640 1 Processor -- start 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.530+0000 7f3808e00640 1 -- start start 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.531+0000 7f3808e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e8005080 0x7f38041f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.531+0000 7f3808e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38040cda90 0x7f38040cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.531+0000 7f3808e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38040c9020 0x7f38040c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.531+0000 7f3808e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f38040d38e0 con 0x7f38040cda90 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.531+0000 7f3808e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38040d33c0 con 0x7f37e8005080 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.531+0000 7f3808e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38040d3a60 con 0x7f38040c9020 2024-04-20T10:27:29.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38040cda90 0x7f38040cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38040cda90 0x7f38040cdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58496/0 (socket says 172.21.15.47:58496) 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 -- 172.21.15.47:0/926486429 learned_addr learned my addr 172.21.15.47:0/926486429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f3803400640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e8005080 0x7f38041f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f3803e00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38040c9020 0x7f38040c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 -- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38040c9020 msgr2=0x7f38040c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38040c9020 0x7f38040c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 -- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e8005080 msgr2=0x7f38041f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e8005080 0x7f38041f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 -- 172.21.15.47:0/926486429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38040ca8d0 con 0x7f38040cda90 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.532+0000 7f37faa00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38040cda90 0x7f38040cdf10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00e730 tx=0x7f37ec00ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.533+0000 7f37fbe00640 1 -- 172.21.15.47:0/926486429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec00ee40 con 0x7f38040cda90 2024-04-20T10:27:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.533+0000 7f37fbe00640 1 -- 172.21.15.47:0/926486429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37ec018070 con 0x7f38040cda90 2024-04-20T10:27:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.533+0000 7f37fbe00640 1 -- 172.21.15.47:0/926486429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec00cb10 con 0x7f38040cda90 2024-04-20T10:27:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.533+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38040c9d50 con 0x7f38040cda90 2024-04-20T10:27:29.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.533+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38040ca310 con 0x7f38040cda90 2024-04-20T10:27:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.534+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3804067ed0 con 0x7f38040cda90 2024-04-20T10:27:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.535+0000 7f37fbe00640 1 -- 172.21.15.47:0/926486429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37ec00ccb0 con 0x7f38040cda90 2024-04-20T10:27:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.536+0000 7f37fbe00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37e006c920 0x7f37e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.536+0000 7f37fbe00640 1 -- 172.21.15.47:0/926486429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f37ec08b680 con 0x7f38040cda90 2024-04-20T10:27:29.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.537+0000 7f3803400640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37e006c920 0x7f37e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.538+0000 7f3803400640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37e006c920 0x7f37e006ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38040cfa40 tx=0x7f37f4007b70 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.541+0000 7f37fbe00640 1 -- 172.21.15.47:0/926486429 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f37ec059d60 con 0x7f38040cda90 2024-04-20T10:27:29.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.743+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f38040cfa70 con 0x7f38040cda90 2024-04-20T10:27:29.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.744+0000 7f37fbe00640 1 -- 172.21.15.47:0/926486429 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f37ec059660 con 0x7f38040cda90 2024-04-20T10:27:29.744 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:29.744 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-20T10:27:29.745 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:27:29.745 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-20T10:27:29.745 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-20T10:27:29.745 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:27:29.745 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.746+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37e006c920 msgr2=0x7f37e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.746+0000 7f3808e00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37e006c920 0x7f37e006ede0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f38040cfa40 tx=0x7f37f4007b70 comp rx=0 tx=0).stop 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.746+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38040cda90 msgr2=0x7f38040cdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.746+0000 7f3808e00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38040cda90 0x7f38040cdf10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f37ec00e730 tx=0x7f37ec00ebf0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 shutdown_connections 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38040c9020 0x7f38040c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37e006c920 0x7f37e006ede0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f38040cda90 0x7f38040cdf10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 --2- 172.21.15.47:0/926486429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37e8005080 0x7f38041f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 >> 172.21.15.47:0/926486429 conn(0x7f38040d4450 msgr2=0x7f38040676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 shutdown_connections 2024-04-20T10:27:29.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.747+0000 7f3808e00640 1 -- 172.21.15.47:0/926486429 wait complete. 2024-04-20T10:27:29.765 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T10:27:29.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T10:27:29.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.979+0000 7f01d0800640 1 Processor -- start 2024-04-20T10:27:29.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.979+0000 7f01d0800640 1 -- start start 2024-04-20T10:27:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.981+0000 7f01d0800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c8156c20 0x7f01c806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.981+0000 7f01d0800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01c815ca40 0x7f01c80ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.981+0000 7f01d0800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01c8141c60 con 0x7f01c80ad8e0 2024-04-20T10:27:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.981+0000 7f01d0800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f01c815cf90 con 0x7f01c815ca40 2024-04-20T10:27:29.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.981+0000 7f01d0800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01c815d270 con 0x7f01c8156c20 2024-04-20T10:27:29.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01c815ca40 0x7f01c80ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f01c815ca40 0x7f01c80ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58510/0 (socket says 172.21.15.47:58510) 2024-04-20T10:27:29.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cd400640 1 -- 172.21.15.47:0/1657179426 learned_addr learned my addr 172.21.15.47:0/1657179426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:29.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cde00640 1 --2- 172.21.15.47:0/1657179426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c8156c20 0x7f01c806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cd400640 1 -- 172.21.15.47:0/1657179426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c8156c20 msgr2=0x7f01c806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:29.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cd400640 1 --2- 172.21.15.47:0/1657179426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c8156c20 0x7f01c806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cd400640 1 -- 172.21.15.47:0/1657179426 >> v1:172.21.15.175:6789/0 conn(0x7f01c80ad8e0 legacy=0x7f01c80d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.982+0000 7f01cd400640 1 -- 172.21.15.47:0/1657179426 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c815d550 con 0x7f01c815ca40 2024-04-20T10:27:29.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.983+0000 7f01cd400640 1 --2- 172.21.15.47:0/1657179426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c815ca40 0x7f01c80ad3a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009870 tx=0x7f01b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2aa59cc3165499 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.984+0000 7f01cca00640 1 -- 172.21.15.47:0/1657179426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8034070 con 0x7f01c815ca40 2024-04-20T10:27:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.984+0000 7f01cca00640 1 -- 172.21.15.47:0/1657179426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8030840 con 0x7f01c815ca40 2024-04-20T10:27:29.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.984+0000 7f01cca00640 1 -- 172.21.15.47:0/1657179426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8030c10 con 0x7f01c815ca40 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.984+0000 7f01d0800640 1 -- 172.21.15.47:0/1657179426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c815ca40 msgr2=0x7f01c80ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.984+0000 7f01d0800640 1 --2- 172.21.15.47:0/1657179426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c815ca40 0x7f01c80ad3a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009870 tx=0x7f01b802b980 comp rx=0 tx=0).stop 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.985+0000 7f01d0800640 1 -- 172.21.15.47:0/1657179426 shutdown_connections 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.985+0000 7f01d0800640 1 --2- 172.21.15.47:0/1657179426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c815ca40 0x7f01c80ad3a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.985+0000 7f01d0800640 1 --2- 172.21.15.47:0/1657179426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c8156c20 0x7f01c806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.985+0000 7f01d0800640 1 -- 172.21.15.47:0/1657179426 >> 172.21.15.47:0/1657179426 conn(0x7f01c80b3350 msgr2=0x7f01c80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.985+0000 7f01d0800640 1 -- 172.21.15.47:0/1657179426 shutdown_connections 2024-04-20T10:27:29.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.985+0000 7f01d0800640 1 -- 172.21.15.47:0/1657179426 wait complete. 2024-04-20T10:27:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.986+0000 7f01d0800640 1 Processor -- start 2024-04-20T10:27:29.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.987+0000 7f01d0800640 1 -- start start 2024-04-20T10:27:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.987+0000 7f01d0800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c80ad8e0 0x7f01c80998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.987+0000 7f01d0800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c8156c20 0x7f01c80ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.987+0000 7f01d0800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c80cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.987+0000 7f01d0800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01c806bfc0 con 0x7f01c8156c20 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.987+0000 7f01d0800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01c806be40 con 0x7f01c815ca40 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.987+0000 7f01d0800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01c80d0a70 con 0x7f01c80ad8e0 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c8156c20 0x7f01c80ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c8156c20 0x7f01c80ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58512/0 (socket says 172.21.15.47:58512) 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 -- 172.21.15.47:0/1333093089 learned_addr learned my addr 172.21.15.47:0/1333093089 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cde00640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c80ad8e0 0x7f01c80998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01ce800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c80cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 -- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c80ad8e0 msgr2=0x7f01c80998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c80ad8e0 0x7f01c80998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 -- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 msgr2=0x7f01c80cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:29.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c80cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:29.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01cd400640 1 -- 172.21.15.47:0/1333093089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c814b160 con 0x7f01c8156c20 2024-04-20T10:27:29.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.988+0000 7f01ce800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c80cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:29.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.989+0000 7f01cd400640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c8156c20 0x7f01c80ccb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01b8000c00 tx=0x7f01b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.989+0000 7f01c2000640 1 -- 172.21.15.47:0/1333093089 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8034040 con 0x7f01c8156c20 2024-04-20T10:27:29.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.989+0000 7f01c2000640 1 -- 172.21.15.47:0/1333093089 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01b8039d90 con 0x7f01c8156c20 2024-04-20T10:27:29.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.989+0000 7f01c2000640 1 -- 172.21.15.47:0/1333093089 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8043630 con 0x7f01c8156c20 2024-04-20T10:27:29.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.989+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c81555b0 con 0x7f01c8156c20 2024-04-20T10:27:29.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.989+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c8155b70 con 0x7f01c8156c20 2024-04-20T10:27:29.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.990+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01c8068980 con 0x7f01c8156c20 2024-04-20T10:27:29.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.991+0000 7f01c2000640 1 -- 172.21.15.47:0/1333093089 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f01b804a060 con 0x7f01c8156c20 2024-04-20T10:27:29.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.992+0000 7f01c2000640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f01ac06c8f0 0x7f01ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:29.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.992+0000 7f01c2000640 1 -- 172.21.15.47:0/1333093089 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f01b80ad7a0 con 0x7f01c8156c20 2024-04-20T10:27:29.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.993+0000 7f01cde00640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f01ac06c8f0 0x7f01ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:29.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.994+0000 7f01cde00640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f01ac06c8f0 0x7f01ac06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009ad0 tx=0x7f01bc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:29.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:29.996+0000 7f01c2000640 1 -- 172.21.15.47:0/1333093089 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f01b803e070 con 0x7f01c8156c20 2024-04-20T10:27:30.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.205+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f01c80cfd90 con 0x7f01c8156c20 2024-04-20T10:27:30.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.961+0000 7f01c2000640 1 -- 172.21.15.47:0/1333093089 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f01b807bf10 con 0x7f01c8156c20 2024-04-20T10:27:30.962 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-20T10:27:30.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.966+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f01ac06c8f0 msgr2=0x7f01ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:30.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.966+0000 7f01d0800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f01ac06c8f0 0x7f01ac06edb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009ad0 tx=0x7f01bc008040 comp rx=0 tx=0).stop 2024-04-20T10:27:30.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.966+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c8156c20 msgr2=0x7f01c80ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:30.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.966+0000 7f01d0800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c8156c20 0x7f01c80ccb30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f01b8000c00 tx=0x7f01b80046c0 comp rx=0 tx=0).stop 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 shutdown_connections 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01c815ca40 0x7f01c80cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f01ac06c8f0 0x7f01ac06edb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01c8156c20 0x7f01c80ccb30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 --2- 172.21.15.47:0/1333093089 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01c80ad8e0 0x7f01c80998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 >> 172.21.15.47:0/1333093089 conn(0x7f01c80b3350 msgr2=0x7f01c81521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 shutdown_connections 2024-04-20T10:27:30.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:30.968+0000 7f01d0800640 1 -- 172.21.15.47:0/1333093089 wait complete. 2024-04-20T10:27:30.994 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-20T10:27:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.196+0000 7f7303200640 1 Processor -- start 2024-04-20T10:27:31.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.197+0000 7f7303200640 1 -- start start 2024-04-20T10:27:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.198+0000 7f7303200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f72fc0d0380 0x7f72fc0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.198+0000 7f7303200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72fc0d1e90 con 0x7f72fc0d4270 2024-04-20T10:27:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.199+0000 7f7303200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f72fc0d0d10 con 0x7f72fc0d1a00 2024-04-20T10:27:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.199+0000 7f7303200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f72fc0d40c0 con 0x7f72fc0d0380 2024-04-20T10:27:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.199+0000 7f72fbe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f72fc0d4270 0x7f72fc0d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:38844/0 (socket says 172.21.15.47:38844) 2024-04-20T10:27:31.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.199+0000 7f72fbe00640 1 -- 172.21.15.47:0/239180950 learned_addr learned my addr 172.21.15.47:0/239180950 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.199+0000 7f7301200640 1 --2- 172.21.15.47:0/239180950 >> v2:172.21.15.175:3300/0 conn(0x7f72fc0d0380 0x7f72fc0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.199+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4225525206 0 0) 0x7f72ec001110 con 0x7f72fc0d4270 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.199+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f72e0003520 con 0x7f72fc0d4270 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1573467671 0 0) 0x7f72e0003520 con 0x7f72fc0d4270 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 >> v2:172.21.15.175:3300/0 conn(0x7f72fc0d0380 msgr2=0x7f72fc0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 --2- 172.21.15.47:0/239180950 >> v2:172.21.15.175:3300/0 conn(0x7f72fc0d0380 0x7f72fc0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 >> v1:172.21.15.175:6789/0 conn(0x7f72fc0d1a00 legacy=0x7f72fc0b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc0d3820 con 0x7f72fc0d4270 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f72ec001840 con 0x7f72fc0d4270 2024-04-20T10:27:31.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 >> v1:172.21.15.47:6789/0 conn(0x7f72fc0d4270 legacy=0x7f72fc0d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e00039b0 0x7f72e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e00051c0 0x7f72e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc0d40c0 con 0x7f72e0004800 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72e000c060 con 0x7f72e00039b0 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72e000c260 con 0x7f72e00051c0 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.200+0000 7f72fb400640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.201+0000 7f72fbe00640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e00039b0 0x7f72e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.201+0000 7f72fb400640 1 -- 172.21.15.47:0/239180950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e00051c0 msgr2=0x7f72e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.201+0000 7f72fb400640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72e00051c0 0x7f72e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.201+0000 7f72fb400640 1 -- 172.21.15.47:0/239180950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e00039b0 msgr2=0x7f72e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.201+0000 7f72fb400640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72e00039b0 0x7f72e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.201+0000 7f72fb400640 1 -- 172.21.15.47:0/239180950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e0003520 con 0x7f72e0004800 2024-04-20T10:27:31.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.201+0000 7f7301200640 1 -- 172.21.15.47:0/239180950 reap_dead start 2024-04-20T10:27:31.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.202+0000 7f72fb400640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72e0004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72e8001750 tx=0x7f72e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d6189fc67ec378 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.202+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8001370 con 0x7f72e0004800 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.202+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72e80052d0 con 0x7f72e0004800 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.202+0000 7f72faa00640 1 -- 172.21.15.47:0/239180950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e8003830 con 0x7f72e0004800 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.203+0000 7f7303200640 1 -- 172.21.15.47:0/239180950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 msgr2=0x7f72e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.203+0000 7f7303200640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72e0004c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f72e8001750 tx=0x7f72e0005830 comp rx=0 tx=0).stop 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.203+0000 7f7303200640 1 -- 172.21.15.47:0/239180950 shutdown_connections 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.203+0000 7f7303200640 1 --2- 172.21.15.47:0/239180950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72e0004c80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.203+0000 7f7303200640 1 -- 172.21.15.47:0/239180950 >> 172.21.15.47:0/239180950 conn(0x7f72fc0b99c0 msgr2=0x7f72fc0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.203+0000 7f7303200640 1 -- 172.21.15.47:0/239180950 shutdown_connections 2024-04-20T10:27:31.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.203+0000 7f7303200640 1 -- 172.21.15.47:0/239180950 wait complete. 2024-04-20T10:27:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.205+0000 7f7303200640 1 Processor -- start 2024-04-20T10:27:31.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.205+0000 7f7303200640 1 -- start start 2024-04-20T10:27:31.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f7303200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72fc14fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f7303200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc0b5090 0x7f72fc0b54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f7303200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72fc06c9a0 0x7f72fc06ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f7303200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc0cfe60 con 0x7f72e0004800 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f7303200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72fc0d3de0 con 0x7f72fc0b5090 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f7303200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72fc0d53b0 con 0x7f72fc06c9a0 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72fc14fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72fc14fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58534/0 (socket says 172.21.15.47:58534) 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 -- 172.21.15.47:0/3169697868 learned_addr learned my addr 172.21.15.47:0/3169697868 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 -- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72fc06c9a0 msgr2=0x7f72fc06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72fc06c9a0 0x7f72fc06ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 -- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc0b5090 msgr2=0x7f72fc0b54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc0b5090 0x7f72fc0b54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.206+0000 7f72fbe00640 1 -- 172.21.15.47:0/3169697868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72fc14ff70 con 0x7f72e0004800 2024-04-20T10:27:31.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.207+0000 7f72fbe00640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72fc14fa30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e700 tx=0x7f72ec00ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:31.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.207+0000 7f72f3e00640 1 -- 172.21.15.47:0/3169697868 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00ee10 con 0x7f72e0004800 2024-04-20T10:27:31.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.207+0000 7f72f3e00640 1 -- 172.21.15.47:0/3169697868 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f72ec018070 con 0x7f72e0004800 2024-04-20T10:27:31.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.207+0000 7f72f3e00640 1 -- 172.21.15.47:0/3169697868 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72ec00ca90 con 0x7f72e0004800 2024-04-20T10:27:31.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.207+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72fc06d360 con 0x7f72e0004800 2024-04-20T10:27:31.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.207+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72fc1f7f90 con 0x7f72e0004800 2024-04-20T10:27:31.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.208+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72fc068980 con 0x7f72e0004800 2024-04-20T10:27:31.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.209+0000 7f72f3e00640 1 -- 172.21.15.47:0/3169697868 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f72ec00cc30 con 0x7f72e0004800 2024-04-20T10:27:31.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.210+0000 7f72f3e00640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f72dc06c8f0 0x7f72dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.210+0000 7f72f3e00640 1 -- 172.21.15.47:0/3169697868 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f72ec08b870 con 0x7f72e0004800 2024-04-20T10:27:31.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.211+0000 7f72fb400640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f72dc06c8f0 0x7f72dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:31.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.212+0000 7f72fb400640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f72dc06c8f0 0x7f72dc06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72e8033000 tx=0x7f72e802f000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:31.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.215+0000 7f72f3e00640 1 -- 172.21.15.47:0/3169697868 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f72fc14ff70 con 0x7f72e0004800 2024-04-20T10:27:31.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:31.419+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f72fc161120 con 0x7f72e0004800 2024-04-20T10:27:32.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.043+0000 7f72f3e00640 1 -- 172.21.15.47:0/3169697868 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v21) ==== 171+0+0 (secure 0 0 0) 0x7f72fc0d1e90 con 0x7f72e0004800 2024-04-20T10:27:32.044 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f72dc06c8f0 msgr2=0x7f72dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f72dc06c8f0 0x7f72dc06edb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f72e8033000 tx=0x7f72e802f000 comp rx=0 tx=0).stop 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 msgr2=0x7f72fc14fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72fc14fa30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f72ec00e700 tx=0x7f72ec00ebc0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 shutdown_connections 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72fc06c9a0 0x7f72fc06ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72fc0b5090 0x7f72fc0b54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f72dc06c8f0 0x7f72dc06edb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 --2- 172.21.15.47:0/3169697868 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f72e0004800 0x7f72fc14fa30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.046+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 >> 172.21.15.47:0/3169697868 conn(0x7f72fc0b99c0 msgr2=0x7f72fc0b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.047+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 shutdown_connections 2024-04-20T10:27:32.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.047+0000 7f7303200640 1 -- 172.21.15.47:0/3169697868 wait complete. 2024-04-20T10:27:32.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T10:27:32.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.269+0000 7f62dd200640 1 Processor -- start 2024-04-20T10:27:32.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.269+0000 7f62dd200640 1 -- start start 2024-04-20T10:27:32.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.271+0000 7f62dd200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62d80d0380 0x7f62d80d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:32.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.271+0000 7f62dd200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80d1e90 con 0x7f62d80d1a00 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.271+0000 7f62dd200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62d80d0d10 con 0x7f62d80d4270 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.271+0000 7f62dd200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f62d80d40c0 con 0x7f62d80d0380 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.271+0000 7f62d6800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62d80d0380 0x7f62d80d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.271+0000 7f62d6800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f62d80d0380 0x7f62d80d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58544/0 (socket says 172.21.15.47:58544) 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.271+0000 7f62d6800640 1 -- 172.21.15.47:0/106469142 learned_addr learned my addr 172.21.15.47:0/106469142 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.272+0000 7f62d6800640 1 -- 172.21.15.47:0/106469142 >> v1:172.21.15.175:6789/0 conn(0x7f62d80d4270 legacy=0x7f62d80d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.272+0000 7f62d6800640 1 -- 172.21.15.47:0/106469142 >> v1:172.21.15.47:6789/0 conn(0x7f62d80d1a00 legacy=0x7f62d80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.272+0000 7f62d6800640 1 -- 172.21.15.47:0/106469142 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d80d3820 con 0x7f62d80d0380 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.273+0000 7f62d6800640 1 --2- 172.21.15.47:0/106469142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d80d07d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009e50 tx=0x7f62d002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7595b66a19a0b97f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.273+0000 7f62d4a00640 1 -- 172.21.15.47:0/106469142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d002cb10 con 0x7f62d80d0380 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.273+0000 7f62d4a00640 1 -- 172.21.15.47:0/106469142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62d002ccb0 con 0x7f62d80d0380 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.273+0000 7f62d4a00640 1 -- 172.21.15.47:0/106469142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62d002ce60 con 0x7f62d80d0380 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.274+0000 7f62dd200640 1 -- 172.21.15.47:0/106469142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 msgr2=0x7f62d80d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.274+0000 7f62dd200640 1 --2- 172.21.15.47:0/106469142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d80d07d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f62d0009e50 tx=0x7f62d002bf70 comp rx=0 tx=0).stop 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.274+0000 7f62dd200640 1 -- 172.21.15.47:0/106469142 shutdown_connections 2024-04-20T10:27:32.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.274+0000 7f62dd200640 1 --2- 172.21.15.47:0/106469142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d80d07d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.274+0000 7f62dd200640 1 -- 172.21.15.47:0/106469142 >> 172.21.15.47:0/106469142 conn(0x7f62d80b99c0 msgr2=0x7f62d80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.274+0000 7f62dd200640 1 -- 172.21.15.47:0/106469142 shutdown_connections 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.274+0000 7f62dd200640 1 -- 172.21.15.47:0/106469142 wait complete. 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.275+0000 7f62dd200640 1 Processor -- start 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.276+0000 7f62dd200640 1 -- start start 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.276+0000 7f62dd200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d8155520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.276+0000 7f62dd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62d80d1a00 0x7f62d806ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62dd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62d80d4270 0x7f62d80cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62dd200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80cfe60 con 0x7f62d80d0380 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62dd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62d80d3de0 con 0x7f62d80d4270 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62dd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62d80b19a0 con 0x7f62d80d1a00 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d8155520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d8155520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58552/0 (socket says 172.21.15.47:58552) 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 -- 172.21.15.47:0/1265154405 learned_addr learned my addr 172.21.15.47:0/1265154405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:32.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5400640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62d80d1a00 0x7f62d806ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 -- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62d80d1a00 msgr2=0x7f62d806ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62d80d1a00 0x7f62d806ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 -- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62d80d4270 msgr2=0x7f62d80cd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62d80d4270 0x7f62d80cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5e00640 1 -- 172.21.15.47:0/1265154405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62d80cab60 con 0x7f62d80d0380 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.277+0000 7f62d5400640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62d80d1a00 0x7f62d806ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.278+0000 7f62d5e00640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d8155520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c410 tx=0x7f62c800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.278+0000 7f62ce000640 1 -- 172.21.15.47:0/1265154405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8017070 con 0x7f62d80d0380 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.278+0000 7f62ce000640 1 -- 172.21.15.47:0/1265154405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62c80035f0 con 0x7f62d80d0380 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.278+0000 7f62ce000640 1 -- 172.21.15.47:0/1265154405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62c8003a20 con 0x7f62d80d0380 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.278+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62d80cae40 con 0x7f62d80d0380 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.278+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62d8155a60 con 0x7f62d80d0380 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.279+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62d8068980 con 0x7f62d80d0380 2024-04-20T10:27:32.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.281+0000 7f62ce000640 1 -- 172.21.15.47:0/1265154405 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f62c801a070 con 0x7f62d80d0380 2024-04-20T10:27:32.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.282+0000 7f62ce000640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f62b806c8f0 0x7f62b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:32.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.282+0000 7f62ce000640 1 -- 172.21.15.47:0/1265154405 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f62c808b000 con 0x7f62d80d0380 2024-04-20T10:27:32.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.284+0000 7f62d5400640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f62b806c8f0 0x7f62b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:32.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.286+0000 7f62d5400640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f62b806c8f0 0x7f62b806edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62d815ec90 tx=0x7f62c4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:32.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.288+0000 7f62ce000640 1 -- 172.21.15.47:0/1265154405 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f62c8059490 con 0x7f62d80d0380 2024-04-20T10:27:32.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:32.531+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f62d80b3b30 con 0x7f62d80d0380 2024-04-20T10:27:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.044+0000 7f62ce000640 1 -- 172.21.15.47:0/1265154405 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f62c80010c0 con 0x7f62d80d0380 2024-04-20T10:27:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 2 and data pool 3 2024-04-20T10:27:33.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f62b806c8f0 msgr2=0x7f62b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f62b806c8f0 0x7f62b806edb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f62d815ec90 tx=0x7f62c4009040 comp rx=0 tx=0).stop 2024-04-20T10:27:33.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 msgr2=0x7f62d8155520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d8155520 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f62c800c410 tx=0x7f62c800c8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 shutdown_connections 2024-04-20T10:27:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62d80d4270 0x7f62d80cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62d80d1a00 0x7f62d806ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f62b806c8f0 0x7f62b806edb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 --2- 172.21.15.47:0/1265154405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f62d80d0380 0x7f62d8155520 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 >> 172.21.15.47:0/1265154405 conn(0x7f62d80b99c0 msgr2=0x7f62d80676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 shutdown_connections 2024-04-20T10:27:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.047+0000 7f62dd200640 1 -- 172.21.15.47:0/1265154405 wait complete. 2024-04-20T10:27:33.070 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-20T10:27:33.070 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-20T10:27:33.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.251+0000 7fe83a600640 1 Processor -- start 2024-04-20T10:27:33.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.251+0000 7fe83a600640 1 -- start start 2024-04-20T10:27:33.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.253+0000 7fe83a600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8340d1a00 0x7fe8340b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:33.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.253+0000 7fe83a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340b17d0 0x7fe8340d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:33.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.253+0000 7fe83a600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe8340d1e90 con 0x7fe8340d4270 2024-04-20T10:27:33.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.253+0000 7fe83a600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe8340d40c0 con 0x7fe8340d1a00 2024-04-20T10:27:33.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.253+0000 7fe83a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8340d3820 con 0x7fe8340b17d0 2024-04-20T10:27:33.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.254+0000 7fe832a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8340d1a00 0x7fe8340b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:33.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.254+0000 7fe833e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340b17d0 0x7fe8340d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:33.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.254+0000 7fe832a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe8340d1a00 0x7fe8340b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:34370/0 (socket says 172.21.15.47:34370) 2024-04-20T10:27:33.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.254+0000 7fe832a00640 1 -- 172.21.15.47:0/3940206816 learned_addr learned my addr 172.21.15.47:0/3940206816 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:33.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.254+0000 7fe832a00640 1 -- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340b17d0 msgr2=0x7fe8340d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:33.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.255+0000 7fe832a00640 1 --2- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340b17d0 0x7fe8340d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.255+0000 7fe832a00640 1 -- 172.21.15.47:0/3940206816 >> v1:172.21.15.175:6789/0 conn(0x7fe8340d4270 legacy=0x7fe8340d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.255+0000 7fe832a00640 1 -- 172.21.15.47:0/3940206816 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8340d3b00 con 0x7fe8340d1a00 2024-04-20T10:27:33.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.255+0000 7fe833e00640 1 --2- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340b17d0 0x7fe8340d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:33.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.256+0000 7fe832a00640 1 --2- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d1a00 0x7fe8340b1290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe820009870 tx=0x7fe82002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=383fc324e34b14b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:33.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.256+0000 7fe832000640 1 -- 172.21.15.47:0/3940206816 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820034070 con 0x7fe8340d1a00 2024-04-20T10:27:33.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.256+0000 7fe832000640 1 -- 172.21.15.47:0/3940206816 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe820030840 con 0x7fe8340d1a00 2024-04-20T10:27:33.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.256+0000 7fe832000640 1 -- 172.21.15.47:0/3940206816 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe820030c10 con 0x7fe8340d1a00 2024-04-20T10:27:33.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 -- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d1a00 msgr2=0x7fe8340b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 --2- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d1a00 0x7fe8340b1290 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe820009870 tx=0x7fe82002b980 comp rx=0 tx=0).stop 2024-04-20T10:27:33.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 -- 172.21.15.47:0/3940206816 shutdown_connections 2024-04-20T10:27:33.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 --2- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340b17d0 0x7fe8340d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 --2- 172.21.15.47:0/3940206816 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d1a00 0x7fe8340b1290 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 -- 172.21.15.47:0/3940206816 >> 172.21.15.47:0/3940206816 conn(0x7fe8340b99c0 msgr2=0x7fe8340b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:33.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 -- 172.21.15.47:0/3940206816 shutdown_connections 2024-04-20T10:27:33.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.257+0000 7fe83a600640 1 -- 172.21.15.47:0/3940206816 wait complete. 2024-04-20T10:27:33.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.259+0000 7fe83a600640 1 Processor -- start 2024-04-20T10:27:33.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.259+0000 7fe83a600640 1 -- start start 2024-04-20T10:27:33.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe83a600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8340b17d0 0x7fe834156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:33.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe83a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340d1a00 0x7fe83406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:33.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe83a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d4270 0x7fe8340c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:33.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe83a600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340d53b0 con 0x7fe8340b17d0 2024-04-20T10:27:33.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe83a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8340cfe60 con 0x7fe8340d4270 2024-04-20T10:27:33.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe83a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe834161f70 con 0x7fe8340d1a00 2024-04-20T10:27:33.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe833400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8340b17d0 0x7fe834156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:33.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe833400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8340b17d0 0x7fe834156ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58564/0 (socket says 172.21.15.47:58564) 2024-04-20T10:27:33.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.260+0000 7fe833400640 1 -- 172.21.15.47:0/1435901138 learned_addr learned my addr 172.21.15.47:0/1435901138 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe833e00640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d4270 0x7fe8340c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe832a00640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340d1a00 0x7fe83406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe833400640 1 -- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340d1a00 msgr2=0x7fe83406d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe833400640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340d1a00 0x7fe83406d930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe833400640 1 -- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d4270 msgr2=0x7fe8340c7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe833400640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d4270 0x7fe8340c7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe833400640 1 -- 172.21.15.47:0/1435901138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8341570e0 con 0x7fe8340b17d0 2024-04-20T10:27:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe833400640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8340b17d0 0x7fe834156ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe82400c430 tx=0x7fe82400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe82b400640 1 -- 172.21.15.47:0/1435901138 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8240052d0 con 0x7fe8340b17d0 2024-04-20T10:27:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe82b400640 1 -- 172.21.15.47:0/1435901138 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe824005490 con 0x7fe8340b17d0 2024-04-20T10:27:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8340cb750 con 0x7fe8340b17d0 2024-04-20T10:27:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe82b400640 1 -- 172.21.15.47:0/1435901138 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe824007260 con 0x7fe8340b17d0 2024-04-20T10:27:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.261+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8340cbd10 con 0x7fe8340b17d0 2024-04-20T10:27:33.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.263+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8340742b0 con 0x7fe8340b17d0 2024-04-20T10:27:33.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.264+0000 7fe82b400640 1 -- 172.21.15.47:0/1435901138 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe824007400 con 0x7fe8340b17d0 2024-04-20T10:27:33.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.265+0000 7fe82b400640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe81406c8f0 0x7fe81406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:33.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.265+0000 7fe82b400640 1 -- 172.21.15.47:0/1435901138 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7fe82408b170 con 0x7fe8340b17d0 2024-04-20T10:27:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.266+0000 7fe832a00640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe81406c8f0 0x7fe81406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:33.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.267+0000 7fe832a00640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe81406c8f0 0x7fe81406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe82003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:33.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.270+0000 7fe82b400640 1 -- 172.21.15.47:0/1435901138 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe8340742b0 con 0x7fe8340b17d0 2024-04-20T10:27:33.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.478+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fe8340cc420 con 0x7fe8340b17d0 2024-04-20T10:27:33.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.973+0000 7fe82b400640 1 -- 172.21.15.47:0/1435901138 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v23) ==== 157+0+0 (secure 0 0 0) 0x7fe8341570e0 con 0x7fe8340b17d0 2024-04-20T10:27:33.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.977+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe81406c8f0 msgr2=0x7fe81406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.977+0000 7fe83a600640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe81406c8f0 0x7fe81406edb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe82003b040 comp rx=0 tx=0).stop 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.977+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8340b17d0 msgr2=0x7fe834156ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.977+0000 7fe83a600640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8340b17d0 0x7fe834156ba0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe82400c430 tx=0x7fe82400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 shutdown_connections 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8340d4270 0x7fe8340c7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8340d1a00 0x7fe83406d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe81406c8f0 0x7fe81406edb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 --2- 172.21.15.47:0/1435901138 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe8340b17d0 0x7fe834156ba0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:33.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 >> 172.21.15.47:0/1435901138 conn(0x7fe8340b99c0 msgr2=0x7fe834067750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 shutdown_connections 2024-04-20T10:27:33.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:33.978+0000 7fe83a600640 1 -- 172.21.15.47:0/1435901138 wait complete. 2024-04-20T10:27:34.001 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T10:27:34.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.230+0000 7f2c77e00640 1 Processor -- start 2024-04-20T10:27:34.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.230+0000 7f2c77e00640 1 -- start start 2024-04-20T10:27:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.231+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7815b030 0x7f2c780af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:34.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.231+0000 7f2c77e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c780af730 0x7f2c78072750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c77e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2c780ae150 con 0x7f2c7815b410 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c77e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2c780ae430 con 0x7f2c780af730 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c77e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c780ae710 con 0x7f2c7815b030 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c76000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c780af730 0x7f2c78072750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c76000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2c780af730 0x7f2c78072750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41726/0 (socket says 172.21.15.47:41726) 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c76000640 1 -- 172.21.15.47:0/3042975905 learned_addr learned my addr 172.21.15.47:0/3042975905 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c76a00640 1 --2- 172.21.15.47:0/3042975905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7815b030 0x7f2c780af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c76000640 1 -- 172.21.15.47:0/3042975905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7815b030 msgr2=0x7f2c780af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.232+0000 7f2c76000640 1 --2- 172.21.15.47:0/3042975905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7815b030 0x7f2c780af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.233+0000 7f2c76000640 1 -- 172.21.15.47:0/3042975905 >> v1:172.21.15.175:6789/0 conn(0x7f2c7815b410 legacy=0x7f2c7806ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:34.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.233+0000 7f2c76000640 1 -- 172.21.15.47:0/3042975905 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c780ae9f0 con 0x7f2c780af730 2024-04-20T10:27:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.234+0000 7f2c76000640 1 --2- 172.21.15.47:0/3042975905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c780af730 0x7f2c78072750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009870 tx=0x7f2c6002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f71b6a98759961d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.234+0000 7f2c75600640 1 -- 172.21.15.47:0/3042975905 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60034070 con 0x7f2c780af730 2024-04-20T10:27:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.234+0000 7f2c75600640 1 -- 172.21.15.47:0/3042975905 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c60030840 con 0x7f2c780af730 2024-04-20T10:27:34.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.234+0000 7f2c75600640 1 -- 172.21.15.47:0/3042975905 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60030c10 con 0x7f2c780af730 2024-04-20T10:27:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 -- 172.21.15.47:0/3042975905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c780af730 msgr2=0x7f2c78072750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 --2- 172.21.15.47:0/3042975905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c780af730 0x7f2c78072750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009870 tx=0x7f2c6002b980 comp rx=0 tx=0).stop 2024-04-20T10:27:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 -- 172.21.15.47:0/3042975905 shutdown_connections 2024-04-20T10:27:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 --2- 172.21.15.47:0/3042975905 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c780af730 0x7f2c78072750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 --2- 172.21.15.47:0/3042975905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c7815b030 0x7f2c780af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:34.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 -- 172.21.15.47:0/3042975905 >> 172.21.15.47:0/3042975905 conn(0x7f2c780b32e0 msgr2=0x7f2c780b36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 -- 172.21.15.47:0/3042975905 shutdown_connections 2024-04-20T10:27:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.235+0000 7f2c77e00640 1 -- 172.21.15.47:0/3042975905 wait complete. 2024-04-20T10:27:34.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.237+0000 7f2c77e00640 1 Processor -- start 2024-04-20T10:27:34.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.237+0000 7f2c77e00640 1 -- start start 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7815b030 0x7f2c78150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7815b410 0x7f2c78153540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c77e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c78153a80 0x7f2c7815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c77e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2c780adec0 con 0x7f2c7815b410 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c77e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c780add40 con 0x7f2c7815b030 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c77e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c7806d120 con 0x7f2c78153a80 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c76000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7815b410 0x7f2c78153540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c76000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7815b410 0x7f2c78153540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41728/0 (socket says 172.21.15.47:41728) 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c76000640 1 -- 172.21.15.47:0/1182776691 learned_addr learned my addr 172.21.15.47:0/1182776691 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:34.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c76a00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7815b030 0x7f2c78150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c77400640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c78153a80 0x7f2c7815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c76000640 1 -- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c78153a80 msgr2=0x7f2c7815b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.238+0000 7f2c76000640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c78153a80 0x7f2c7815b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c76000640 1 -- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7815b030 msgr2=0x7f2c78150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c76000640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7815b030 0x7f2c78150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c76000640 1 -- 172.21.15.47:0/1182776691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c7815c5d0 con 0x7f2c7815b410 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c77400640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c78153a80 0x7f2c7815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c76000640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7815b410 0x7f2c78153540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2c60000c00 tx=0x7f2c60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c6ea00640 1 -- 172.21.15.47:0/1182776691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60048070 con 0x7f2c7815b410 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c6ea00640 1 -- 172.21.15.47:0/1182776691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2c60030640 con 0x7f2c7815b410 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c6ea00640 1 -- 172.21.15.47:0/1182776691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c60034040 con 0x7f2c7815b410 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c7815c8b0 con 0x7f2c7815b410 2024-04-20T10:27:34.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.239+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c7815ce70 con 0x7f2c7815b410 2024-04-20T10:27:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.241+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c78075b60 con 0x7f2c7815b410 2024-04-20T10:27:34.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.241+0000 7f2c6ea00640 1 -- 172.21.15.47:0/1182776691 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2c6004b070 con 0x7f2c7815b410 2024-04-20T10:27:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.243+0000 7f2c6ea00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2c5406c8f0 0x7f2c5406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.243+0000 7f2c6ea00640 1 -- 172.21.15.47:0/1182776691 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6007+0+0 (secure 0 0 0) 0x7f2c600adc50 con 0x7f2c7815b410 2024-04-20T10:27:34.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.244+0000 7f2c76a00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2c5406c8f0 0x7f2c5406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:34.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.245+0000 7f2c76a00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2c5406c8f0 0x7f2c5406edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c781f7a60 tx=0x7f2c6800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:34.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.248+0000 7f2c6ea00640 1 -- 172.21.15.47:0/1182776691 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2c6007beb0 con 0x7f2c7815b410 2024-04-20T10:27:34.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:34.454+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2c78151750 con 0x7f2c7815b410 2024-04-20T10:27:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.070+0000 7f2c6ea00640 1 -- 172.21.15.47:0/1182776691 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v25) ==== 228+0+0 (secure 0 0 0) 0x7f2c6007b7b0 con 0x7f2c7815b410 2024-04-20T10:27:36.071 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' created 2024-04-20T10:27:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.075+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2c5406c8f0 msgr2=0x7f2c5406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:36.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.075+0000 7f2c77e00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2c5406c8f0 0x7f2c5406edb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2c781f7a60 tx=0x7f2c6800b040 comp rx=0 tx=0).stop 2024-04-20T10:27:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.075+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7815b410 msgr2=0x7f2c78153540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.075+0000 7f2c77e00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7815b410 0x7f2c78153540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f2c60000c00 tx=0x7f2c60004730 comp rx=0 tx=0).stop 2024-04-20T10:27:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.076+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 shutdown_connections 2024-04-20T10:27:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.076+0000 7f2c77e00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c78153a80 0x7f2c7815b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.076+0000 7f2c77e00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2c5406c8f0 0x7f2c5406edb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.076+0000 7f2c77e00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2c7815b410 0x7f2c78153540 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.076+0000 7f2c77e00640 1 --2- 172.21.15.47:0/1182776691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c7815b030 0x7f2c78150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.076+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 >> 172.21.15.47:0/1182776691 conn(0x7f2c780b32e0 msgr2=0x7f2c780d02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.077+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 shutdown_connections 2024-04-20T10:27:36.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.077+0000 7f2c77e00640 1 -- 172.21.15.47:0/1182776691 wait complete. 2024-04-20T10:27:36.108 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-20T10:27:36.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.303+0000 7fa1a1a00640 1 Processor -- start 2024-04-20T10:27:36.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.303+0000 7fa1a1a00640 1 -- start start 2024-04-20T10:27:36.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.304+0000 7fa1a1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c15b320 0x7fa19c06ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:36.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.304+0000 7fa1a1a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa19c0d0eb0 0x7fa19c06f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:36.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.305+0000 7fa1a1a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa19c15fe90 con 0x7fa19c15f9b0 2024-04-20T10:27:36.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.305+0000 7fa1a1a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa19c0cf1a0 con 0x7fa19c0d0eb0 2024-04-20T10:27:36.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.305+0000 7fa1a1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa19c0cf480 con 0x7fa19c15b320 2024-04-20T10:27:36.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.305+0000 7fa193e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa19c0d0eb0 0x7fa19c06f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:36.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.305+0000 7fa193e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa19c0d0eb0 0x7fa19c06f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:53900/0 (socket says 172.21.15.47:53900) 2024-04-20T10:27:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.306+0000 7fa193e00640 1 -- 172.21.15.47:0/3016696924 learned_addr learned my addr 172.21.15.47:0/3016696924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.306+0000 7fa19aa00640 1 --2- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c15b320 0x7fa19c06ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.306+0000 7fa193e00640 1 -- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c15b320 msgr2=0x7fa19c06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.306+0000 7fa193e00640 1 --2- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c15b320 0x7fa19c06ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.306+0000 7fa193e00640 1 -- 172.21.15.47:0/3016696924 >> v1:172.21.15.175:6789/0 conn(0x7fa19c15f9b0 legacy=0x7fa19c0b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:36.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.306+0000 7fa193e00640 1 -- 172.21.15.47:0/3016696924 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c0cf760 con 0x7fa19c0d0eb0 2024-04-20T10:27:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.308+0000 7fa193e00640 1 --2- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c0d0eb0 0x7fa19c06f390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa18400bb10 tx=0x7fa18402e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a170cc87638911 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:36.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.308+0000 7fa19a000640 1 -- 172.21.15.47:0/3016696924 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18403b070 con 0x7fa19c0d0eb0 2024-04-20T10:27:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa19a000640 1 -- 172.21.15.47:0/3016696924 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa184036440 con 0x7fa19c0d0eb0 2024-04-20T10:27:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa19a000640 1 -- 172.21.15.47:0/3016696924 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184036810 con 0x7fa19c0d0eb0 2024-04-20T10:27:36.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 -- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c0d0eb0 msgr2=0x7fa19c06f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c0d0eb0 0x7fa19c06f390 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa18400bb10 tx=0x7fa18402e0c0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 -- 172.21.15.47:0/3016696924 shutdown_connections 2024-04-20T10:27:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c0d0eb0 0x7fa19c06f390 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/3016696924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c15b320 0x7fa19c06ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 -- 172.21.15.47:0/3016696924 >> 172.21.15.47:0/3016696924 conn(0x7fa19c070450 msgr2=0x7fa19c070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 -- 172.21.15.47:0/3016696924 shutdown_connections 2024-04-20T10:27:36.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.309+0000 7fa1a1a00640 1 -- 172.21.15.47:0/3016696924 wait complete. 2024-04-20T10:27:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.311+0000 7fa1a1a00640 1 Processor -- start 2024-04-20T10:27:36.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.311+0000 7fa1a1a00640 1 -- start start 2024-04-20T10:27:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.312+0000 7fa1a1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c15b320 0x7fa19c1f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.312+0000 7fa1a1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa19c15f9b0 0x7fa19c0ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.312+0000 7fa1a1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c1ee580 0x7fa19c1559b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.312+0000 7fa1a1a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c15b880 con 0x7fa19c15f9b0 2024-04-20T10:27:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.312+0000 7fa1a1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa19c15b700 con 0x7fa19c15b320 2024-04-20T10:27:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.312+0000 7fa1a1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa19c0b4a60 con 0x7fa19c1ee580 2024-04-20T10:27:36.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa19c15f9b0 0x7fa19c0ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa19c15f9b0 0x7fa19c0ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41734/0 (socket says 172.21.15.47:41734) 2024-04-20T10:27:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 -- 172.21.15.47:0/1958778090 learned_addr learned my addr 172.21.15.47:0/1958778090 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa19aa00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c15b320 0x7fa19c1f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:36.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa19b400640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c1ee580 0x7fa19c1559b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 -- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c1ee580 msgr2=0x7fa19c1559b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c1ee580 0x7fa19c1559b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 -- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c15b320 msgr2=0x7fa19c1f47f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c15b320 0x7fa19c1f47f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.313+0000 7fa193e00640 1 -- 172.21.15.47:0/1958778090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa19c0ae7c0 con 0x7fa19c15f9b0 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.314+0000 7fa193e00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa19c15f9b0 0x7fa19c0ae280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa18402e100 tx=0x7fa1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.314+0000 7fa192a00640 1 -- 172.21.15.47:0/1958778090 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18403b030 con 0x7fa19c15f9b0 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.314+0000 7fa192a00640 1 -- 172.21.15.47:0/1958778090 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa18403d050 con 0x7fa19c15f9b0 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.314+0000 7fa192a00640 1 -- 172.21.15.47:0/1958778090 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184034870 con 0x7fa19c15f9b0 2024-04-20T10:27:36.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.314+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa19c1f4dd0 con 0x7fa19c15f9b0 2024-04-20T10:27:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.314+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa19c1f5390 con 0x7fa19c15f9b0 2024-04-20T10:27:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.315+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa19c068980 con 0x7fa19c15f9b0 2024-04-20T10:27:36.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.317+0000 7fa192a00640 1 -- 172.21.15.47:0/1958778090 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa184048060 con 0x7fa19c15f9b0 2024-04-20T10:27:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.318+0000 7fa192a00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa17806c8f0 0x7fa17806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:36.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.318+0000 7fa192a00640 1 -- 172.21.15.47:0/1958778090 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6500+0+0 (secure 0 0 0) 0x7fa1840ad6c0 con 0x7fa19c15f9b0 2024-04-20T10:27:36.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.318+0000 7fa19aa00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa17806c8f0 0x7fa17806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:36.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.319+0000 7fa19aa00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa17806c8f0 0x7fa17806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0b2040 tx=0x7fa18c009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:36.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.323+0000 7fa192a00640 1 -- 172.21.15.47:0/1958778090 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa18407b860 con 0x7fa19c15f9b0 2024-04-20T10:27:36.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:36.531+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fa19c072fe0 con 0x7fa19c15f9b0 2024-04-20T10:27:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.075+0000 7fa192a00640 1 -- 172.21.15.47:0/1958778090 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v26) ==== 169+0+0 (secure 0 0 0) 0x7fa18407b160 con 0x7fa19c15f9b0 2024-04-20T10:27:37.076 INFO:teuthology.orchestra.run.smithi047.stderr:set pool 4 allow_ec_overwrites to true 2024-04-20T10:27:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.079+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa17806c8f0 msgr2=0x7fa17806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.079+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa17806c8f0 0x7fa17806edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0b2040 tx=0x7fa18c009190 comp rx=0 tx=0).stop 2024-04-20T10:27:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.079+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa19c15f9b0 msgr2=0x7fa19c0ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.079+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa19c15f9b0 0x7fa19c0ae280 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa18402e100 tx=0x7fa1840046c0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.080+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 shutdown_connections 2024-04-20T10:27:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.080+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa19c1ee580 0x7fa19c1559b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.080+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa17806c8f0 0x7fa17806edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.080+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa19c15f9b0 0x7fa19c0ae280 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.080+0000 7fa1a1a00640 1 --2- 172.21.15.47:0/1958778090 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa19c15b320 0x7fa19c1f47f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.080+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 >> 172.21.15.47:0/1958778090 conn(0x7fa19c070450 msgr2=0x7fa19c1575d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.081+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 shutdown_connections 2024-04-20T10:27:37.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.084+0000 7fa1a1a00640 1 -- 172.21.15.47:0/1958778090 wait complete. 2024-04-20T10:27:37.124 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-20T10:27:37.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.305+0000 7fe4a8e00640 1 Processor -- start 2024-04-20T10:27:37.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.306+0000 7fe4a8e00640 1 -- start start 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a8e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe4a406db90 0x7fe4a40d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a40ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a8e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe4a40afb40 con 0x7fe4a4099900 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a8e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe4a406baa0 con 0x7fe4a406db90 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a406bd80 con 0x7fe4a40ad160 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe4a406db90 0x7fe4a40d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a2a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe4a4099900 0x7fe4a406df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57024/0 (socket says 172.21.15.47:57024) 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.307+0000 7fe4a2a00640 1 -- 172.21.15.47:0/4162095828 learned_addr learned my addr 172.21.15.47:0/4162095828 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:37.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.308+0000 7fe4a3e00640 1 --2- 172.21.15.47:0/4162095828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a40ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.308+0000 7fe4a3400640 1 -- 172.21.15.47:0/4162095828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 msgr2=0x7fe4a40ad5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.308+0000 7fe4a3400640 1 --2- 172.21.15.47:0/4162095828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a40ad5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.308+0000 7fe4a3400640 1 -- 172.21.15.47:0/4162095828 >> v1:172.21.15.47:6789/0 conn(0x7fe4a4099900 legacy=0x7fe4a406df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.308+0000 7fe4a3400640 1 -- 172.21.15.47:0/4162095828 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a406c060 con 0x7fe4a406db90 2024-04-20T10:27:37.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.308+0000 7fe4a2000640 1 -- 172.21.15.47:0/4162095828 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2789145233 0 0) 0x7fe4a40afb40 con 0x7fe4a4099900 2024-04-20T10:27:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.309+0000 7fe4a3400640 1 --2- 172.21.15.47:0/4162095828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a406db90 0x7fe4a40d0760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe494009d90 tx=0x7fe49402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1012811a1d93be6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.309+0000 7fe4a2000640 1 -- 172.21.15.47:0/4162095828 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494039070 con 0x7fe4a406db90 2024-04-20T10:27:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.309+0000 7fe4a2000640 1 -- 172.21.15.47:0/4162095828 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe49402cc50 con 0x7fe4a406db90 2024-04-20T10:27:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.309+0000 7fe4a2000640 1 -- 172.21.15.47:0/4162095828 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe494034440 con 0x7fe4a406db90 2024-04-20T10:27:37.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 -- 172.21.15.47:0/4162095828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a406db90 msgr2=0x7fe4a40d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/4162095828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a406db90 0x7fe4a40d0760 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe494009d90 tx=0x7fe49402bea0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 -- 172.21.15.47:0/4162095828 shutdown_connections 2024-04-20T10:27:37.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/4162095828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a40ad5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/4162095828 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a406db90 0x7fe4a40d0760 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 -- 172.21.15.47:0/4162095828 >> 172.21.15.47:0/4162095828 conn(0x7fe4a40b5270 msgr2=0x7fe4a40b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:37.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 -- 172.21.15.47:0/4162095828 shutdown_connections 2024-04-20T10:27:37.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.310+0000 7fe4a8e00640 1 -- 172.21.15.47:0/4162095828 wait complete. 2024-04-20T10:27:37.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.312+0000 7fe4a8e00640 1 Processor -- start 2024-04-20T10:27:37.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.312+0000 7fe4a8e00640 1 -- start start 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a406db90 0x7fe4a415e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a8e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4099900 0x7fe4a4073510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a4073a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a8e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a4162af0 con 0x7fe4a4099900 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a406c620 con 0x7fe4a406db90 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a4162c70 con 0x7fe4a40ad160 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4099900 0x7fe4a4073510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4099900 0x7fe4a4073510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41738/0 (socket says 172.21.15.47:41738) 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.313+0000 7fe4a2a00640 1 -- 172.21.15.47:0/3936995405 learned_addr learned my addr 172.21.15.47:0/3936995405 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a3e00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a4073a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:37.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a3400640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a406db90 0x7fe4a415e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a2a00640 1 -- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 msgr2=0x7fe4a4073a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a2a00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a4073a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a2a00640 1 -- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a406db90 msgr2=0x7fe4a415e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a2a00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a406db90 0x7fe4a415e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a2a00640 1 -- 172.21.15.47:0/3936995405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a409e280 con 0x7fe4a4099900 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe4a2a00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4099900 0x7fe4a4073510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c420 tx=0x7fe48c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.314+0000 7fe49b400640 1 -- 172.21.15.47:0/3936995405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c0074e0 con 0x7fe4a4099900 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.315+0000 7fe49b400640 1 -- 172.21.15.47:0/3936995405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe48c0047e0 con 0x7fe4a4099900 2024-04-20T10:27:37.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.315+0000 7fe49b400640 1 -- 172.21.15.47:0/3936995405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c004c10 con 0x7fe4a4099900 2024-04-20T10:27:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.315+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a415e5c0 con 0x7fe4a4099900 2024-04-20T10:27:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.315+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a409dfa0 con 0x7fe4a4099900 2024-04-20T10:27:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.316+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a4075b30 con 0x7fe4a4099900 2024-04-20T10:27:37.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.317+0000 7fe49b400640 1 -- 172.21.15.47:0/3936995405 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe48c011070 con 0x7fe4a4099900 2024-04-20T10:27:37.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.318+0000 7fe49b400640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe48006c8f0 0x7fe48006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:37.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.318+0000 7fe49b400640 1 -- 172.21.15.47:0/3936995405 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 6500+0+0 (secure 0 0 0) 0x7fe48c08ba50 con 0x7fe4a4099900 2024-04-20T10:27:37.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.318+0000 7fe4a3400640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe48006c8f0 0x7fe48006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:37.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.319+0000 7fe4a3400640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe48006c8f0 0x7fe48006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe494036000 tx=0x7fe494030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:37.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.322+0000 7fe49b400640 1 -- 172.21.15.47:0/3936995405 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe48c00e280 con 0x7fe4a4099900 2024-04-20T10:27:37.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:37.569+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe4a40b1270 con 0x7fe4a4099900 2024-04-20T10:27:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.112+0000 7fe49b400640 1 -- 172.21.15.47:0/3936995405 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) ==== 137+0+0 (secure 0 0 0) 0x7fe48c059b50 con 0x7fe4a4099900 2024-04-20T10:27:38.113 INFO:teuthology.orchestra.run.smithi047.stderr:added data pool 4 to fsmap 2024-04-20T10:27:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe48006c8f0 msgr2=0x7fe48006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe48006c8f0 0x7fe48006edb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe494036000 tx=0x7fe494030000 comp rx=0 tx=0).stop 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4099900 msgr2=0x7fe4a4073510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4099900 0x7fe4a4073510 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00c420 tx=0x7fe48c00c8e0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 shutdown_connections 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a40ad160 0x7fe4a4073a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe48006c8f0 0x7fe48006edb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe4a4099900 0x7fe4a4073510 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 --2- 172.21.15.47:0/3936995405 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a406db90 0x7fe4a415e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 >> 172.21.15.47:0/3936995405 conn(0x7fe4a40b5270 msgr2=0x7fe4a40b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.116+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 shutdown_connections 2024-04-20T10:27:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.117+0000 7fe4a8e00640 1 -- 172.21.15.47:0/3936995405 wait complete. 2024-04-20T10:27:38.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:27:38.217 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-04-20T10:27:38.215+0000 7fc5b5400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:27:38.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.347+0000 7f263b200640 1 Processor -- start 2024-04-20T10:27:38.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.348+0000 7f263b200640 1 -- start start 2024-04-20T10:27:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.349+0000 7f263b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634162f90 0x7f26340ce550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.349+0000 7f263b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2634166b90 0x7f2634167000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.349+0000 7f263b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2634163690 con 0x7f26340c55b0 2024-04-20T10:27:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.349+0000 7f263b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2634163440 con 0x7f2634166b90 2024-04-20T10:27:38.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.349+0000 7f263b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2634163970 con 0x7f2634162f90 2024-04-20T10:27:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.350+0000 7f263a800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2634166b90 0x7f2634167000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.350+0000 7f2639400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634162f90 0x7f26340ce550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.350+0000 7f263a800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2634166b90 0x7f2634167000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:53916/0 (socket says 172.21.15.47:53916) 2024-04-20T10:27:38.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.350+0000 7f263a800640 1 -- 172.21.15.47:0/1069192889 learned_addr learned my addr 172.21.15.47:0/1069192889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.351+0000 7f263a800640 1 -- 172.21.15.47:0/1069192889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634162f90 msgr2=0x7f26340ce550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.351+0000 7f263a800640 1 --2- 172.21.15.47:0/1069192889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634162f90 0x7f26340ce550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.351+0000 7f263a800640 1 -- 172.21.15.47:0/1069192889 >> v1:172.21.15.175:6789/0 conn(0x7f26340c55b0 legacy=0x7f26340cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.351+0000 7f263a800640 1 -- 172.21.15.47:0/1069192889 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2634163c50 con 0x7f2634166b90 2024-04-20T10:27:38.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.352+0000 7f263a800640 1 --2- 172.21.15.47:0/1069192889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634166b90 0x7f2634167000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2630009e50 tx=0x7f263002bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8fb85442dea64d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.353+0000 7f2638a00640 1 -- 172.21.15.47:0/1069192889 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263002ca70 con 0x7f2634166b90 2024-04-20T10:27:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.353+0000 7f2638a00640 1 -- 172.21.15.47:0/1069192889 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f263002cc10 con 0x7f2634166b90 2024-04-20T10:27:38.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.353+0000 7f2638a00640 1 -- 172.21.15.47:0/1069192889 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300343f0 con 0x7f2634166b90 2024-04-20T10:27:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 -- 172.21.15.47:0/1069192889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634166b90 msgr2=0x7f2634167000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 --2- 172.21.15.47:0/1069192889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634166b90 0x7f2634167000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2630009e50 tx=0x7f263002bed0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 -- 172.21.15.47:0/1069192889 shutdown_connections 2024-04-20T10:27:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 --2- 172.21.15.47:0/1069192889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2634166b90 0x7f2634167000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 --2- 172.21.15.47:0/1069192889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634162f90 0x7f26340ce550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 -- 172.21.15.47:0/1069192889 >> 172.21.15.47:0/1069192889 conn(0x7f26340b3df0 msgr2=0x7f26340b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 -- 172.21.15.47:0/1069192889 shutdown_connections 2024-04-20T10:27:38.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.354+0000 7f263b200640 1 -- 172.21.15.47:0/1069192889 wait complete. 2024-04-20T10:27:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.356+0000 7f263b200640 1 Processor -- start 2024-04-20T10:27:38.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.356+0000 7f263b200640 1 -- start start 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.357+0000 7f263b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340c55b0 0x7f2634070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.357+0000 7f263b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2634162f90 0x7f2634162850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.357+0000 7f263b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634166b90 0x7f2634165170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.357+0000 7f263b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f26340d0540 con 0x7f2634162f90 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.357+0000 7f263b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26340c5990 con 0x7f26340c55b0 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.357+0000 7f263b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26340711f0 con 0x7f2634166b90 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2634162f90 0x7f2634162850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2634162f90 0x7f2634162850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41740/0 (socket says 172.21.15.47:41740) 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 -- 172.21.15.47:0/1724448464 learned_addr learned my addr 172.21.15.47:0/1724448464 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639e00640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340c55b0 0x7f2634070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f263a800640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634166b90 0x7f2634165170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 -- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634166b90 msgr2=0x7f2634165170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634166b90 0x7f2634165170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 -- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340c55b0 msgr2=0x7f2634070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340c55b0 0x7f2634070cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 -- 172.21.15.47:0/1724448464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2634070110 con 0x7f2634162f90 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.358+0000 7f2639400640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2634162f90 0x7f2634162850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f262400e9b0 tx=0x7f262400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.359+0000 7f262e000640 1 -- 172.21.15.47:0/1724448464 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2624017070 con 0x7f2634162f90 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.359+0000 7f262e000640 1 -- 172.21.15.47:0/1724448464 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f262400c9a0 con 0x7f2634162f90 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.359+0000 7f262e000640 1 -- 172.21.15.47:0/1724448464 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262400cd90 con 0x7f2634162f90 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.359+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2634162d90 con 0x7f2634162f90 2024-04-20T10:27:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.359+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263406daf0 con 0x7f2634162f90 2024-04-20T10:27:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.360+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634075b30 con 0x7f2634162f90 2024-04-20T10:27:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.361+0000 7f262e000640 1 -- 172.21.15.47:0/1724448464 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f262401a070 con 0x7f2634162f90 2024-04-20T10:27:38.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.362+0000 7f262e000640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f261806c8f0 0x7f261806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.362+0000 7f262e000640 1 -- 172.21.15.47:0/1724448464 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f262408bfd0 con 0x7f2634162f90 2024-04-20T10:27:38.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.362+0000 7f2639e00640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f261806c8f0 0x7f261806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.364+0000 7f2639e00640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f261806c8f0 0x7f261806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2628001640 tx=0x7f2628009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:38.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.367+0000 7f262e000640 1 -- 172.21.15.47:0/1724448464 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2634070110 con 0x7f2634162f90 2024-04-20T10:27:38.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.614+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26340d1970 con 0x7f2634162f90 2024-04-20T10:27:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.616+0000 7f262e000640 1 -- 172.21.15.47:0/1724448464 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5201 (secure 0 0 0) 0x7f2624059f90 con 0x7f2634162f90 2024-04-20T10:27:38.616 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:38.616 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:37.571421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:27:38.616 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-04-20T10:27:38.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f261806c8f0 msgr2=0x7f261806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f261806c8f0 0x7f261806edb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2628001640 tx=0x7f2628009040 comp rx=0 tx=0).stop 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2634162f90 msgr2=0x7f2634162850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2634162f90 0x7f2634162850 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f262400e9b0 tx=0x7f262400ee70 comp rx=0 tx=0).stop 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 shutdown_connections 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2634166b90 0x7f2634165170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f261806c8f0 0x7f261806edb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2634162f90 0x7f2634162850 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 --2- 172.21.15.47:0/1724448464 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26340c55b0 0x7f2634070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 >> 172.21.15.47:0/1724448464 conn(0x7f26340b3df0 msgr2=0x7f26340b4dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.618+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 shutdown_connections 2024-04-20T10:27:38.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.619+0000 7f263b200640 1 -- 172.21.15.47:0/1724448464 wait complete. 2024-04-20T10:27:38.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:38.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.844+0000 7fb3fd000640 1 Processor -- start 2024-04-20T10:27:38.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.844+0000 7fb3fd000640 1 -- start start 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3fd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f815b3b0 0x7fb3f806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3fd000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3f80d0360 0x7fb3f80d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3fd000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3f80ade40 con 0x7fb3f806dcb0 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3fd000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb3f80ae0e0 con 0x7fb3f80d0360 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3fd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80d0cf0 con 0x7fb3f815b3b0 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3f6800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3f80d0360 0x7fb3f80d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3f5400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb3f806dcb0 0x7fb3f8071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57028/0 (socket says 172.21.15.47:57028) 2024-04-20T10:27:38.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3f5400640 1 -- 172.21.15.47:0/4011636077 learned_addr learned my addr 172.21.15.47:0/4011636077 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.846+0000 7fb3f6800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3f80d0360 0x7fb3f80d07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41744/0 (socket says 172.21.15.47:41744) 2024-04-20T10:27:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.847+0000 7fb3f5e00640 1 --2- 172.21.15.47:0/4011636077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f815b3b0 0x7fb3f806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.847+0000 7fb3f6800640 1 -- 172.21.15.47:0/4011636077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f815b3b0 msgr2=0x7fb3f806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.847+0000 7fb3f6800640 1 --2- 172.21.15.47:0/4011636077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f815b3b0 0x7fb3f806eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.847+0000 7fb3f6800640 1 -- 172.21.15.47:0/4011636077 >> v1:172.21.15.47:6789/0 conn(0x7fb3f806dcb0 legacy=0x7fb3f8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.847+0000 7fb3f4a00640 1 -- 172.21.15.47:0/4011636077 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2279231446 0 0) 0x7fb3f80ade40 con 0x7fb3f806dcb0 2024-04-20T10:27:38.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.847+0000 7fb3f6800640 1 -- 172.21.15.47:0/4011636077 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f815b790 con 0x7fb3f80d0360 2024-04-20T10:27:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.848+0000 7fb3f6800640 1 --2- 172.21.15.47:0/4011636077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f80d0360 0x7fb3f80d07b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009d90 tx=0x7fb3f002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1d1fb413a8084f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:38.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.848+0000 7fb3f4a00640 1 -- 172.21.15.47:0/4011636077 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0039070 con 0x7fb3f80d0360 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.848+0000 7fb3f4a00640 1 -- 172.21.15.47:0/4011636077 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f002cc50 con 0x7fb3f80d0360 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.848+0000 7fb3f4a00640 1 -- 172.21.15.47:0/4011636077 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0034440 con 0x7fb3f80d0360 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 -- 172.21.15.47:0/4011636077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f80d0360 msgr2=0x7fb3f80d07b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 --2- 172.21.15.47:0/4011636077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f80d0360 0x7fb3f80d07b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009d90 tx=0x7fb3f002bea0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 -- 172.21.15.47:0/4011636077 shutdown_connections 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 --2- 172.21.15.47:0/4011636077 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f80d0360 0x7fb3f80d07b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 --2- 172.21.15.47:0/4011636077 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f815b3b0 0x7fb3f806eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 -- 172.21.15.47:0/4011636077 >> 172.21.15.47:0/4011636077 conn(0x7fb3f80b3350 msgr2=0x7fb3f80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 -- 172.21.15.47:0/4011636077 shutdown_connections 2024-04-20T10:27:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.849+0000 7fb3fd000640 1 -- 172.21.15.47:0/4011636077 wait complete. 2024-04-20T10:27:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.851+0000 7fb3fd000640 1 Processor -- start 2024-04-20T10:27:38.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.851+0000 7fb3fd000640 1 -- start start 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3fd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f806dcb0 0x7fb3f815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3fd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f80d0360 0x7fb3f8151ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3fd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f815b3b0 0x7fb3f8151210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3fd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80ae660 con 0x7fb3f815b3b0 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3fd000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80ae380 con 0x7fb3f806dcb0 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3fd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80ae940 con 0x7fb3f80d0360 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f815b3b0 0x7fb3f8151210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f815b3b0 0x7fb3f8151210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41750/0 (socket says 172.21.15.47:41750) 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3f6800640 1 -- 172.21.15.47:0/2532195865 learned_addr learned my addr 172.21.15.47:0/2532195865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.852+0000 7fb3f5e00640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f806dcb0 0x7fb3f815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f5400640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f80d0360 0x7fb3f8151ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f6800640 1 -- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f80d0360 msgr2=0x7fb3f8151ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f6800640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f80d0360 0x7fb3f8151ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f6800640 1 -- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f806dcb0 msgr2=0x7fb3f815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f6800640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f806dcb0 0x7fb3f815c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f6800640 1 -- 172.21.15.47:0/2532195865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f815e260 con 0x7fb3f815b3b0 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f5400640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f80d0360 0x7fb3f8151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3f6800640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f815b3b0 0x7fb3f8151210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0037040 tx=0x7fb3f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3ee000640 1 -- 172.21.15.47:0/2532195865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0039050 con 0x7fb3f815b3b0 2024-04-20T10:27:38.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3ee000640 1 -- 172.21.15.47:0/2532195865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3f0045070 con 0x7fb3f815b3b0 2024-04-20T10:27:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f815e540 con 0x7fb3f815b3b0 2024-04-20T10:27:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.853+0000 7fb3ee000640 1 -- 172.21.15.47:0/2532195865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0009450 con 0x7fb3f815b3b0 2024-04-20T10:27:38.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.854+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f8165fc0 con 0x7fb3f815b3b0 2024-04-20T10:27:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.855+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f8068980 con 0x7fb3f815b3b0 2024-04-20T10:27:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.856+0000 7fb3ee000640 1 -- 172.21.15.47:0/2532195865 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3f003e610 con 0x7fb3f815b3b0 2024-04-20T10:27:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.856+0000 7fb3ee000640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3d806c8f0 0x7fb3d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.857+0000 7fb3ee000640 1 -- 172.21.15.47:0/2532195865 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb3f00aea10 con 0x7fb3f815b3b0 2024-04-20T10:27:38.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.857+0000 7fb3f5e00640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3d806c8f0 0x7fb3d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:38.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.858+0000 7fb3f5e00640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3d806c8f0 0x7fb3d806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006fd0 tx=0x7fb3e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:38.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:38.861+0000 7fb3ee000640 1 -- 172.21.15.47:0/2532195865 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb3f007ca60 con 0x7fb3f815b3b0 2024-04-20T10:27:39.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.065+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3f815fe00 con 0x7fb3f815b3b0 2024-04-20T10:27:39.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.067+0000 7fb3ee000640 1 -- 172.21.15.47:0/2532195865 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7fb3f007c360 con 0x7fb3f815b3b0 2024-04-20T10:27:39.068 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:39.068 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3d806c8f0 msgr2=0x7fb3d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3d806c8f0 0x7fb3d806edb0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8006fd0 tx=0x7fb3e8008040 comp rx=0 tx=0).stop 2024-04-20T10:27:39.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f815b3b0 msgr2=0x7fb3f8151210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f815b3b0 0x7fb3f8151210 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0037040 tx=0x7fb3f0004a40 comp rx=0 tx=0).stop 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 shutdown_connections 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3d806c8f0 0x7fb3d806edb0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3f815b3b0 0x7fb3f8151210 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3f80d0360 0x7fb3f8151ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 --2- 172.21.15.47:0/2532195865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3f806dcb0 0x7fb3f815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 >> 172.21.15.47:0/2532195865 conn(0x7fb3f80b3350 msgr2=0x7fb3f80b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 shutdown_connections 2024-04-20T10:27:39.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.070+0000 7fb3fd000640 1 -- 172.21.15.47:0/2532195865 wait complete. 2024-04-20T10:27:39.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:39.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.301+0000 7ff8cb200640 1 Processor -- start 2024-04-20T10:27:39.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.302+0000 7ff8cb200640 1 -- start start 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8cb200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8cc1651d0 0x7ff8cc071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8cb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff8cc163450 0x7ff8cc1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8cb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff8cc06c310 con 0x7ff8cc163070 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8cb200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc073cf0 con 0x7ff8cc1651d0 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8cb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff8cc06c5f0 con 0x7ff8cc163450 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8c9400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8cc1651d0 0x7ff8cc071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8c9400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff8cc1651d0 0x7ff8cc071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41764/0 (socket says 172.21.15.47:41764) 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.304+0000 7ff8c9400640 1 -- 172.21.15.47:0/2841510010 learned_addr learned my addr 172.21.15.47:0/2841510010 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:39.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.305+0000 7ff8ca800640 1 --2- 172.21.15.47:0/2841510010 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc163450 0x7ff8cc1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.305+0000 7ff8c9400640 1 -- 172.21.15.47:0/2841510010 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc163450 msgr2=0x7ff8cc1638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.305+0000 7ff8c9400640 1 --2- 172.21.15.47:0/2841510010 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc163450 0x7ff8cc1638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.305+0000 7ff8c9400640 1 -- 172.21.15.47:0/2841510010 >> v1:172.21.15.175:6789/0 conn(0x7ff8cc163070 legacy=0x7ff8cc0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.305+0000 7ff8c9400640 1 -- 172.21.15.47:0/2841510010 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc06c8d0 con 0x7ff8cc1651d0 2024-04-20T10:27:39.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.306+0000 7ff8c9400640 1 --2- 172.21.15.47:0/2841510010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc071d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0d1140 tx=0x7ff8b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ebbb4c20fc33fe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8c8a00640 1 -- 172.21.15.47:0/2841510010 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b402ca80 con 0x7ff8cc1651d0 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8c8a00640 1 -- 172.21.15.47:0/2841510010 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8b402cc20 con 0x7ff8cc1651d0 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8c8a00640 1 -- 172.21.15.47:0/2841510010 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b40343f0 con 0x7ff8cc1651d0 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8cb200640 1 -- 172.21.15.47:0/2841510010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc1651d0 msgr2=0x7ff8cc071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8cb200640 1 --2- 172.21.15.47:0/2841510010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc071d90 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc0d1140 tx=0x7ff8b402bee0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8cb200640 1 -- 172.21.15.47:0/2841510010 shutdown_connections 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8cb200640 1 --2- 172.21.15.47:0/2841510010 >> v2:172.21.15.175:3300/0 conn(0x7ff8cc163450 0x7ff8cc1638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8cb200640 1 --2- 172.21.15.47:0/2841510010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc071d90 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.307+0000 7ff8cb200640 1 -- 172.21.15.47:0/2841510010 >> 172.21.15.47:0/2841510010 conn(0x7ff8cc0d0040 msgr2=0x7ff8cc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.308+0000 7ff8cb200640 1 -- 172.21.15.47:0/2841510010 shutdown_connections 2024-04-20T10:27:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.308+0000 7ff8cb200640 1 -- 172.21.15.47:0/2841510010 wait complete. 2024-04-20T10:27:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.309+0000 7ff8cb200640 1 Processor -- start 2024-04-20T10:27:39.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.309+0000 7ff8cb200640 1 -- start start 2024-04-20T10:27:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.310+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc163070 0x7ff8cc09d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.310+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc163450 0x7ff8cc153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.310+0000 7ff8cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.312 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.310+0000 7ff8cb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc163e20 con 0x7ff8cc163070 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.310+0000 7ff8cb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc06ce90 con 0x7ff8cc1651d0 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.310+0000 7ff8cb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff8cc1662a0 con 0x7ff8cc163450 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc163070 0x7ff8cc09d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc163070 0x7ff8cc09d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41772/0 (socket says 172.21.15.47:41772) 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 -- 172.21.15.47:0/151201892 learned_addr learned my addr 172.21.15.47:0/151201892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8ca800640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 -- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc163450 msgr2=0x7ff8cc153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc163450 0x7ff8cc153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 -- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1651d0 msgr2=0x7ff8cc0c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc0c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 -- 172.21.15.47:0/151201892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc1541e0 con 0x7ff8cc163070 2024-04-20T10:27:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8ca800640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.311+0000 7ff8c9e00640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc163070 0x7ff8cc09d470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c430 tx=0x7ff8bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.312+0000 7ff8c2000640 1 -- 172.21.15.47:0/151201892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc017070 con 0x7ff8cc163070 2024-04-20T10:27:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.312+0000 7ff8c2000640 1 -- 172.21.15.47:0/151201892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0052d0 con 0x7ff8cc163070 2024-04-20T10:27:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.312+0000 7ff8c2000640 1 -- 172.21.15.47:0/151201892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc003830 con 0x7ff8cc163070 2024-04-20T10:27:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.312+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc0c5420 con 0x7ff8cc163070 2024-04-20T10:27:39.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.312+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc06e920 con 0x7ff8cc163070 2024-04-20T10:27:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.313+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc075b60 con 0x7ff8cc163070 2024-04-20T10:27:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.314+0000 7ff8c2000640 1 -- 172.21.15.47:0/151201892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff8bc01a070 con 0x7ff8cc163070 2024-04-20T10:27:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.315+0000 7ff8c2000640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff8a806c940 0x7ff8a806ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.315+0000 7ff8c2000640 1 -- 172.21.15.47:0/151201892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff8bc08b1a0 con 0x7ff8cc163070 2024-04-20T10:27:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.316+0000 7ff8c9400640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff8a806c940 0x7ff8a806ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.317+0000 7ff8c9400640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff8a806c940 0x7ff8a806ee00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4035000 tx=0x7ff8b402e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:39.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.319+0000 7ff8c2000640 1 -- 172.21.15.47:0/151201892 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff8bc059370 con 0x7ff8cc163070 2024-04-20T10:27:39.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.526+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8cc164d70 con 0x7ff8cc163070 2024-04-20T10:27:39.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.528+0000 7ff8c2000640 1 -- 172.21.15.47:0/151201892 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7ff8bc05f020 con 0x7ff8cc163070 2024-04-20T10:27:39.529 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:39.529 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:39.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff8a806c940 msgr2=0x7ff8a806ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff8a806c940 0x7ff8a806ee00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4035000 tx=0x7ff8b402e000 comp rx=0 tx=0).stop 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc163070 msgr2=0x7ff8cc09d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc163070 0x7ff8cc09d470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c430 tx=0x7ff8bc00c8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 shutdown_connections 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff8cc1651d0 0x7ff8cc0c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff8cc163450 0x7ff8cc153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff8a806c940 0x7ff8a806ee00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 --2- 172.21.15.47:0/151201892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff8cc163070 0x7ff8cc09d470 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 >> 172.21.15.47:0/151201892 conn(0x7ff8cc0d0040 msgr2=0x7ff8cc164030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 shutdown_connections 2024-04-20T10:27:39.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.531+0000 7ff8cb200640 1 -- 172.21.15.47:0/151201892 wait complete. 2024-04-20T10:27:39.567 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:27:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.780+0000 7f4717000640 1 Processor -- start 2024-04-20T10:27:39.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.781+0000 7f4717000640 1 -- start start 2024-04-20T10:27:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4717000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47100d1600 0x7f47100b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4717000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f47100d19e0 0x7f47100b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4717000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47100cf810 con 0x7f47100d05d0 2024-04-20T10:27:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4717000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f47100cfaf0 con 0x7f47100d1600 2024-04-20T10:27:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4717000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f47100cfdd0 con 0x7f47100d19e0 2024-04-20T10:27:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4715c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47100d1600 0x7f47100b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4716600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f47100d05d0 0x7f47101622b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57042/0 (socket says 172.21.15.47:57042) 2024-04-20T10:27:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4716600640 1 -- 172.21.15.47:0/2501543555 learned_addr learned my addr 172.21.15.47:0/2501543555 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:39.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.783+0000 7f4715200640 1 --2- 172.21.15.47:0/2501543555 >> v2:172.21.15.175:3300/0 conn(0x7f47100d19e0 0x7f47100b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.784+0000 7f4715c00640 1 -- 172.21.15.47:0/2501543555 >> v2:172.21.15.175:3300/0 conn(0x7f47100d19e0 msgr2=0x7f47100b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:39.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.784+0000 7f4715c00640 1 --2- 172.21.15.47:0/2501543555 >> v2:172.21.15.175:3300/0 conn(0x7f47100d19e0 0x7f47100b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.784+0000 7f4715c00640 1 -- 172.21.15.47:0/2501543555 >> v1:172.21.15.47:6789/0 conn(0x7f47100d05d0 legacy=0x7f47101622b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.784+0000 7f4715c00640 1 -- 172.21.15.47:0/2501543555 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47100d00b0 con 0x7f47100d1600 2024-04-20T10:27:39.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.785+0000 7f4715c00640 1 --2- 172.21.15.47:0/2501543555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d1600 0x7f47100b4bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47100b64f0 tx=0x7f470402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d66f6da10391f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:39.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.785+0000 7f470fe00640 1 -- 172.21.15.47:0/2501543555 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704038070 con 0x7f47100d1600 2024-04-20T10:27:39.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.785+0000 7f470fe00640 1 -- 172.21.15.47:0/2501543555 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4704031710 con 0x7f47100d1600 2024-04-20T10:27:39.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.785+0000 7f470fe00640 1 -- 172.21.15.47:0/2501543555 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4704031ae0 con 0x7f47100d1600 2024-04-20T10:27:39.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 -- 172.21.15.47:0/2501543555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d1600 msgr2=0x7f47100b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 --2- 172.21.15.47:0/2501543555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d1600 0x7f47100b4bf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f47100b64f0 tx=0x7f470402b7b0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 -- 172.21.15.47:0/2501543555 shutdown_connections 2024-04-20T10:27:39.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 --2- 172.21.15.47:0/2501543555 >> v2:172.21.15.175:3300/0 conn(0x7f47100d19e0 0x7f47100b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 --2- 172.21.15.47:0/2501543555 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d1600 0x7f47100b4bf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 -- 172.21.15.47:0/2501543555 >> 172.21.15.47:0/2501543555 conn(0x7f47100b2e70 msgr2=0x7f47100b3250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:39.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 -- 172.21.15.47:0/2501543555 shutdown_connections 2024-04-20T10:27:39.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.786+0000 7f4717000640 1 -- 172.21.15.47:0/2501543555 wait complete. 2024-04-20T10:27:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.788+0000 7f4717000640 1 Processor -- start 2024-04-20T10:27:39.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.788+0000 7f4717000640 1 -- start start 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4717000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47100d05d0 0x7f4710159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4717000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d19e0 0x7f47101614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4717000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f471014cc70 0x7f47101e9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4717000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4710162d50 con 0x7f47100d19e0 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4717000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4710162bd0 con 0x7f47100d05d0 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4717000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47100aded0 con 0x7f471014cc70 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4715200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d19e0 0x7f47101614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4715200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d19e0 0x7f47101614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41798/0 (socket says 172.21.15.47:41798) 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.789+0000 7f4715200640 1 -- 172.21.15.47:0/2865164383 learned_addr learned my addr 172.21.15.47:0/2865164383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:39.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4715200640 1 -- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f471014cc70 msgr2=0x7f47101e9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4715200640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f471014cc70 0x7f47101e9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4715200640 1 -- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47100d05d0 msgr2=0x7f4710159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:39.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4715200640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47100d05d0 0x7f4710159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:39.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4715200640 1 -- 172.21.15.47:0/2865164383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471014d100 con 0x7f47100d19e0 2024-04-20T10:27:39.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4715200640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d19e0 0x7f47101614f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00d7d0 tx=0x7f46fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f470d600640 1 -- 172.21.15.47:0/2865164383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc004020 con 0x7f47100d19e0 2024-04-20T10:27:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f470d600640 1 -- 172.21.15.47:0/2865164383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f46fc0041e0 con 0x7f47100d19e0 2024-04-20T10:27:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f470d600640 1 -- 172.21.15.47:0/2865164383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0043d0 con 0x7f47100d19e0 2024-04-20T10:27:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47101e8bd0 con 0x7f47100d19e0 2024-04-20T10:27:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.790+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47101e9190 con 0x7f47100d19e0 2024-04-20T10:27:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.792+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4710075b60 con 0x7f47100d19e0 2024-04-20T10:27:39.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.793+0000 7f470d600640 1 -- 172.21.15.47:0/2865164383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f46fc01b020 con 0x7f47100d19e0 2024-04-20T10:27:39.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.793+0000 7f470d600640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f46f006c8f0 0x7f46f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:39.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.794+0000 7f470d600640 1 -- 172.21.15.47:0/2865164383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f46fc08bc70 con 0x7f47100d19e0 2024-04-20T10:27:39.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.794+0000 7f4715c00640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f46f006c8f0 0x7f46f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:39.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.795+0000 7f4715c00640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f46f006c8f0 0x7f46f006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4704035000 tx=0x7f4704030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:39.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:39.798+0000 7f470d600640 1 -- 172.21.15.47:0/2865164383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f46fc059dd0 con 0x7f47100d19e0 2024-04-20T10:27:40.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.041+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f471015a2d0 con 0x7f47100d19e0 2024-04-20T10:27:40.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.042+0000 7f470d600640 1 -- 172.21.15.47:0/2865164383 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5201 (secure 0 0 0) 0x7f46fc0596d0 con 0x7f47100d19e0 2024-04-20T10:27:40.042 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:40.042 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:37.571421+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:27:40.043 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 7 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f46f006c8f0 msgr2=0x7f46f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f46f006c8f0 0x7f46f006edb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f4704035000 tx=0x7f4704030000 comp rx=0 tx=0).stop 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d19e0 msgr2=0x7f47101614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d19e0 0x7f47101614f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00d7d0 tx=0x7f46fc00dc90 comp rx=0 tx=0).stop 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 shutdown_connections 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f471014cc70 0x7f47101e9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f46f006c8f0 0x7f46f006edb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47100d19e0 0x7f47101614f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 --2- 172.21.15.47:0/2865164383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47100d05d0 0x7f4710159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.044+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 >> 172.21.15.47:0/2865164383 conn(0x7f47100b2e70 msgr2=0x7f47100afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:40.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.045+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 shutdown_connections 2024-04-20T10:27:40.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.045+0000 7f4717000640 1 -- 172.21.15.47:0/2865164383 wait complete. 2024-04-20T10:27:40.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.276+0000 7fa62b000640 1 Processor -- start 2024-04-20T10:27:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.276+0000 7fa62b000640 1 -- start start 2024-04-20T10:27:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.278+0000 7fa62b000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa6240b2f70 0x7fa6240b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.278+0000 7fa62b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240adf60 0x7fa62415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.278+0000 7fa62b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa624071a80 con 0x7fa62415faf0 2024-04-20T10:27:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.278+0000 7fa62b000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa6240ae3c0 con 0x7fa6240b2f70 2024-04-20T10:27:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.278+0000 7fa62b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa624071cf0 con 0x7fa6240adf60 2024-04-20T10:27:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.279+0000 7fa629200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240adf60 0x7fa62415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.279+0000 7fa629c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa6240b2f70 0x7fa6240b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.279+0000 7fa629200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240adf60 0x7fa62415f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:56284/0 (socket says 172.21.15.47:56284) 2024-04-20T10:27:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.279+0000 7fa629200640 1 -- 172.21.15.47:0/1803527799 learned_addr learned my addr 172.21.15.47:0/1803527799 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.280+0000 7fa629c00640 1 -- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240adf60 msgr2=0x7fa62415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.280+0000 7fa629c00640 1 --2- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240adf60 0x7fa62415f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.280+0000 7fa629c00640 1 -- 172.21.15.47:0/1803527799 >> v1:172.21.15.175:6789/0 conn(0x7fa62415faf0 legacy=0x7fa62406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.280+0000 7fa629c00640 1 -- 172.21.15.47:0/1803527799 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa624071fd0 con 0x7fa6240b2f70 2024-04-20T10:27:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.280+0000 7fa629200640 1 --2- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240adf60 0x7fa62415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:40.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.281+0000 7fa629c00640 1 --2- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6240b2f70 0x7fa6240b4320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6140097a0 tx=0x7fa61402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99f5a53b34919f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:40.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.281+0000 7fa623e00640 1 -- 172.21.15.47:0/1803527799 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614031780 con 0x7fa6240b2f70 2024-04-20T10:27:40.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa623e00640 1 -- 172.21.15.47:0/1803527799 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa614037070 con 0x7fa6240b2f70 2024-04-20T10:27:40.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa623e00640 1 -- 172.21.15.47:0/1803527799 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614031b40 con 0x7fa6240b2f70 2024-04-20T10:27:40.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 -- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6240b2f70 msgr2=0x7fa6240b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:40.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 --2- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6240b2f70 0x7fa6240b4320 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa6140097a0 tx=0x7fa61402b820 comp rx=0 tx=0).stop 2024-04-20T10:27:40.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 -- 172.21.15.47:0/1803527799 shutdown_connections 2024-04-20T10:27:40.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 --2- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240adf60 0x7fa62415f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 --2- 172.21.15.47:0/1803527799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6240b2f70 0x7fa6240b4320 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 -- 172.21.15.47:0/1803527799 >> 172.21.15.47:0/1803527799 conn(0x7fa6240d0040 msgr2=0x7fa6240d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 -- 172.21.15.47:0/1803527799 shutdown_connections 2024-04-20T10:27:40.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.282+0000 7fa62b000640 1 -- 172.21.15.47:0/1803527799 wait complete. 2024-04-20T10:27:40.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.284+0000 7fa62b000640 1 Processor -- start 2024-04-20T10:27:40.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.284+0000 7fa62b000640 1 -- start start 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.285+0000 7fa62b000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6240adf60 0x7fa6241f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.285+0000 7fa62b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240b2f70 0x7fa6240b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.285+0000 7fa62b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62415faf0 0x7fa62414c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.285+0000 7fa62b000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa62406c010 con 0x7fa6240adf60 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.285+0000 7fa62b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6240736c0 con 0x7fa62415faf0 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.285+0000 7fa62b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa62406c190 con 0x7fa6240b2f70 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.285+0000 7fa629c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6240adf60 0x7fa6241f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6240adf60 0x7fa6241f5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41800/0 (socket says 172.21.15.47:41800) 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 -- 172.21.15.47:0/680783018 learned_addr learned my addr 172.21.15.47:0/680783018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629200640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240b2f70 0x7fa6240b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa62a600640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62415faf0 0x7fa62414c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 -- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240b2f70 msgr2=0x7fa6240b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240b2f70 0x7fa6240b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 -- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62415faf0 msgr2=0x7fa62414c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62415faf0 0x7fa62414c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 -- 172.21.15.47:0/680783018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6241f9350 con 0x7fa6240adf60 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa62a600640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62415faf0 0x7fa62414c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.286+0000 7fa629c00640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6240adf60 0x7fa6241f5870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa614000c00 tx=0x7fa6140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.287+0000 7fa621600640 1 -- 172.21.15.47:0/680783018 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614032610 con 0x7fa6240adf60 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.287+0000 7fa621600640 1 -- 172.21.15.47:0/680783018 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa6140327d0 con 0x7fa6240adf60 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.287+0000 7fa621600640 1 -- 172.21.15.47:0/680783018 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa614032bc0 con 0x7fa6240adf60 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.287+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6241f5db0 con 0x7fa6240adf60 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.287+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6241f8ab0 con 0x7fa6240adf60 2024-04-20T10:27:40.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.288+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa624068ce0 con 0x7fa6240adf60 2024-04-20T10:27:40.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.289+0000 7fa621600640 1 -- 172.21.15.47:0/680783018 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa61404c060 con 0x7fa6240adf60 2024-04-20T10:27:40.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.290+0000 7fa621600640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa60806c8f0 0x7fa60806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:40.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.290+0000 7fa621600640 1 -- 172.21.15.47:0/680783018 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa6140ad530 con 0x7fa6240adf60 2024-04-20T10:27:40.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.291+0000 7fa629200640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa60806c8f0 0x7fa60806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:40.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.292+0000 7fa629200640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa60806c8f0 0x7fa60806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6240d0af0 tx=0x7fa618008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:40.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.295+0000 7fa621600640 1 -- 172.21.15.47:0/680783018 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa624068ce0 con 0x7fa6240adf60 2024-04-20T10:27:40.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.499+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa62406e6b0 con 0x7fa6240adf60 2024-04-20T10:27:40.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.501+0000 7fa621600640 1 -- 172.21.15.47:0/680783018 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7fa614040070 con 0x7fa6240adf60 2024-04-20T10:27:40.501 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:40.502 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:40.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa60806c8f0 msgr2=0x7fa60806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:40.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa60806c8f0 0x7fa60806edb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa6240d0af0 tx=0x7fa618008040 comp rx=0 tx=0).stop 2024-04-20T10:27:40.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6240adf60 msgr2=0x7fa6241f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:40.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6240adf60 0x7fa6241f5870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa614000c00 tx=0x7fa6140046c0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 shutdown_connections 2024-04-20T10:27:40.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa62415faf0 0x7fa62414c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6240b2f70 0x7fa6240b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa60806c8f0 0x7fa60806edb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 --2- 172.21.15.47:0/680783018 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa6240adf60 0x7fa6241f5870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:40.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 >> 172.21.15.47:0/680783018 conn(0x7fa6240d0040 msgr2=0x7fa6240cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:40.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 shutdown_connections 2024-04-20T10:27:40.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:40.504+0000 7fa62b000640 1 -- 172.21.15.47:0/680783018 wait complete. 2024-04-20T10:27:40.925 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:27:32.532668+0000', 'modified': '2024-04-20T10:27:37.571421+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/2667922035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6835', 'nonce': 2667922035}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2667922035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-20T10:27:40.925 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:27:40.925 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T10:27:41.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.148+0000 7f35f7200640 1 Processor -- start 2024-04-20T10:27:41.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.148+0000 7f35f7200640 1 -- start start 2024-04-20T10:27:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.150+0000 7f35f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.150+0000 7f35f7200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f35f815a1f0 0x7f35f8071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.150+0000 7f35f7200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35f815cc90 con 0x7f35f806da90 2024-04-20T10:27:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.150+0000 7f35f7200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f35f8072900 con 0x7f35f815a1f0 2024-04-20T10:27:41.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.150+0000 7f35f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f35f815ca40 con 0x7f35f8159e10 2024-04-20T10:27:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.150+0000 7f35f5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.151+0000 7f35f5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f8071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:56296/0 (socket says 172.21.15.47:56296) 2024-04-20T10:27:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.151+0000 7f35f5e00640 1 -- 172.21.15.47:0/1213385125 learned_addr learned my addr 172.21.15.47:0/1213385125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.151+0000 7f35ed400640 1 --2- 172.21.15.47:0/1213385125 >> v2:172.21.15.175:3300/0 conn(0x7f35f815a1f0 0x7f35f8071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.151+0000 7f35f5e00640 1 -- 172.21.15.47:0/1213385125 >> v2:172.21.15.175:3300/0 conn(0x7f35f815a1f0 msgr2=0x7f35f8071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.151+0000 7f35f5e00640 1 --2- 172.21.15.47:0/1213385125 >> v2:172.21.15.175:3300/0 conn(0x7f35f815a1f0 0x7f35f8071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.151+0000 7f35f5e00640 1 -- 172.21.15.47:0/1213385125 >> v1:172.21.15.175:6789/0 conn(0x7f35f806da90 legacy=0x7f35f8160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.151+0000 7f35f5e00640 1 -- 172.21.15.47:0/1213385125 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f815cf70 con 0x7f35f8159e10 2024-04-20T10:27:41.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.153+0000 7f35f5e00640 1 --2- 172.21.15.47:0/1213385125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f8071a80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35f815cbc0 tx=0x7f35e802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=699a6c808c2d63c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.153+0000 7f35f5400640 1 -- 172.21.15.47:0/1213385125 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e8035020 con 0x7f35f8159e10 2024-04-20T10:27:41.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.153+0000 7f35f5400640 1 -- 172.21.15.47:0/1213385125 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35e8031640 con 0x7f35f8159e10 2024-04-20T10:27:41.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.153+0000 7f35f5400640 1 -- 172.21.15.47:0/1213385125 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35e802d040 con 0x7f35f8159e10 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 -- 172.21.15.47:0/1213385125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 msgr2=0x7f35f8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 --2- 172.21.15.47:0/1213385125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f8071a80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f35f815cbc0 tx=0x7f35e802b8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 -- 172.21.15.47:0/1213385125 shutdown_connections 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 --2- 172.21.15.47:0/1213385125 >> v2:172.21.15.175:3300/0 conn(0x7f35f815a1f0 0x7f35f8071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 --2- 172.21.15.47:0/1213385125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f8071a80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 -- 172.21.15.47:0/1213385125 >> 172.21.15.47:0/1213385125 conn(0x7f35f80d0040 msgr2=0x7f35f80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 -- 172.21.15.47:0/1213385125 shutdown_connections 2024-04-20T10:27:41.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.154+0000 7f35f7200640 1 -- 172.21.15.47:0/1213385125 wait complete. 2024-04-20T10:27:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.156+0000 7f35f7200640 1 Processor -- start 2024-04-20T10:27:41.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.157+0000 7f35f7200640 1 -- start start 2024-04-20T10:27:41.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f806da90 0x7f35f80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f809d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f815a1f0 0x7f35f81565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f35f815d810 con 0x7f35f815a1f0 2024-04-20T10:27:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35f815d530 con 0x7f35f806da90 2024-04-20T10:27:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f35f806b900 con 0x7f35f8159e10 2024-04-20T10:27:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f815a1f0 0x7f35f81565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f815a1f0 0x7f35f81565c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41814/0 (socket says 172.21.15.47:41814) 2024-04-20T10:27:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f6800640 1 -- 172.21.15.47:0/1693621002 learned_addr learned my addr 172.21.15.47:0/1693621002 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35f5e00640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f806da90 0x7f35f80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.158+0000 7f35ed400640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f809d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f6800640 1 -- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 msgr2=0x7f35f809d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f6800640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f809d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f6800640 1 -- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f806da90 msgr2=0x7f35f80cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f6800640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f806da90 0x7f35f80cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f6800640 1 -- 172.21.15.47:0/1693621002 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35f80cb800 con 0x7f35f815a1f0 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f5e00640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f806da90 0x7f35f80cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f6800640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f815a1f0 0x7f35f81565c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c7b0 tx=0x7f35f000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35eea00640 1 -- 172.21.15.47:0/1693621002 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0004b60 con 0x7f35f815a1f0 2024-04-20T10:27:41.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35eea00640 1 -- 172.21.15.47:0/1693621002 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0004d00 con 0x7f35f815a1f0 2024-04-20T10:27:41.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35eea00640 1 -- 172.21.15.47:0/1693621002 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f000aa90 con 0x7f35f815a1f0 2024-04-20T10:27:41.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.159+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35f815a700 con 0x7f35f815a1f0 2024-04-20T10:27:41.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.160+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35f80cb520 con 0x7f35f815a1f0 2024-04-20T10:27:41.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.162+0000 7f35eea00640 1 -- 172.21.15.47:0/1693621002 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35f000ac30 con 0x7f35f815a1f0 2024-04-20T10:27:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.163+0000 7f35eea00640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35d406c8f0 0x7f35d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.163+0000 7f35eea00640 1 -- 172.21.15.47:0/1693621002 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f35f008c0f0 con 0x7f35f815a1f0 2024-04-20T10:27:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.163+0000 7f35ffba3740 1 Processor -- start 2024-04-20T10:27:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.163+0000 7f35f5e00640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35d406c8f0 0x7f35d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.163+0000 7f35ffba3740 1 -- start start 2024-04-20T10:27:41.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.164+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55837296c670 0x558372923ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.164+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 0x5583729241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.164+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837291eb90 0x55837294e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.164+0000 7f35ffba3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55837288a9d0 con 0x55837291eb90 2024-04-20T10:27:41.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.164+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x558372831270 con 0x55837296c670 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.164+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x558372886990 con 0x5583729226f0 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.164+0000 7f35f5e00640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35d406c8f0 0x7f35d406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35e8036000 tx=0x7f35e803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837291eb90 0x55837294e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837291eb90 0x55837294e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41824/0 (socket says 172.21.15.47:41824) 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 -- 172.21.15.47:0/490491556 learned_addr learned my addr 172.21.15.47:0/490491556 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35ed400640 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 0x5583729241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f5e00640 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55837296c670 0x558372923ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 -- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 msgr2=0x5583729241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 0x5583729241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 -- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55837296c670 msgr2=0x558372923ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55837296c670 0x558372923ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 -- 172.21.15.47:0/490491556 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837259d2d0 con 0x55837291eb90 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35f6800640 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837291eb90 0x55837294e980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35f008a310 tx=0x7f35f005a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8d6882295b0c271 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35e6a00640 1 -- 172.21.15.47:0/490491556 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f005df20 con 0x55837291eb90 2024-04-20T10:27:41.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35e6a00640 1 -- 172.21.15.47:0/490491556 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0062750 con 0x55837291eb90 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.165+0000 7f35e6a00640 1 -- 172.21.15.47:0/490491556 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0072100 con 0x55837291eb90 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 -- 172.21.15.47:0/490491556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837291eb90 msgr2=0x55837294e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837291eb90 0x55837294e980 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f35f008a310 tx=0x7f35f005a270 comp rx=0 tx=0).stop 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 -- 172.21.15.47:0/490491556 shutdown_connections 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837291eb90 0x55837294e980 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 0x5583729241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 --2- 172.21.15.47:0/490491556 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55837296c670 0x558372923ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 -- 172.21.15.47:0/490491556 >> 172.21.15.47:0/490491556 conn(0x558372920e00 msgr2=0x558372969980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 -- 172.21.15.47:0/490491556 shutdown_connections 2024-04-20T10:27:41.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.166+0000 7f35ffba3740 1 -- 172.21.15.47:0/490491556 wait complete. 2024-04-20T10:27:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.167+0000 7f35ffba3740 1 -- start start 2024-04-20T10:27:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.167+0000 7f35ffba3740 1 Processor -- start 2024-04-20T10:27:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.168+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x558372946720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.168+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55837296c670 0x5583729717e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.168+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583729472a0 0x55837297a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.168+0000 7f35ffba3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55837288a9d0 con 0x5583729472a0 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.168+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5583728445e0 con 0x5583729226f0 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.168+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x558372922c00 con 0x55837296c670 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.168+0000 7f35ffba3740 10 client.0 _ll_register_callbacks cb 0x55837259cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583729472a0 0x55837297a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583729472a0 0x55837297a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41834/0 (socket says 172.21.15.47:41834) 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 -- 172.21.15.47:0/2968797358 learned_addr learned my addr 172.21.15.47:0/2968797358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35ed400640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55837296c670 0x5583729717e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f5e00640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x558372946720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 -- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55837296c670 msgr2=0x5583729717e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55837296c670 0x5583729717e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 -- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 msgr2=0x558372946720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x558372946720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 -- 172.21.15.47:0/2968797358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583726676b0 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f5e00640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x558372946720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35f6800640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583729472a0 0x55837297a4d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35f0059d30 tx=0x7f35f0007f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697d45259899e1ca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35db400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.169+0000 7f35db400640 1 -- 172.21.15.47:0/2968797358 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f007e060 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35db400640 1 -- 172.21.15.47:0/2968797358 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f35f0057c10 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f0062750 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35db400640 1 -- 172.21.15.47:0/2968797358 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f35f0072100 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 10 client.4386 fetch_fsmap learned FSMap version 7 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5583729712c0 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35db400640 1 -- 172.21.15.47:0/2968797358 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0062750 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35db400640 1 -- 172.21.15.47:0/2968797358 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 7) ==== 3503+0+0 (secure 0 0 0) 0x7f35f00561e0 con 0x5583729472a0 2024-04-20T10:27:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 10 client.4386 fetch_fsmap finished waiting for FSMap version 7 2024-04-20T10:27:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 10 client.4386 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:27:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 20 client.4386 populate_metadata read hostname 'smithi047' 2024-04-20T10:27:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837296e1c0 0x55837296e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 4 client.4386 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:27:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.170+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 --> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583728983f0 con 0x55837296e1c0 2024-04-20T10:27:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.171+0000 7f35f5e00640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837296e1c0 0x55837296e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.172+0000 7f35f5e00640 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837296e1c0 0x55837296e5a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a224b62ea7d92873 server_cookie=96c70e7bf1f60a0c in_seq=0 out_seq=0 2024-04-20T10:27:41.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.172+0000 7f35db400640 10 client.4386 ms_handle_connect on v2:172.21.15.175:6835/2667922035 2024-04-20T10:27:41.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.174+0000 7f35db400640 1 -- 172.21.15.47:0/2968797358 <== mds.0 v2:172.21.15.175:6835/2667922035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5583728983f0 con 0x55837296e1c0 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.174+0000 7f35db400640 10 client.4386 handle_command_reply: tid=0 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.174+0000 7f35ffba3740 1 client.4386 shutdown 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583729472a0 msgr2=0x55837297a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583729472a0 0x55837297a4d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f35f0059d30 tx=0x7f35f0007f50 comp rx=0 tx=0).stop 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 shutdown_connections 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5583729472a0 0x55837297a4d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55837296c670 0x5583729717e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837296e1c0 0x55837296e5a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 --2- 172.21.15.47:0/2968797358 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x558372946720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 >> 172.21.15.47:0/2968797358 conn(0x55837291eb90 msgr2=0x558372969780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35db400640 0 client.4386 ms_handle_reset on v2:172.21.15.175:6835/2667922035 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 shutdown_connections 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 1 -- 172.21.15.47:0/2968797358 wait complete. 2024-04-20T10:27:41.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.175+0000 7f35ffba3740 20 client.4386 trim_cache size 0 max 16384 2024-04-20T10:27:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.195+0000 7f35ffba3740 1 Processor -- start 2024-04-20T10:27:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.195+0000 7f35ffba3740 1 -- start start 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x55837296ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729472a0 0x5583729524d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372952a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35ffba3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f35f82010a0 con 0x55837296c670 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35f815c2e0 con 0x5583729226f0 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f35f8200a80 con 0x5583729472a0 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372952a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372952a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41840/0 (socket says 172.21.15.47:41840) 2024-04-20T10:27:41.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 -- 172.21.15.47:0/335199735 learned_addr learned my addr 172.21.15.47:0/335199735 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f5e00640 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x55837296ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 -- 172.21.15.47:0/335199735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729472a0 msgr2=0x5583729524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729472a0 0x5583729524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 -- 172.21.15.47:0/335199735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 msgr2=0x55837296ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x55837296ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.196+0000 7f35f6800640 1 -- 172.21.15.47:0/335199735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558372970460 con 0x55837296c670 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.197+0000 7f35f6800640 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372952a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35f000a430 tx=0x7f35f0073000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eccb2486e66d690d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.197+0000 7f35e6000640 1 -- 172.21.15.47:0/335199735 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f0062750 con 0x55837296c670 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.197+0000 7f35e6000640 1 -- 172.21.15.47:0/335199735 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0057c10 con 0x55837296c670 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.197+0000 7f35e6000640 1 -- 172.21.15.47:0/335199735 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f00561e0 con 0x55837296c670 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 -- 172.21.15.47:0/335199735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 msgr2=0x558372952a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372952a10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f35f000a430 tx=0x7f35f0073000 comp rx=0 tx=0).stop 2024-04-20T10:27:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 -- 172.21.15.47:0/335199735 shutdown_connections 2024-04-20T10:27:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372952a10 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729472a0 0x5583729524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 --2- 172.21.15.47:0/335199735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729226f0 0x55837296ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 -- 172.21.15.47:0/335199735 >> 172.21.15.47:0/335199735 conn(0x55837291eb90 msgr2=0x558372946530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 -- 172.21.15.47:0/335199735 shutdown_connections 2024-04-20T10:27:41.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 -- 172.21.15.47:0/335199735 wait complete. 2024-04-20T10:27:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.198+0000 7f35ffba3740 1 -- start start 2024-04-20T10:27:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.199+0000 7f35ffba3740 1 Processor -- start 2024-04-20T10:27:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 0x558372944dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729472a0 0x558372945310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35ffba3740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372949c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35ffba3740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55837288a9d0 con 0x55837296c670 2024-04-20T10:27:41.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5583728445e0 con 0x5583729472a0 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35ffba3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x558372971df0 con 0x5583729226f0 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35ffba3740 10 client.0 _ll_register_callbacks cb 0x7f35f0072050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372949c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372949c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41842/0 (socket says 172.21.15.47:41842) 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 -- 172.21.15.47:0/543644321 learned_addr learned my addr 172.21.15.47:0/543644321 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 -- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 msgr2=0x558372944dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 0x558372944dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 -- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729472a0 msgr2=0x558372945310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729472a0 0x558372945310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.200+0000 7f35f6800640 1 -- 172.21.15.47:0/543644321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583729e4e20 con 0x55837296c670 2024-04-20T10:27:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35f6800640 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372949c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c780 tx=0x7f35f0055f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f85103693ce2d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35db400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:27:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35db400640 1 -- 172.21.15.47:0/543644321 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35f006a070 con 0x55837296c670 2024-04-20T10:27:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35db400640 1 -- 172.21.15.47:0/543644321 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f35f00624d0 con 0x55837296c670 2024-04-20T10:27:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35f0057c10 con 0x55837296c670 2024-04-20T10:27:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35db400640 1 -- 172.21.15.47:0/543644321 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f35f0062750 con 0x55837296c670 2024-04-20T10:27:41.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 10 client.4392 fetch_fsmap learned FSMap version 7 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x5583729e4e20 con 0x55837296c670 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35db400640 1 -- 172.21.15.47:0/543644321 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35f0057c10 con 0x55837296c670 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35db400640 1 -- 172.21.15.47:0/543644321 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 7) ==== 3503+0+0 (secure 0 0 0) 0x7f35f0006b50 con 0x55837296c670 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 10 client.4392 fetch_fsmap finished waiting for FSMap version 7 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 10 client.4392 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 20 client.4392 populate_metadata read hostname 'smithi047' 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837294a340 0x55837294a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 4 client.4392 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.201+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 --> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] -- command(tid 0: {"prefix": "status"}) -- 0x558372892260 con 0x55837294a340 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.202+0000 7f35f5e00640 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837294a340 0x55837294a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35f5e00640 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837294a340 0x55837294a720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42612e5aead2cbf server_cookie=34e72e21979f29 in_seq=0 out_seq=0 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35db400640 10 client.4392 ms_handle_connect on v2:172.21.15.175:6835/2667922035 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35db400640 1 -- 172.21.15.47:0/543644321 <== mds.0 v2:172.21.15.175:6835/2667922035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x558372892260 con 0x55837294a340 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35db400640 10 client.4392 handle_command_reply: tid=0 2024-04-20T10:27:41.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35ffba3740 1 client.4392 shutdown 2024-04-20T10:27:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 msgr2=0x558372949c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35ffba3740 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372949c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c780 tx=0x7f35f0055f40 comp rx=0 tx=0).stop 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.203+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 shutdown_connections 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55837296c670 0x558372949c80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55837294a340 0x55837294a720 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5583729472a0 0x558372945310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 1 --2- 172.21.15.47:0/543644321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5583729226f0 0x558372944dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35db400640 0 client.4392 ms_handle_reset on v2:172.21.15.175:6835/2667922035 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 >> 172.21.15.47:0/543644321 conn(0x55837291eb90 msgr2=0x55837296ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 shutdown_connections 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 1 -- 172.21.15.47:0/543644321 wait complete. 2024-04-20T10:27:41.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35ffba3740 20 client.4392 trim_cache size 0 max 16384 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "775946e6-93c9-4843-aaf4-db758801eec1", 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4285, 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 8.1578382030000007, 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 7, 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 27, 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 27, 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 13.982874915 2024-04-20T10:27:41.207 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:27:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35d406c8f0 msgr2=0x7f35d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35f7200640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35d406c8f0 0x7f35d406edb0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f35e8036000 tx=0x7f35e803c040 comp rx=0 tx=0).stop 2024-04-20T10:27:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f815a1f0 msgr2=0x7f35f81565c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.204+0000 7f35f7200640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f815a1f0 0x7f35f81565c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f35f000c7b0 tx=0x7f35f000cc70 comp rx=0 tx=0).stop 2024-04-20T10:27:41.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 shutdown_connections 2024-04-20T10:27:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35d406c8f0 0x7f35d406edb0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35f815a1f0 0x7f35f81565c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35f8159e10 0x7f35f809d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 --2- 172.21.15.47:0/1693621002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35f806da90 0x7f35f80cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 >> 172.21.15.47:0/1693621002 conn(0x7f35f80d0040 msgr2=0x7f35f80afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 shutdown_connections 2024-04-20T10:27:41.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.205+0000 7f35f7200640 1 -- 172.21.15.47:0/1693621002 wait complete. 2024-04-20T10:27:41.221 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:27:41.221 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:41.221 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:27:41.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.412+0000 7fd5b1e00640 1 Processor -- start 2024-04-20T10:27:41.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.413+0000 7fd5b1e00640 1 -- start start 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.414+0000 7fd5b1e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5ac15c360 0x7fd5ac166250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.414+0000 7fd5b1e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ac06ccc0 con 0x7fd5ac15b400 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.414+0000 7fd5b1e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5ac06cfa0 con 0x7fd5ac15b7e0 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.414+0000 7fd5b1e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd5ac06d280 con 0x7fd5ac15c360 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.414+0000 7fd5b0a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5ac15b400 0x7fd5ac072d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57056/0 (socket says 172.21.15.47:57056) 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.414+0000 7fd5b0a00640 1 -- 172.21.15.47:0/726290601 learned_addr learned my addr 172.21.15.47:0/726290601 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.414+0000 7fd5b1400640 1 --2- 172.21.15.47:0/726290601 >> v2:172.21.15.47:3300/0 conn(0x7fd5ac15c360 0x7fd5ac166250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.415+0000 7fd5b1400640 1 -- 172.21.15.47:0/726290601 >> v1:172.21.15.175:6789/0 conn(0x7fd5ac15b7e0 legacy=0x7fd5ac15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.415+0000 7fd5b1400640 1 -- 172.21.15.47:0/726290601 >> v1:172.21.15.47:6789/0 conn(0x7fd5ac15b400 legacy=0x7fd5ac072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.415+0000 7fd5abe00640 1 -- 172.21.15.47:0/726290601 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2618706900 0 0) 0x7fd5ac06ccc0 con 0x7fd5ac15b400 2024-04-20T10:27:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.415+0000 7fd5b1400640 1 -- 172.21.15.47:0/726290601 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac06e440 con 0x7fd5ac15c360 2024-04-20T10:27:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.416+0000 7fd5b1400640 1 --2- 172.21.15.47:0/726290601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15c360 0x7fd5ac166250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097a0 tx=0x7fd5a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe90a7181b55add7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.416+0000 7fd5abe00640 1 -- 172.21.15.47:0/726290601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4035020 con 0x7fd5ac15c360 2024-04-20T10:27:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.416+0000 7fd5abe00640 1 -- 172.21.15.47:0/726290601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5a4031830 con 0x7fd5ac15c360 2024-04-20T10:27:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.416+0000 7fd5abe00640 1 -- 172.21.15.47:0/726290601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a4031c00 con 0x7fd5ac15c360 2024-04-20T10:27:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.417+0000 7fd5b1e00640 1 -- 172.21.15.47:0/726290601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15c360 msgr2=0x7fd5ac166250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.417+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/726290601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15c360 0x7fd5ac166250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd5a40097a0 tx=0x7fd5a402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.417+0000 7fd5b1e00640 1 -- 172.21.15.47:0/726290601 shutdown_connections 2024-04-20T10:27:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.417+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/726290601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15c360 0x7fd5ac166250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.417+0000 7fd5b1e00640 1 -- 172.21.15.47:0/726290601 >> 172.21.15.47:0/726290601 conn(0x7fd5ac0d0040 msgr2=0x7fd5ac0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.417+0000 7fd5b1e00640 1 -- 172.21.15.47:0/726290601 shutdown_connections 2024-04-20T10:27:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.417+0000 7fd5b1e00640 1 -- 172.21.15.47:0/726290601 wait complete. 2024-04-20T10:27:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.418+0000 7fd5b1e00640 1 Processor -- start 2024-04-20T10:27:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.419+0000 7fd5b1e00640 1 -- start start 2024-04-20T10:27:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.419+0000 7fd5b1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15b400 0x7fd5ac152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.419+0000 7fd5b1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac15b7e0 0x7fd5ac0cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.419+0000 7fd5b1e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac15c360 0x7fd5ac159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.419+0000 7fd5b1e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac06fc50 con 0x7fd5ac15b400 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.419+0000 7fd5b1e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5ac15c7c0 con 0x7fd5ac15b7e0 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.419+0000 7fd5b1e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5ac06fdd0 con 0x7fd5ac15c360 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15b400 0x7fd5ac152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15b400 0x7fd5ac152da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41862/0 (socket says 172.21.15.47:41862) 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 -- 172.21.15.47:0/138767566 learned_addr learned my addr 172.21.15.47:0/138767566 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b1400640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac15c360 0x7fd5ac159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5a3e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac15b7e0 0x7fd5ac0cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 -- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac15c360 msgr2=0x7fd5ac159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac15c360 0x7fd5ac159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 -- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac15b7e0 msgr2=0x7fd5ac0cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac15b7e0 0x7fd5ac0cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 -- 172.21.15.47:0/138767566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5ac1532e0 con 0x7fd5ac15b400 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5a3e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac15b7e0 0x7fd5ac0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:41.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.420+0000 7fd5b0a00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15b400 0x7fd5ac152da0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c8c0 tx=0x7fd59c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.421+0000 7fd5a9600640 1 -- 172.21.15.47:0/138767566 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c018070 con 0x7fd5ac15b400 2024-04-20T10:27:41.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.421+0000 7fd5a9600640 1 -- 172.21.15.47:0/138767566 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd59c00a640 con 0x7fd5ac15b400 2024-04-20T10:27:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.421+0000 7fd5a9600640 1 -- 172.21.15.47:0/138767566 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd59c00aa30 con 0x7fd5ac15b400 2024-04-20T10:27:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.421+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5ac153980 con 0x7fd5ac15b400 2024-04-20T10:27:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.421+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5ac153f40 con 0x7fd5ac15b400 2024-04-20T10:27:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.422+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5ac075b60 con 0x7fd5ac15b400 2024-04-20T10:27:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.423+0000 7fd5a9600640 1 -- 172.21.15.47:0/138767566 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd59c00abd0 con 0x7fd5ac15b400 2024-04-20T10:27:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.424+0000 7fd5a9600640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd58806c8f0 0x7fd58806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.424+0000 7fd5a9600640 1 -- 172.21.15.47:0/138767566 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd59c08b5d0 con 0x7fd5ac15b400 2024-04-20T10:27:41.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.424+0000 7fd5a3e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd58806c8f0 0x7fd58806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:41.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.426+0000 7fd5a3e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd58806c8f0 0x7fd58806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac1f7b80 tx=0x7fd594009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:41.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.429+0000 7fd5a9600640 1 -- 172.21.15.47:0/138767566 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd59c0596b0 con 0x7fd5ac15b400 2024-04-20T10:27:41.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.672+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd5ac0b5220 con 0x7fd5ac15b400 2024-04-20T10:27:41.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.673+0000 7fd5a9600640 1 -- 172.21.15.47:0/138767566 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7fd59c058fb0 con 0x7fd5ac15b400 2024-04-20T10:27:41.673 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd58806c8f0 msgr2=0x7fd58806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd58806c8f0 0x7fd58806edb0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac1f7b80 tx=0x7fd594009040 comp rx=0 tx=0).stop 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15b400 msgr2=0x7fd5ac152da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15b400 0x7fd5ac152da0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00c8c0 tx=0x7fd59c00cd80 comp rx=0 tx=0).stop 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 shutdown_connections 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5ac15c360 0x7fd5ac159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5ac15b7e0 0x7fd5ac0cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd58806c8f0 0x7fd58806edb0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 --2- 172.21.15.47:0/138767566 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5ac15b400 0x7fd5ac152da0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 >> 172.21.15.47:0/138767566 conn(0x7fd5ac0d0040 msgr2=0x7fd5ac157850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.675+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 shutdown_connections 2024-04-20T10:27:41.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41.676+0000 7fd5b1e00640 1 -- 172.21.15.47:0/138767566 wait complete. 2024-04-20T10:27:41.695 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:27:41.696 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-20T10:27:41.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:41.696 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-20T10:27:41.748 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:27:41.748 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:27:41.748 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:27:41.748 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:27:41.748 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:27:41.748 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:27:41.748 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42821sec preferred_lft 42821sec 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-20T10:27:41.749 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-20T10:27:41.751 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:27:41.751 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:27:41.751 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:27:41.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:27:41.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-04-20T10:27:41.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-04-20T10:27:41.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:27:41.751 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:27:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:41 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:27:42.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:42 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:27:42.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:42.263 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:27:42.329 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-20T10:27:42.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:42.333 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-20T10:27:42.394 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-20T10:27:42.394 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:27:42.395 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:27:42.395 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:27:42.395 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:27:42.395 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:27:42.395 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:27:42.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:27:42.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:27:42.395 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:27:42.395 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:27:42.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:42 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:27:43.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:27:43.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:43.182 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:27:43.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:43.229 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-20T10:27:43.316 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:27:43.316 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:27:43.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:27:43.316 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-20T10:27:43.316 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:27:43.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:27:43.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:27:43.536 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-20T10:27:43.536 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:27:43.536 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:27:43.536 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-20T10:27:43.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:27:43.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-20T10:27:43.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-20T10:27:43.537 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-20T10:27:43.537 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:27:43.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:27:43.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:27:43.814 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:27:43.814 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:27:43.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-20T10:27:43.814 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:27:43.814 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:27:43.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:27:43.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:27:44.007 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-20T10:27:44.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:27:44.007 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:44.046 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-20T10:27:44.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:27:44.046 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:44.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:27:44.204 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:27:44.249 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:27:44.250 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:27:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:27:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:27:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:27:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:27:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:27:44.251 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:27:44.252 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:44.252 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:27:44.340 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:27:44.340 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-20T10:27:44.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:27:44.409 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:27:44.453 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:27:44.454 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:27:44.455 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:27:44.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:44.456 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:27:44.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:2024-04-20T10:27:44.523+0000 7f5ba7f72480 -1 init, newargv = 0x55628a287270 newargc=15 2024-04-20T10:27:44.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[27722]: starting ceph client 2024-04-20T10:27:44.539 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[27722]: starting fuse 2024-04-20T10:27:44.540 INFO:teuthology.orchestra.run.smithi047.stdout:42 2024-04-20T10:27:44.541 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:27:44.541 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.admin")) 2024-04-20T10:27:44.542 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:27:44.739 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.admin.27722.asok 2024-04-20T10:27:44.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:27:44.756 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.27722.asok 2024-04-20T10:27:44.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:44.756 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.27722.asok status 2024-04-20T10:27:44.991 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:27:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-04-20T10:27:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:27:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:27:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "admin", 2024-04-20T10:27:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-04-20T10:27:44.991 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "27722", 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4410, 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4410 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:27:44.992 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2240276344 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2240276344 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:27:44.993 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4410 192.168.0.1:0/2240276344", 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2240276344", 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 7, 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 27, 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-04-20T10:27:44.994 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:27:45.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:45.005 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-04-20T10:27:45.005 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:45.057 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-04-20T10:27:45.057 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:45.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:45.057 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:45.139 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-20T10:27:45.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-20T10:27:45.291 INFO:teuthology.orchestra.run.smithi047.stdout:# file: . 2024-04-20T10:27:45.291 INFO:teuthology.orchestra.run.smithi047.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-20T10:27:45.291 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:45.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:27:45.303 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:45.303 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:27:45.357 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:45.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.562+0000 7fb3dca00640 1 Processor -- start 2024-04-20T10:27:45.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.562+0000 7fb3dca00640 1 -- start start 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.564+0000 7fb3dca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3d815a1f0 0x7fb3d8072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.564+0000 7fb3dca00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d815cc90 con 0x7fb3d8159e10 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.564+0000 7fb3dca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb3d815cf70 con 0x7fb3d8072190 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.564+0000 7fb3dca00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb3d8072900 con 0x7fb3d815a1f0 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.564+0000 7fb3cfe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3d815a1f0 0x7fb3d8072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.564+0000 7fb3cfe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb3d815a1f0 0x7fb3d8072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47034/0 (socket says 172.21.15.47:47034) 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.564+0000 7fb3cfe00640 1 -- 172.21.15.47:0/4056705080 learned_addr learned my addr 172.21.15.47:0/4056705080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.565+0000 7fb3cfe00640 1 -- 172.21.15.47:0/4056705080 >> v1:172.21.15.175:6789/0 conn(0x7fb3d8072190 legacy=0x7fb3d8073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:45.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.565+0000 7fb3cfe00640 1 -- 172.21.15.47:0/4056705080 >> v1:172.21.15.47:6789/0 conn(0x7fb3d8159e10 legacy=0x7fb3d8071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.565+0000 7fb3cfe00640 1 -- 172.21.15.47:0/4056705080 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d815d250 con 0x7fb3d815a1f0 2024-04-20T10:27:45.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.566+0000 7fb3cfe00640 1 --2- 172.21.15.47:0/4056705080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d8072ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009870 tx=0x7fb3c002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f69aa93e6d5cf20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:45.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.566+0000 7fb3d5e00640 1 -- 172.21.15.47:0/4056705080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0034070 con 0x7fb3d815a1f0 2024-04-20T10:27:45.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.566+0000 7fb3d5e00640 1 -- 172.21.15.47:0/4056705080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3c0030850 con 0x7fb3d815a1f0 2024-04-20T10:27:45.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.566+0000 7fb3d5e00640 1 -- 172.21.15.47:0/4056705080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0030c20 con 0x7fb3d815a1f0 2024-04-20T10:27:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.567+0000 7fb3dca00640 1 -- 172.21.15.47:0/4056705080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 msgr2=0x7fb3d8072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.567+0000 7fb3dca00640 1 --2- 172.21.15.47:0/4056705080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d8072ee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb3c0009870 tx=0x7fb3c002b990 comp rx=0 tx=0).stop 2024-04-20T10:27:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.567+0000 7fb3dca00640 1 -- 172.21.15.47:0/4056705080 shutdown_connections 2024-04-20T10:27:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.567+0000 7fb3dca00640 1 --2- 172.21.15.47:0/4056705080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d8072ee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.567+0000 7fb3dca00640 1 -- 172.21.15.47:0/4056705080 >> 172.21.15.47:0/4056705080 conn(0x7fb3d80d0040 msgr2=0x7fb3d80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.567+0000 7fb3dca00640 1 -- 172.21.15.47:0/4056705080 shutdown_connections 2024-04-20T10:27:45.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.567+0000 7fb3dca00640 1 -- 172.21.15.47:0/4056705080 wait complete. 2024-04-20T10:27:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.569+0000 7fb3dca00640 1 Processor -- start 2024-04-20T10:27:45.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.569+0000 7fb3dca00640 1 -- start start 2024-04-20T10:27:45.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3dca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d8072190 0x7fb3d815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3dca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d8159e10 0x7fb3d81f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3dca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d81f15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3dca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d815d810 con 0x7fb3d815a1f0 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3dca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d815ca40 con 0x7fb3d8159e10 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3dca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb3d8073b30 con 0x7fb3d8072190 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3d7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d81f15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3d7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d81f15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47040/0 (socket says 172.21.15.47:47040) 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.570+0000 7fb3d7200640 1 -- 172.21.15.47:0/1480793508 learned_addr learned my addr 172.21.15.47:0/1480793508 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3cfe00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d8159e10 0x7fb3d81f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d7200640 1 -- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d8072190 msgr2=0x7fb3d815c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d6800640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d8072190 0x7fb3d815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d7200640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d8072190 0x7fb3d815c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d7200640 1 -- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d8159e10 msgr2=0x7fb3d81f4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d7200640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d8159e10 0x7fb3d81f4190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d7200640 1 -- 172.21.15.47:0/1480793508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d81f4900 con 0x7fb3d815a1f0 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3cfe00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d8159e10 0x7fb3d81f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d6800640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d8072190 0x7fb3d815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3d7200640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d81f15c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c730 tx=0x7fb3d000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3cea00640 1 -- 172.21.15.47:0/1480793508 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0016020 con 0x7fb3d815a1f0 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3cea00640 1 -- 172.21.15.47:0/1480793508 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb3d000a9f0 con 0x7fb3d815a1f0 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d81f4be0 con 0x7fb3d815a1f0 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.571+0000 7fb3cea00640 1 -- 172.21.15.47:0/1480793508 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000ade0 con 0x7fb3d815a1f0 2024-04-20T10:27:45.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.572+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81f1b00 con 0x7fb3d815a1f0 2024-04-20T10:27:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.573+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d8075b60 con 0x7fb3d815a1f0 2024-04-20T10:27:45.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.574+0000 7fb3cea00640 1 -- 172.21.15.47:0/1480793508 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb3d0019020 con 0x7fb3d815a1f0 2024-04-20T10:27:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.575+0000 7fb3cea00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3b406c8f0 0x7fb3b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.575+0000 7fb3cea00640 1 -- 172.21.15.47:0/1480793508 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb3d008bf10 con 0x7fb3d815a1f0 2024-04-20T10:27:45.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.575+0000 7fb3d6800640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3b406c8f0 0x7fb3b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:45.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.576+0000 7fb3d6800640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3b406c8f0 0x7fb3b406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8001680 tx=0x7fb3c8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:45.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.580+0000 7fb3cea00640 1 -- 172.21.15.47:0/1480793508 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb3d0020030 con 0x7fb3d815a1f0 2024-04-20T10:27:45.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.784+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb3d81f5100 con 0x7fb3d815a1f0 2024-04-20T10:27:45.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.786+0000 7fb3cea00640 1 -- 172.21.15.47:0/1480793508 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7fb3d00182c0 con 0x7fb3d815a1f0 2024-04-20T10:27:45.787 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:45.787 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3b406c8f0 msgr2=0x7fb3b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3b406c8f0 0x7fb3b406edb0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8001680 tx=0x7fb3c8009040 comp rx=0 tx=0).stop 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 msgr2=0x7fb3d81f15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d81f15c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c730 tx=0x7fb3d000cbf0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 shutdown_connections 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb3b406c8f0 0x7fb3b406edb0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb3d815a1f0 0x7fb3d81f15c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb3d8159e10 0x7fb3d81f4190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 --2- 172.21.15.47:0/1480793508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3d8072190 0x7fb3d815c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 >> 172.21.15.47:0/1480793508 conn(0x7fb3d80d0040 msgr2=0x7fb3d80b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 shutdown_connections 2024-04-20T10:27:45.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:45.789+0000 7fb3dca00640 1 -- 172.21.15.47:0/1480793508 wait complete. 2024-04-20T10:27:45.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:27:45.810 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:27:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.023+0000 7f3abd800640 1 Processor -- start 2024-04-20T10:27:46.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.023+0000 7f3abd800640 1 -- start start 2024-04-20T10:27:46.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.025+0000 7f3abd800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3ab815a130 0x7f3ab8073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.025+0000 7f3abd800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 0x7f3ab80739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.025+0000 7f3abd800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3ab815cc20 con 0x7f3ab8071a60 2024-04-20T10:27:46.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.025+0000 7f3abd800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3ab815d7f0 con 0x7f3ab815a130 2024-04-20T10:27:46.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.025+0000 7f3abd800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab815cec0 con 0x7f3ab806f870 2024-04-20T10:27:46.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.025+0000 7f3ab7400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3ab8071a60 0x7f3ab806f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41564/0 (socket says 172.21.15.47:41564) 2024-04-20T10:27:46.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.025+0000 7f3ab7400640 1 -- 172.21.15.47:0/2048671765 learned_addr learned my addr 172.21.15.47:0/2048671765 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:46.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab7e00640 1 --2- 172.21.15.47:0/2048671765 >> v2:172.21.15.175:3300/0 conn(0x7f3ab815a130 0x7f3ab8073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3abce00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 0x7f3ab80739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3631140957 0 0) 0x7f3ab815cc20 con 0x7f3ab8071a60 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3a9c003520 con 0x7f3ab8071a60 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1644835453 0 0) 0x7f3a9c003520 con 0x7f3ab8071a60 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 msgr2=0x7f3ab80739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 0x7f3ab80739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 >> v2:172.21.15.175:3300/0 conn(0x7f3ab815a130 msgr2=0x7f3ab8073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 --2- 172.21.15.47:0/2048671765 >> v2:172.21.15.175:3300/0 conn(0x7f3ab815a130 0x7f3ab8073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab815d1a0 con 0x7f3ab8071a60 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f3aa4001670 con 0x7f3ab8071a60 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 >> v1:172.21.15.47:6789/0 conn(0x7f3ab8071a60 legacy=0x7f3ab806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c0039b0 0x7f3a9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 0x7f3a9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004d40 0x7f3a9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab815d7f0 con 0x7f3a9c004d40 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab815cec0 con 0x7f3a9c0039b0 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.026+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a9c00bac0 con 0x7f3ab806f870 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3abce00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004d40 0x7f3a9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3ab7e00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c0039b0 0x7f3a9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3ab7400640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 0x7f3a9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3abce00640 1 -- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 msgr2=0x7f3a9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3abce00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab806f870 0x7f3a9c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3abce00640 1 -- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c0039b0 msgr2=0x7f3a9c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3abce00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c0039b0 0x7f3a9c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.027+0000 7f3abce00640 1 -- 172.21.15.47:0/2048671765 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a9c003520 con 0x7f3a9c004d40 2024-04-20T10:27:46.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.028+0000 7f3abce00640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004d40 0x7f3a9c0051c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009fc0 tx=0x7f3a9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ed3f85dc15efafb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0039070 con 0x7f3a9c004d40 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab002cc20 con 0x7f3a9c004d40 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3ab6a00640 1 -- 172.21.15.47:0/2048671765 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0009c10 con 0x7f3a9c004d40 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3abd800640 1 -- 172.21.15.47:0/2048671765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004d40 msgr2=0x7f3a9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3abd800640 1 --2- 172.21.15.47:0/2048671765 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a9c004d40 0x7f3a9c0051c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009fc0 tx=0x7f3a9c005830 comp rx=0 tx=0).stop 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3abce00640 1 -- 172.21.15.47:0/2048671765 reap_dead start 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3abd800640 1 -- 172.21.15.47:0/2048671765 shutdown_connections 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3abd800640 1 -- 172.21.15.47:0/2048671765 >> 172.21.15.47:0/2048671765 conn(0x7f3ab80cffa0 msgr2=0x7f3ab80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.029+0000 7f3abd800640 1 -- 172.21.15.47:0/2048671765 shutdown_connections 2024-04-20T10:27:46.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.030+0000 7f3abd800640 1 -- 172.21.15.47:0/2048671765 wait complete. 2024-04-20T10:27:46.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.031+0000 7f3abd800640 1 Processor -- start 2024-04-20T10:27:46.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.032+0000 7f3abd800640 1 -- start start 2024-04-20T10:27:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c004d40 0x7f3ab8140340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abd800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab80afae0 0x7f3ab80aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abd800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab80c92f0 0x7f3ab80c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abd800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab8073b30 con 0x7f3ab80c92f0 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abd800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab80739b0 con 0x7f3a9c004d40 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abd800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ab8073cb0 con 0x7f3ab80afae0 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab80c92f0 0x7f3ab80c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab80c92f0 0x7f3ab80c9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47064/0 (socket says 172.21.15.47:47064) 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 -- 172.21.15.47:0/1713426586 learned_addr learned my addr 172.21.15.47:0/1713426586 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3ab7e00640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c004d40 0x7f3ab8140340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3ab7400640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab80afae0 0x7f3ab80aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 -- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab80afae0 msgr2=0x7f3ab80aff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab80afae0 0x7f3ab80aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 -- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c004d40 msgr2=0x7f3ab8140340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c004d40 0x7f3ab8140340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.033+0000 7f3abce00640 1 -- 172.21.15.47:0/1713426586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab80b0540 con 0x7f3ab80c92f0 2024-04-20T10:27:46.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.034+0000 7f3abce00640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab80c92f0 0x7f3ab80c9770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ab002c690 tx=0x7f3ab0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.034+0000 7f3aafe00640 1 -- 172.21.15.47:0/1713426586 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0044070 con 0x7f3ab80c92f0 2024-04-20T10:27:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.034+0000 7f3aafe00640 1 -- 172.21.15.47:0/1713426586 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3ab003a040 con 0x7f3ab80c92f0 2024-04-20T10:27:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.034+0000 7f3aafe00640 1 -- 172.21.15.47:0/1713426586 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab0039030 con 0x7f3ab80c92f0 2024-04-20T10:27:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.034+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab80b0820 con 0x7f3ab80c92f0 2024-04-20T10:27:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.034+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab80b0de0 con 0x7f3ab80c92f0 2024-04-20T10:27:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.035+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab8075b30 con 0x7f3ab80c92f0 2024-04-20T10:27:46.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.036+0000 7f3aafe00640 1 -- 172.21.15.47:0/1713426586 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3ab0047070 con 0x7f3ab80c92f0 2024-04-20T10:27:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.037+0000 7f3aafe00640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a9806c8f0 0x7f3a9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.037+0000 7f3aafe00640 1 -- 172.21.15.47:0/1713426586 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3ab00ad8d0 con 0x7f3ab80c92f0 2024-04-20T10:27:46.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.038+0000 7f3ab7e00640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a9806c8f0 0x7f3a9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.039+0000 7f3ab7e00640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a9806c8f0 0x7f3a9806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009a10 tx=0x7f3aa8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.041+0000 7f3aafe00640 1 -- 172.21.15.47:0/1713426586 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3ab007b360 con 0x7f3ab80c92f0 2024-04-20T10:27:46.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.243+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3ab80d0d70 con 0x7f3ab80c92f0 2024-04-20T10:27:46.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.244+0000 7f3aafe00640 1 -- 172.21.15.47:0/1713426586 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f3ab007f660 con 0x7f3ab80c92f0 2024-04-20T10:27:46.244 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:27:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a9806c8f0 msgr2=0x7f3a9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a9806c8f0 0x7f3a9806edb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8009a10 tx=0x7f3aa8008040 comp rx=0 tx=0).stop 2024-04-20T10:27:46.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab80c92f0 msgr2=0x7f3ab80c9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab80c92f0 0x7f3ab80c9770 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ab002c690 tx=0x7f3ab0009750 comp rx=0 tx=0).stop 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 shutdown_connections 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a9806c8f0 0x7f3a9806edb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3ab80c92f0 0x7f3ab80c9770 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ab80afae0 0x7f3ab80aff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 --2- 172.21.15.47:0/1713426586 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a9c004d40 0x7f3ab8140340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 >> 172.21.15.47:0/1713426586 conn(0x7f3ab80cffa0 msgr2=0x7f3ab80ccfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 shutdown_connections 2024-04-20T10:27:46.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.247+0000 7f3abd800640 1 -- 172.21.15.47:0/1713426586 wait complete. 2024-04-20T10:27:46.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:27:46.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.487+0000 7fdd98c00640 1 Processor -- start 2024-04-20T10:27:46.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.487+0000 7fdd98c00640 1 -- start start 2024-04-20T10:27:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.488+0000 7fdd98c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd940d1010 0x7fdd940b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.488+0000 7fdd98c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd940d46f0 0x7fdd940d4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.488+0000 7fdd98c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd940d15b0 con 0x7fdd9415e8c0 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.488+0000 7fdd98c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdd940b6170 con 0x7fdd940d46f0 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.488+0000 7fdd98c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd940d1890 con 0x7fdd940d1010 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd93e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd940d46f0 0x7fdd940d4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd93e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdd940d46f0 0x7fdd940d4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47070/0 (socket says 172.21.15.47:47070) 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd93e00640 1 -- 172.21.15.47:0/328500112 learned_addr learned my addr 172.21.15.47:0/328500112 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd92a00640 1 --2- 172.21.15.47:0/328500112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd940d1010 0x7fdd940b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd93e00640 1 -- 172.21.15.47:0/328500112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd940d1010 msgr2=0x7fdd940b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd93e00640 1 --2- 172.21.15.47:0/328500112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd940d1010 0x7fdd940b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd93e00640 1 -- 172.21.15.47:0/328500112 >> v1:172.21.15.175:6789/0 conn(0x7fdd9415e8c0 legacy=0x7fdd9406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.489+0000 7fdd93e00640 1 -- 172.21.15.47:0/328500112 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd940d1b70 con 0x7fdd940d46f0 2024-04-20T10:27:46.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.491+0000 7fdd93e00640 1 --2- 172.21.15.47:0/328500112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d46f0 0x7fdd940d4b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009870 tx=0x7fdd8c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f6b69380c213ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.491+0000 7fdd92000640 1 -- 172.21.15.47:0/328500112 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c034070 con 0x7fdd940d46f0 2024-04-20T10:27:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.491+0000 7fdd92000640 1 -- 172.21.15.47:0/328500112 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd8c030840 con 0x7fdd940d46f0 2024-04-20T10:27:46.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.491+0000 7fdd92000640 1 -- 172.21.15.47:0/328500112 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd8c030c10 con 0x7fdd940d46f0 2024-04-20T10:27:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 -- 172.21.15.47:0/328500112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d46f0 msgr2=0x7fdd940d4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 --2- 172.21.15.47:0/328500112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d46f0 0x7fdd940d4b80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009870 tx=0x7fdd8c02b980 comp rx=0 tx=0).stop 2024-04-20T10:27:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 -- 172.21.15.47:0/328500112 shutdown_connections 2024-04-20T10:27:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 --2- 172.21.15.47:0/328500112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d46f0 0x7fdd940d4b80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 --2- 172.21.15.47:0/328500112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd940d1010 0x7fdd940b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 -- 172.21.15.47:0/328500112 >> 172.21.15.47:0/328500112 conn(0x7fdd9409d6f0 msgr2=0x7fdd940b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 -- 172.21.15.47:0/328500112 shutdown_connections 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.492+0000 7fdd98c00640 1 -- 172.21.15.47:0/328500112 wait complete. 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.494+0000 7fdd98c00640 1 Processor -- start 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.494+0000 7fdd98c00640 1 -- start start 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd98c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d1010 0x7fdd940ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd98c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd940d46f0 0x7fdd940cc920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd98c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd9415e8c0 0x7fdd941f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd98c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd940b6cc0 con 0x7fdd940d1010 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd98c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd940b6b40 con 0x7fdd940d46f0 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd98c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd940cfa10 con 0x7fdd9415e8c0 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d1010 0x7fdd940ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d1010 0x7fdd940ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47072/0 (socket says 172.21.15.47:47072) 2024-04-20T10:27:46.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 -- 172.21.15.47:0/2985839782 learned_addr learned my addr 172.21.15.47:0/2985839782 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd92a00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd940d46f0 0x7fdd940cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 -- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd9415e8c0 msgr2=0x7fdd941f7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd9415e8c0 0x7fdd941f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 -- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd940d46f0 msgr2=0x7fdd940cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd940d46f0 0x7fdd940cc920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.495+0000 7fdd93400640 1 -- 172.21.15.47:0/2985839782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd940cd250 con 0x7fdd940d1010 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.496+0000 7fdd93400640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d1010 0x7fdd940ca890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c3f0 tx=0x7fdd8400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.496+0000 7fdd8b400640 1 -- 172.21.15.47:0/2985839782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd840052d0 con 0x7fdd940d1010 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.496+0000 7fdd8b400640 1 -- 172.21.15.47:0/2985839782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd84005490 con 0x7fdd940d1010 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.496+0000 7fdd8b400640 1 -- 172.21.15.47:0/2985839782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd84007260 con 0x7fdd940d1010 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.496+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd940cd530 con 0x7fdd940d1010 2024-04-20T10:27:46.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.496+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd940cdaf0 con 0x7fdd940d1010 2024-04-20T10:27:46.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.498+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd94075b30 con 0x7fdd940d1010 2024-04-20T10:27:46.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.499+0000 7fdd8b400640 1 -- 172.21.15.47:0/2985839782 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd84007400 con 0x7fdd940d1010 2024-04-20T10:27:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.500+0000 7fdd8b400640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd7006c8f0 0x7fdd7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.500+0000 7fdd8b400640 1 -- 172.21.15.47:0/2985839782 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdd8408b360 con 0x7fdd940d1010 2024-04-20T10:27:46.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.501+0000 7fdd92a00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd7006c8f0 0x7fdd7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.502+0000 7fdd92a00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd7006c8f0 0x7fdd7006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd941f7970 tx=0x7fdd7c00b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.504+0000 7fdd8b400640 1 -- 172.21.15.47:0/2985839782 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdd84059440 con 0x7fdd940d1010 2024-04-20T10:27:46.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.708+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd940d2690 con 0x7fdd940d1010 2024-04-20T10:27:46.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.709+0000 7fdd8b400640 1 -- 172.21.15.47:0/2985839782 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fdd84058d40 con 0x7fdd940d1010 2024-04-20T10:27:46.710 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:27:46.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd7006c8f0 msgr2=0x7fdd7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd7006c8f0 0x7fdd7006edb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fdd941f7970 tx=0x7fdd7c00b040 comp rx=0 tx=0).stop 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d1010 msgr2=0x7fdd940ca890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d1010 0x7fdd940ca890 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fdd8400c3f0 tx=0x7fdd8400c8b0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 shutdown_connections 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd9415e8c0 0x7fdd941f7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd940d46f0 0x7fdd940cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd7006c8f0 0x7fdd7006edb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 --2- 172.21.15.47:0/2985839782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd940d1010 0x7fdd940ca890 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 >> 172.21.15.47:0/2985839782 conn(0x7fdd9409d6f0 msgr2=0x7fdd940b2450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:46.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.712+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 shutdown_connections 2024-04-20T10:27:46.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.713+0000 7fdd98c00640 1 -- 172.21.15.47:0/2985839782 wait complete. 2024-04-20T10:27:46.732 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:27:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.949+0000 7ffa5da00640 1 Processor -- start 2024-04-20T10:27:46.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.949+0000 7ffa5da00640 1 -- start start 2024-04-20T10:27:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa5da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 0x7ffa5806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa5da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffa5806dbb0 0x7ffa580ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa5da00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa5806f060 con 0x7ffa5815e8a0 2024-04-20T10:27:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa5da00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ffa5806e7c0 con 0x7ffa5806dbb0 2024-04-20T10:27:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa5da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5806eaa0 con 0x7ffa580d0a70 2024-04-20T10:27:46.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa56a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffa5815e8a0 0x7ffa580d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41568/0 (socket says 172.21.15.47:41568) 2024-04-20T10:27:46.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa56a00640 1 -- 172.21.15.47:0/1726103632 learned_addr learned my addr 172.21.15.47:0/1726103632 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.951+0000 7ffa57400640 1 --2- 172.21.15.47:0/1726103632 >> v2:172.21.15.175:3300/0 conn(0x7ffa5806dbb0 0x7ffa580ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa56000640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 0x7ffa5806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2592095967 0 0) 0x7ffa5806f060 con 0x7ffa5815e8a0 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ffa3c003520 con 0x7ffa5815e8a0 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4121365151 0 0) 0x7ffa3c003520 con 0x7ffa5815e8a0 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 msgr2=0x7ffa5806e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 0x7ffa5806e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 >> v2:172.21.15.175:3300/0 conn(0x7ffa5806dbb0 msgr2=0x7ffa580ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 --2- 172.21.15.47:0/1726103632 >> v2:172.21.15.175:3300/0 conn(0x7ffa5806dbb0 0x7ffa580ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5806ed80 con 0x7ffa5815e8a0 2024-04-20T10:27:46.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7ffa48001670 con 0x7ffa5815e8a0 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 >> v1:172.21.15.47:6789/0 conn(0x7ffa5815e8a0 legacy=0x7ffa580d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3c0039b0 0x7ffa3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa3c004800 0x7ffa3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 0x7ffa3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5806e7c0 con 0x7ffa3c004800 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5806eaa0 con 0x7ffa3c0039b0 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.952+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c00b9c0 con 0x7ffa580d0a70 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa56000640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa3c004800 0x7ffa3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa56a00640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3c0039b0 0x7ffa3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa57400640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 0x7ffa3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa56000640 1 -- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 msgr2=0x7ffa3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa56000640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa580d0a70 0x7ffa3c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa56000640 1 -- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3c0039b0 msgr2=0x7ffa3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa56000640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa3c0039b0 0x7ffa3c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.953+0000 7ffa56000640 1 -- 172.21.15.47:0/1726103632 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c003520 con 0x7ffa3c004800 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.954+0000 7ffa56000640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa3c004800 0x7ffa3c004c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009ac0 tx=0x7ffa3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e944ec1d32c2d8cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44039070 con 0x7ffa3c004800 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4402c6b0 con 0x7ffa3c004800 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa55600640 1 -- 172.21.15.47:0/1726103632 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44009690 con 0x7ffa3c004800 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa5da00640 1 -- 172.21.15.47:0/1726103632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa3c004800 msgr2=0x7ffa3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa5da00640 1 --2- 172.21.15.47:0/1726103632 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa3c004800 0x7ffa3c004c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ffa44009ac0 tx=0x7ffa3c005830 comp rx=0 tx=0).stop 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa57400640 1 -- 172.21.15.47:0/1726103632 reap_dead start 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa5da00640 1 -- 172.21.15.47:0/1726103632 shutdown_connections 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.955+0000 7ffa5da00640 1 -- 172.21.15.47:0/1726103632 >> 172.21.15.47:0/1726103632 conn(0x7ffa580b3350 msgr2=0x7ffa580b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.956+0000 7ffa5da00640 1 -- 172.21.15.47:0/1726103632 shutdown_connections 2024-04-20T10:27:46.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.956+0000 7ffa5da00640 1 -- 172.21.15.47:0/1726103632 wait complete. 2024-04-20T10:27:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.957+0000 7ffa5da00640 1 Processor -- start 2024-04-20T10:27:46.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.958+0000 7ffa5da00640 1 -- start start 2024-04-20T10:27:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.958+0000 7ffa5da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3c004800 0x7ffa581616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.958+0000 7ffa5da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa58158570 0x7ffa581589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.958+0000 7ffa5da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa58151200 0x7ffa58151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa5da00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5806fef0 con 0x7ffa58158570 2024-04-20T10:27:46.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa5da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5806fd70 con 0x7ffa58151200 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa5da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa5806e1e0 con 0x7ffa3c004800 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa58158570 0x7ffa581589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa58158570 0x7ffa581589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47076/0 (socket says 172.21.15.47:47076) 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 -- 172.21.15.47:0/3672497279 learned_addr learned my addr 172.21.15.47:0/3672497279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa57400640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa58151200 0x7ffa58151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56a00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3c004800 0x7ffa581616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 -- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3c004800 msgr2=0x7ffa581616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3c004800 0x7ffa581616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 -- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa58151200 msgr2=0x7ffa58151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa58151200 0x7ffa58151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.959+0000 7ffa56000640 1 -- 172.21.15.47:0/3672497279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa58160620 con 0x7ffa58158570 2024-04-20T10:27:46.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.960+0000 7ffa56000640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa58158570 0x7ffa581589d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa44037040 tx=0x7ffa44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.960+0000 7ffa4ea00640 1 -- 172.21.15.47:0/3672497279 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44042020 con 0x7ffa58158570 2024-04-20T10:27:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.960+0000 7ffa4ea00640 1 -- 172.21.15.47:0/3672497279 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa4403a040 con 0x7ffa58158570 2024-04-20T10:27:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.960+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa58160900 con 0x7ffa58158570 2024-04-20T10:27:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.960+0000 7ffa4ea00640 1 -- 172.21.15.47:0/3672497279 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa44039070 con 0x7ffa58158570 2024-04-20T10:27:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.960+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa58160ec0 con 0x7ffa58158570 2024-04-20T10:27:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.962+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa580b3c90 con 0x7ffa58158570 2024-04-20T10:27:46.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.962+0000 7ffa4ea00640 1 -- 172.21.15.47:0/3672497279 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa44045020 con 0x7ffa58158570 2024-04-20T10:27:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.963+0000 7ffa4ea00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffa3806c920 0x7ffa3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:46.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.964+0000 7ffa4ea00640 1 -- 172.21.15.47:0/3672497279 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ffa440acf10 con 0x7ffa58158570 2024-04-20T10:27:46.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.964+0000 7ffa56a00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffa3806c920 0x7ffa3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:46.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.965+0000 7ffa56a00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffa3806c920 0x7ffa3806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa48000f60 tx=0x7ffa48000ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:46.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:46.968+0000 7ffa4ea00640 1 -- 172.21.15.47:0/3672497279 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffa580b3c90 con 0x7ffa58158570 2024-04-20T10:27:47.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.183+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ffa58068980 con 0x7ffa58158570 2024-04-20T10:27:47.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.184+0000 7ffa4ea00640 1 -- 172.21.15.47:0/3672497279 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ffa4404c020 con 0x7ffa58158570 2024-04-20T10:27:47.185 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffa3806c920 msgr2=0x7ffa3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffa3806c920 0x7ffa3806ede0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffa48000f60 tx=0x7ffa48000ed0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa58158570 msgr2=0x7ffa581589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa58158570 0x7ffa581589d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ffa44037040 tx=0x7ffa44004730 comp rx=0 tx=0).stop 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 shutdown_connections 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa58151200 0x7ffa58151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ffa3806c920 0x7ffa3806ede0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa58158570 0x7ffa581589d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 --2- 172.21.15.47:0/3672497279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa3c004800 0x7ffa581616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.187+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 >> 172.21.15.47:0/3672497279 conn(0x7ffa580b3350 msgr2=0x7ffa58159920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.188+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 shutdown_connections 2024-04-20T10:27:47.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.188+0000 7ffa5da00640 1 -- 172.21.15.47:0/3672497279 wait complete. 2024-04-20T10:27:47.207 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.426+0000 7f0986400640 1 Processor -- start 2024-04-20T10:27:47.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.426+0000 7f0986400640 1 -- start start 2024-04-20T10:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.427+0000 7f0986400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0980161a50 0x7f09800b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.428+0000 7f0986400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09800d3b20 0x7f098006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.428+0000 7f0986400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f09800d0980 con 0x7f09800d0480 2024-04-20T10:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.428+0000 7f0986400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f09800d0c60 con 0x7f09800d3b20 2024-04-20T10:27:47.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.428+0000 7f0986400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0980162ae0 con 0x7f0980161a50 2024-04-20T10:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.428+0000 7f097fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09800d3b20 0x7f098006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.429+0000 7f097fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f09800d3b20 0x7f098006b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:51542/0 (socket says 172.21.15.47:51542) 2024-04-20T10:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.429+0000 7f097fe00640 1 -- 172.21.15.47:0/27269921 learned_addr learned my addr 172.21.15.47:0/27269921 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.428+0000 7f0985000640 1 --2- 172.21.15.47:0/27269921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0980161a50 0x7f09800b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.429+0000 7f0985000640 1 -- 172.21.15.47:0/27269921 >> v2:172.21.15.175:3300/0 conn(0x7f09800d3b20 msgr2=0x7f098006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.429+0000 7f0985000640 1 --2- 172.21.15.47:0/27269921 >> v2:172.21.15.175:3300/0 conn(0x7f09800d3b20 0x7f098006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.429+0000 7f0985000640 1 -- 172.21.15.47:0/27269921 >> v1:172.21.15.175:6789/0 conn(0x7f09800d0480 legacy=0x7f098006c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.429+0000 7f0985000640 1 -- 172.21.15.47:0/27269921 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0980162490 con 0x7f0980161a50 2024-04-20T10:27:47.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.431+0000 7f0985000640 1 --2- 172.21.15.47:0/27269921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0980161a50 0x7f09800b3180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0970009d90 tx=0x7f097002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36891ecd6e66cb24 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.431+0000 7f097f400640 1 -- 172.21.15.47:0/27269921 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970039070 con 0x7f0980161a50 2024-04-20T10:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.431+0000 7f097f400640 1 -- 172.21.15.47:0/27269921 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f097002cc50 con 0x7f0980161a50 2024-04-20T10:27:47.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.431+0000 7f097f400640 1 -- 172.21.15.47:0/27269921 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970034440 con 0x7f0980161a50 2024-04-20T10:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 -- 172.21.15.47:0/27269921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0980161a50 msgr2=0x7f09800b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 --2- 172.21.15.47:0/27269921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0980161a50 0x7f09800b3180 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0970009d90 tx=0x7f097002bea0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 -- 172.21.15.47:0/27269921 shutdown_connections 2024-04-20T10:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 --2- 172.21.15.47:0/27269921 >> v2:172.21.15.175:3300/0 conn(0x7f09800d3b20 0x7f098006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 --2- 172.21.15.47:0/27269921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0980161a50 0x7f09800b3180 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 -- 172.21.15.47:0/27269921 >> 172.21.15.47:0/27269921 conn(0x7f09800d0ed0 msgr2=0x7f09800d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:47.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 -- 172.21.15.47:0/27269921 shutdown_connections 2024-04-20T10:27:47.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.432+0000 7f0986400640 1 -- 172.21.15.47:0/27269921 wait complete. 2024-04-20T10:27:47.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.435+0000 7f0986400640 1 Processor -- start 2024-04-20T10:27:47.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.435+0000 7f0986400640 1 -- start start 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.436+0000 7f0986400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09800d0480 0x7f098015ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.436+0000 7f0986400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980161a50 0x7f09800c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.436+0000 7f0986400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09801f4290 0x7f09800ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.436+0000 7f0986400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0980161fb0 con 0x7f0980161a50 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.436+0000 7f0986400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0980161e30 con 0x7f09801f4290 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.436+0000 7f0986400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0980162130 con 0x7f09800d0480 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.436+0000 7f097fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980161a50 0x7f09800c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980161a50 0x7f09800c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47092/0 (socket says 172.21.15.47:47092) 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 -- 172.21.15.47:0/3890772035 learned_addr learned my addr 172.21.15.47:0/3890772035 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f0985a00640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09801f4290 0x7f09800ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f0985000640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09800d0480 0x7f098015ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 -- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09800d0480 msgr2=0x7f098015ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09800d0480 0x7f098015ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 -- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09801f4290 msgr2=0x7f09800ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09801f4290 0x7f09800ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 -- 172.21.15.47:0/3890772035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09800c8f90 con 0x7f0980161a50 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.437+0000 7f097fe00640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980161a50 0x7f09800c9b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f096800e8b0 tx=0x7f096800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.438+0000 7f097cc00640 1 -- 172.21.15.47:0/3890772035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0968017070 con 0x7f0980161a50 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.438+0000 7f097cc00640 1 -- 172.21.15.47:0/3890772035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f096800c940 con 0x7f0980161a50 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.438+0000 7f097cc00640 1 -- 172.21.15.47:0/3890772035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f096800cd10 con 0x7f0980161a50 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.438+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09800c8cb0 con 0x7f0980161a50 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.438+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09800ab110 con 0x7f0980161a50 2024-04-20T10:27:47.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.439+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0980075b30 con 0x7f0980161a50 2024-04-20T10:27:47.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.440+0000 7f097cc00640 1 -- 172.21.15.47:0/3890772035 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f096801a070 con 0x7f0980161a50 2024-04-20T10:27:47.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.441+0000 7f097cc00640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f095c06c920 0x7f095c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.441+0000 7f097cc00640 1 -- 172.21.15.47:0/3890772035 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f096808be20 con 0x7f0980161a50 2024-04-20T10:27:47.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.442+0000 7f0985000640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f095c06c920 0x7f095c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.443+0000 7f0985000640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f095c06c920 0x7f095c06ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0970036000 tx=0x7f0970030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:47.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.445+0000 7f097cc00640 1 -- 172.21.15.47:0/3890772035 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0968059e70 con 0x7f0980161a50 2024-04-20T10:27:47.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.649+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f0980074dd0 con 0x7f0980161a50 2024-04-20T10:27:47.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.650+0000 7f097cc00640 1 -- 172.21.15.47:0/3890772035 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f0968059770 con 0x7f0980161a50 2024-04-20T10:27:47.650 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.652+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f095c06c920 msgr2=0x7f095c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.652+0000 7f0986400640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f095c06c920 0x7f095c06ede0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0970036000 tx=0x7f0970030000 comp rx=0 tx=0).stop 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.652+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980161a50 msgr2=0x7f09800c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.652+0000 7f0986400640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980161a50 0x7f09800c9b40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f096800e8b0 tx=0x7f096800ed70 comp rx=0 tx=0).stop 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 shutdown_connections 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09801f4290 0x7f09800ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f095c06c920 0x7f095c06ede0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0980161a50 0x7f09800c9b40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 --2- 172.21.15.47:0/3890772035 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09800d0480 0x7f098015ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 >> 172.21.15.47:0/3890772035 conn(0x7f09800d0ed0 msgr2=0x7f09800b4e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:47.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 shutdown_connections 2024-04-20T10:27:47.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.653+0000 7f0986400640 1 -- 172.21.15.47:0/3890772035 wait complete. 2024-04-20T10:27:47.671 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:27:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.885+0000 7fdef9600640 1 Processor -- start 2024-04-20T10:27:47.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.885+0000 7fdef9600640 1 -- start start 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.887+0000 7fdef9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 0x7fdef406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.887+0000 7fdef9600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdef40d0360 0x7fdef40d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.887+0000 7fdef9600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdef40ade40 con 0x7fdef406dcb0 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.887+0000 7fdef9600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdef40ae0e0 con 0x7fdef40d0360 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.887+0000 7fdef9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdef40d0cf0 con 0x7fdef415b3b0 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.887+0000 7fdeea000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdef406dcb0 0x7fdef4071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41580/0 (socket says 172.21.15.47:41580) 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.887+0000 7fdeea000640 1 -- 172.21.15.47:0/2025888585 learned_addr learned my addr 172.21.15.47:0/2025888585 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:47.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2a00640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 0x7fdef406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef3400640 1 --2- 172.21.15.47:0/2025888585 >> v2:172.21.15.175:3300/0 conn(0x7fdef40d0360 0x7fdef40d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2846154446 0 0) 0x7fdef40ade40 con 0x7fdef406dcb0 2024-04-20T10:27:47.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fded8003520 con 0x7fdef406dcb0 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1082462653 0 0) 0x7fded8003520 con 0x7fdef406dcb0 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 msgr2=0x7fdef406eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 0x7fdef406eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 >> v2:172.21.15.175:3300/0 conn(0x7fdef40d0360 msgr2=0x7fdef40d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 --2- 172.21.15.47:0/2025888585 >> v2:172.21.15.175:3300/0 conn(0x7fdef40d0360 0x7fdef40d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef415b790 con 0x7fdef406dcb0 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fdedc001670 con 0x7fdef406dcb0 2024-04-20T10:27:47.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 >> v1:172.21.15.47:6789/0 conn(0x7fdef406dcb0 legacy=0x7fdef4071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fded80039b0 0x7fded8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.888+0000 7fdef2000640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 0x7fded8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.889+0000 7fdef2000640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fded80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.889+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdef40ae0e0 con 0x7fded8004d40 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.889+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdef40d0cf0 con 0x7fded80039b0 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.889+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fded800dbd0 con 0x7fdef415b3b0 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.889+0000 7fdeea000640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 0x7fded8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.889+0000 7fdef2a00640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fded80039b0 0x7fded8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.889+0000 7fdef3400640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fded80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.890+0000 7fdef3400640 1 -- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 msgr2=0x7fded8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.890+0000 7fdef3400640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef415b3b0 0x7fded8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.890+0000 7fdef3400640 1 -- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fded80039b0 msgr2=0x7fded8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:47.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.890+0000 7fdef3400640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fded80039b0 0x7fded8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.890+0000 7fdef3400640 1 -- 172.21.15.47:0/2025888585 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fded8003520 con 0x7fded8004d40 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.891+0000 7fdef3400640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fded80051c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009fc0 tx=0x7fded8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7d836f008a8d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.891+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec039070 con 0x7fded8004d40 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.891+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdeec02cc60 con 0x7fded8004d40 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.891+0000 7fdef2000640 1 -- 172.21.15.47:0/2025888585 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec009c10 con 0x7fded8004d40 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.891+0000 7fdef9600640 1 -- 172.21.15.47:0/2025888585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 msgr2=0x7fded80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.891+0000 7fdef9600640 1 --2- 172.21.15.47:0/2025888585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fded80051c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fdeec009fc0 tx=0x7fded8005830 comp rx=0 tx=0).stop 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.892+0000 7fdef3400640 1 -- 172.21.15.47:0/2025888585 reap_dead start 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.892+0000 7fdef9600640 1 -- 172.21.15.47:0/2025888585 shutdown_connections 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.892+0000 7fdef9600640 1 -- 172.21.15.47:0/2025888585 >> 172.21.15.47:0/2025888585 conn(0x7fdef40b3350 msgr2=0x7fdef40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.892+0000 7fdef9600640 1 -- 172.21.15.47:0/2025888585 shutdown_connections 2024-04-20T10:27:47.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.892+0000 7fdef9600640 1 -- 172.21.15.47:0/2025888585 wait complete. 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.894+0000 7fdef9600640 1 Processor -- start 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.894+0000 7fdef9600640 1 -- start start 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef9600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fdef4147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef9600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef4149400 0x7fdef41466f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef9600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef41489f0 0x7fdef41f6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef9600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdef40ae660 con 0x7fded8004d40 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef9600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdef40ae380 con 0x7fdef4149400 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef9600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdef40ae940 con 0x7fdef41489f0 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fdef4147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fdef4147fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47120/0 (socket says 172.21.15.47:47120) 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 -- 172.21.15.47:0/3626039928 learned_addr learned my addr 172.21.15.47:0/3626039928 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef3400640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef41489f0 0x7fdef41f6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdeea000640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef4149400 0x7fdef41466f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 -- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef41489f0 msgr2=0x7fdef41f6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef41489f0 0x7fdef41f6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 -- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef4149400 msgr2=0x7fdef41466f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef4149400 0x7fdef41466f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdef2a00640 1 -- 172.21.15.47:0/3626039928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdef4146f40 con 0x7fded8004d40 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.895+0000 7fdeea000640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef4149400 0x7fdef41466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.896+0000 7fdef2a00640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fdef4147fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdee4011940 tx=0x7fdee4011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.896+0000 7fdeeb400640 1 -- 172.21.15.47:0/3626039928 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee400faf0 con 0x7fded8004d40 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.896+0000 7fdeeb400640 1 -- 172.21.15.47:0/3626039928 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdee400fc90 con 0x7fded8004d40 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.896+0000 7fdeeb400640 1 -- 172.21.15.47:0/3626039928 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdee4014440 con 0x7fded8004d40 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.896+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdef4147220 con 0x7fded8004d40 2024-04-20T10:27:47.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.896+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdef41477e0 con 0x7fded8004d40 2024-04-20T10:27:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.898+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdef4068980 con 0x7fded8004d40 2024-04-20T10:27:47.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.899+0000 7fdeeb400640 1 -- 172.21.15.47:0/3626039928 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdee4012040 con 0x7fded8004d40 2024-04-20T10:27:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.899+0000 7fdeeb400640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fded006c8f0 0x7fded006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.900+0000 7fdeeb400640 1 -- 172.21.15.47:0/3626039928 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdee408c480 con 0x7fded8004d40 2024-04-20T10:27:47.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.900+0000 7fdeea000640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fded006c8f0 0x7fded006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:47.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.901+0000 7fdeea000640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fded006c8f0 0x7fded006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdef409e3a0 tx=0x7fdedc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:47.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:47.904+0000 7fdeeb400640 1 -- 172.21.15.47:0/3626039928 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdee4019030 con 0x7fded8004d40 2024-04-20T10:27:48.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.126+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdef4148520 con 0x7fded8004d40 2024-04-20T10:27:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.131+0000 7fdeeb400640 1 -- 172.21.15.47:0/3626039928 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7fdee405a600 con 0x7fded8004d40 2024-04-20T10:27:48.131 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:27:48.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fded006c8f0 msgr2=0x7fded006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fded006c8f0 0x7fded006edb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fdef409e3a0 tx=0x7fdedc009040 comp rx=0 tx=0).stop 2024-04-20T10:27:48.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 msgr2=0x7fdef4147fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fdef4147fe0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdee4011940 tx=0x7fdee4011e00 comp rx=0 tx=0).stop 2024-04-20T10:27:48.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 shutdown_connections 2024-04-20T10:27:48.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdef41489f0 0x7fdef41f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdef4149400 0x7fdef41466f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fded006c8f0 0x7fded006edb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 --2- 172.21.15.47:0/3626039928 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fded8004d40 0x7fdef4147fe0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 >> 172.21.15.47:0/3626039928 conn(0x7fdef40b3350 msgr2=0x7fdef4073360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:48.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 shutdown_connections 2024-04-20T10:27:48.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.134+0000 7fdef9600640 1 -- 172.21.15.47:0/3626039928 wait complete. 2024-04-20T10:27:48.152 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:27:48.152 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:27:48.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:48.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.414+0000 7fbaa2200640 1 Processor -- start 2024-04-20T10:27:48.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.415+0000 7fbaa2200640 1 -- start start 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.416+0000 7fbaa2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.416+0000 7fbaa2200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba9c0b3330 0x7fba9c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.416+0000 7fbaa2200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fba9c06c190 con 0x7fba9c06c6d0 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.416+0000 7fbaa2200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fba9c162130 con 0x7fba9c0b3330 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.416+0000 7fbaa2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c06c470 con 0x7fba9c161d50 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fba9be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba9c0b3330 0x7fba9c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fba9be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba9c0b3330 0x7fba9c0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47132/0 (socket says 172.21.15.47:47132) 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fba9be00640 1 -- 172.21.15.47:0/2386058711 learned_addr learned my addr 172.21.15.47:0/2386058711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:48.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fbaa0e00640 1 --2- 172.21.15.47:0/2386058711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fba9be00640 1 -- 172.21.15.47:0/2386058711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 msgr2=0x7fba9c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fba9be00640 1 --2- 172.21.15.47:0/2386058711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fba9be00640 1 -- 172.21.15.47:0/2386058711 >> v1:172.21.15.175:6789/0 conn(0x7fba9c06c6d0 legacy=0x7fba9c0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:48.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.417+0000 7fba9be00640 1 -- 172.21.15.47:0/2386058711 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c0d2e00 con 0x7fba9c0b3330 2024-04-20T10:27:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.418+0000 7fba9be00640 1 --2- 172.21.15.47:0/2386058711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b3880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba88009870 tx=0x7fba8802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c75cb59a11d4bde server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fba9b400640 1 -- 172.21.15.47:0/2386058711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88035020 con 0x7fba9c0b3330 2024-04-20T10:27:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fba9b400640 1 -- 172.21.15.47:0/2386058711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba88031640 con 0x7fba9c0b3330 2024-04-20T10:27:48.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fba9b400640 1 -- 172.21.15.47:0/2386058711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba8802d040 con 0x7fba9c0b3330 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fbaa2200640 1 -- 172.21.15.47:0/2386058711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 msgr2=0x7fba9c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fbaa2200640 1 --2- 172.21.15.47:0/2386058711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b3880 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fba88009870 tx=0x7fba8802b8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fbaa2200640 1 -- 172.21.15.47:0/2386058711 shutdown_connections 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fbaa2200640 1 --2- 172.21.15.47:0/2386058711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b3880 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fbaa2200640 1 --2- 172.21.15.47:0/2386058711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.419+0000 7fbaa2200640 1 -- 172.21.15.47:0/2386058711 >> 172.21.15.47:0/2386058711 conn(0x7fba9c0d4450 msgr2=0x7fba9c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.420+0000 7fbaa2200640 1 -- 172.21.15.47:0/2386058711 shutdown_connections 2024-04-20T10:27:48.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.420+0000 7fbaa2200640 1 -- 172.21.15.47:0/2386058711 wait complete. 2024-04-20T10:27:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.421+0000 7fbaa2200640 1 Processor -- start 2024-04-20T10:27:48.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.421+0000 7fbaa2200640 1 -- start start 2024-04-20T10:27:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.422+0000 7fbaa2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba9c06c6d0 0x7fba9c15b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.422+0000 7fbaa2200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.422+0000 7fbaa2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0b2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.422+0000 7fbaa2200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c06cb60 con 0x7fba9c0b3330 2024-04-20T10:27:48.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.422+0000 7fbaa2200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba9c0d33c0 con 0x7fba9c06c6d0 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.422+0000 7fbaa2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba9c06cce0 con 0x7fba9c161d50 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47134/0 (socket says 172.21.15.47:47134) 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 -- 172.21.15.47:0/3318651343 learned_addr learned my addr 172.21.15.47:0/3318651343 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fbaa0e00640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba9c06c6d0 0x7fba9c15b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fbaa1800640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0b2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 -- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 msgr2=0x7fba9c0b2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0b2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 -- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba9c06c6d0 msgr2=0x7fba9c15b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba9c06c6d0 0x7fba9c15b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.423+0000 7fba9be00640 1 -- 172.21.15.47:0/3318651343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba9c0ce380 con 0x7fba9c0b3330 2024-04-20T10:27:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.424+0000 7fba9be00640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b2330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.424+0000 7fba98c00640 1 -- 172.21.15.47:0/3318651343 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88040070 con 0x7fba9c0b3330 2024-04-20T10:27:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.424+0000 7fba98c00640 1 -- 172.21.15.47:0/3318651343 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba880399f0 con 0x7fba9c0b3330 2024-04-20T10:27:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.424+0000 7fba98c00640 1 -- 172.21.15.47:0/3318651343 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba88035040 con 0x7fba9c0b3330 2024-04-20T10:27:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.424+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba9c0aaf00 con 0x7fba9c0b3330 2024-04-20T10:27:48.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.424+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba9c0ab4c0 con 0x7fba9c0b3330 2024-04-20T10:27:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.425+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba9c0cf370 con 0x7fba9c0b3330 2024-04-20T10:27:48.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.426+0000 7fba98c00640 1 -- 172.21.15.47:0/3318651343 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba8804b060 con 0x7fba9c0b3330 2024-04-20T10:27:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.427+0000 7fba98c00640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fba7c06c8f0 0x7fba7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.427+0000 7fba98c00640 1 -- 172.21.15.47:0/3318651343 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fba880ad8e0 con 0x7fba9c0b3330 2024-04-20T10:27:48.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.428+0000 7fbaa0e00640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fba7c06c8f0 0x7fba7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.429+0000 7fbaa0e00640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fba7c06c8f0 0x7fba7c06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba90006fd0 tx=0x7fba90008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:48.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.432+0000 7fba98c00640 1 -- 172.21.15.47:0/3318651343 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fba9c0cf370 con 0x7fba9c0b3330 2024-04-20T10:27:48.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.640+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba9c0cef60 con 0x7fba9c0b3330 2024-04-20T10:27:48.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.644+0000 7fba98c00640 1 -- 172.21.15.47:0/3318651343 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7fba9c0cef60 con 0x7fba9c0b3330 2024-04-20T10:27:48.644 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:48.645 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.646+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fba7c06c8f0 msgr2=0x7fba7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.646+0000 7fbaa2200640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fba7c06c8f0 0x7fba7c06edb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fba90006fd0 tx=0x7fba90008040 comp rx=0 tx=0).stop 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.646+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 msgr2=0x7fba9c0b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.646+0000 7fbaa2200640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b2330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba880046c0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 shutdown_connections 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba9c161d50 0x7fba9c0b2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fba7c06c8f0 0x7fba7c06edb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba9c0b3330 0x7fba9c0b2330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 --2- 172.21.15.47:0/3318651343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba9c06c6d0 0x7fba9c15b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 >> 172.21.15.47:0/3318651343 conn(0x7fba9c0d4450 msgr2=0x7fba9c15ee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 shutdown_connections 2024-04-20T10:27:48.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.647+0000 7fbaa2200640 1 -- 172.21.15.47:0/3318651343 wait complete. 2024-04-20T10:27:48.666 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:27:48.666 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:27:48.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.873+0000 7f0342e00640 1 Processor -- start 2024-04-20T10:27:48.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.873+0000 7f0342e00640 1 -- start start 2024-04-20T10:27:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.874+0000 7f0342e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f033c15a130 0x7f033c073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.874+0000 7f0342e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f033c15cc20 con 0x7f033c06f870 2024-04-20T10:27:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.874+0000 7f0342e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f033c15cec0 con 0x7f033c071a60 2024-04-20T10:27:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.874+0000 7f0342e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f033c15d7f0 con 0x7f033c15a130 2024-04-20T10:27:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.875+0000 7f0341a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f033c15a130 0x7f033c073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.875+0000 7f0342400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f033c06f870 0x7f033c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41596/0 (socket says 172.21.15.47:41596) 2024-04-20T10:27:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.875+0000 7f0342400640 1 -- 172.21.15.47:0/4157484541 learned_addr learned my addr 172.21.15.47:0/4157484541 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:48.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.875+0000 7f033be00640 1 -- 172.21.15.47:0/4157484541 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1459346629 0 0) 0x7f033c15cc20 con 0x7f033c06f870 2024-04-20T10:27:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.876+0000 7f033be00640 1 -- 172.21.15.47:0/4157484541 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0324003520 con 0x7f033c06f870 2024-04-20T10:27:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.876+0000 7f0341a00640 1 -- 172.21.15.47:0/4157484541 >> v1:172.21.15.175:6789/0 conn(0x7f033c071a60 legacy=0x7f033c06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.876+0000 7f0341a00640 1 -- 172.21.15.47:0/4157484541 >> v1:172.21.15.47:6789/0 conn(0x7f033c06f870 legacy=0x7f033c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.876+0000 7f0341a00640 1 -- 172.21.15.47:0/4157484541 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c15d1a0 con 0x7f033c15a130 2024-04-20T10:27:48.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.877+0000 7f0341a00640 1 --2- 172.21.15.47:0/4157484541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c15a130 0x7f033c073470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f03300097a0 tx=0x7f033002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a84db159e8390bd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:48.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.877+0000 7f033be00640 1 -- 172.21.15.47:0/4157484541 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03300315a0 con 0x7f033c15a130 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.877+0000 7f033be00640 1 -- 172.21.15.47:0/4157484541 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0330037070 con 0x7f033c15a130 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.877+0000 7f033be00640 1 -- 172.21.15.47:0/4157484541 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0330031960 con 0x7f033c15a130 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.878+0000 7f0342e00640 1 -- 172.21.15.47:0/4157484541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c15a130 msgr2=0x7f033c073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.878+0000 7f0342e00640 1 --2- 172.21.15.47:0/4157484541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c15a130 0x7f033c073470 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f03300097a0 tx=0x7f033002b840 comp rx=0 tx=0).stop 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.878+0000 7f0342e00640 1 -- 172.21.15.47:0/4157484541 shutdown_connections 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.878+0000 7f0342e00640 1 --2- 172.21.15.47:0/4157484541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c15a130 0x7f033c073470 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.878+0000 7f0342e00640 1 -- 172.21.15.47:0/4157484541 >> 172.21.15.47:0/4157484541 conn(0x7f033c0cffa0 msgr2=0x7f033c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.878+0000 7f0342e00640 1 -- 172.21.15.47:0/4157484541 shutdown_connections 2024-04-20T10:27:48.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.878+0000 7f0342e00640 1 -- 172.21.15.47:0/4157484541 wait complete. 2024-04-20T10:27:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.880+0000 7f0342e00640 1 Processor -- start 2024-04-20T10:27:48.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.880+0000 7f0342e00640 1 -- start start 2024-04-20T10:27:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0342e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f033c06f870 0x7f033c0c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0342e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c071a60 0x7f033c0cf020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0342e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033c15a130 0x7f033c0cf560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0342e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f033c073b30 con 0x7f033c071a60 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0342e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f033c0739b0 con 0x7f033c15a130 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0342e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f033c073cb0 con 0x7f033c06f870 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0341000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c071a60 0x7f033c0cf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0341000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c071a60 0x7f033c0cf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47140/0 (socket says 172.21.15.47:47140) 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0341000640 1 -- 172.21.15.47:0/1790867315 learned_addr learned my addr 172.21.15.47:0/1790867315 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0341a00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f033c06f870 0x7f033c0c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.881+0000 7f0342400640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033c15a130 0x7f033c0cf560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0341000640 1 -- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f033c06f870 msgr2=0x7f033c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0341000640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f033c06f870 0x7f033c0c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0341000640 1 -- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033c15a130 msgr2=0x7f033c0cf560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0341000640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033c15a130 0x7f033c0cf560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0341000640 1 -- 172.21.15.47:0/1790867315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033c09b1f0 con 0x7f033c071a60 2024-04-20T10:27:48.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0341000640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c071a60 0x7f033c0cf020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f032800c400 tx=0x7f032800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0339600640 1 -- 172.21.15.47:0/1790867315 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03280052d0 con 0x7f033c071a60 2024-04-20T10:27:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0339600640 1 -- 172.21.15.47:0/1790867315 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0328005490 con 0x7f033c071a60 2024-04-20T10:27:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0339600640 1 -- 172.21.15.47:0/1790867315 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328007260 con 0x7f033c071a60 2024-04-20T10:27:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.882+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f033c09b4d0 con 0x7f033c071a60 2024-04-20T10:27:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.883+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f033c09ba90 con 0x7f033c071a60 2024-04-20T10:27:48.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.884+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033c075b30 con 0x7f033c071a60 2024-04-20T10:27:48.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.885+0000 7f0339600640 1 -- 172.21.15.47:0/1790867315 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0328007400 con 0x7f033c071a60 2024-04-20T10:27:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.886+0000 7f0339600640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f031c06c8f0 0x7f031c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.886+0000 7f0339600640 1 -- 172.21.15.47:0/1790867315 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f032808b3c0 con 0x7f033c071a60 2024-04-20T10:27:48.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.886+0000 7f0341a00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f031c06c8f0 0x7f031c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:48.888 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.888+0000 7f0341a00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f031c06c8f0 0x7f031c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0330036000 tx=0x7f033003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:48.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:48.891+0000 7f0339600640 1 -- 172.21.15.47:0/1790867315 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03280594b0 con 0x7f033c071a60 2024-04-20T10:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.099+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f033c06baa0 con 0x7f033c071a60 2024-04-20T10:27:49.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.100+0000 7f0339600640 1 -- 172.21.15.47:0/1790867315 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f0328058db0 con 0x7f033c071a60 2024-04-20T10:27:49.101 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:27:49.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f031c06c8f0 msgr2=0x7f031c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f031c06c8f0 0x7f031c06edb0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0330036000 tx=0x7f033003d040 comp rx=0 tx=0).stop 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c071a60 msgr2=0x7f033c0cf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c071a60 0x7f033c0cf020 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f032800c400 tx=0x7f032800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 shutdown_connections 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f033c15a130 0x7f033c0cf560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f031c06c8f0 0x7f031c06edb0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f033c071a60 0x7f033c0cf020 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 --2- 172.21.15.47:0/1790867315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f033c06f870 0x7f033c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 >> 172.21.15.47:0/1790867315 conn(0x7f033c0cffa0 msgr2=0x7f033c06e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 shutdown_connections 2024-04-20T10:27:49.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.103+0000 7f0342e00640 1 -- 172.21.15.47:0/1790867315 wait complete. 2024-04-20T10:27:49.123 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:27:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.343+0000 7fe7c1600640 1 Processor -- start 2024-04-20T10:27:49.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.343+0000 7fe7c1600640 1 -- start start 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.344+0000 7fe7c1600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe7bc06c8a0 0x7fe7bc15ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.344+0000 7fe7c1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc0cf210 0x7fe7bc145350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7c1600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe7bc0cf6f0 con 0x7fe7bc15a000 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7c1600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe7bc0cf9d0 con 0x7fe7bc06c8a0 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7c1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7bc0cfcb0 con 0x7fe7bc0cf210 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7baa00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe7bc15a000 0x7fe7bc06c190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41612/0 (socket says 172.21.15.47:41612) 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7baa00640 1 -- 172.21.15.47:0/1802722588 learned_addr learned my addr 172.21.15.47:0/1802722588 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7ba000640 1 --2- 172.21.15.47:0/1802722588 >> v2:172.21.15.47:3300/0 conn(0x7fe7bc06c8a0 0x7fe7bc15ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7bb400640 1 --2- 172.21.15.47:0/1802722588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc0cf210 0x7fe7bc145350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.345+0000 7fe7b9600640 1 -- 172.21.15.47:0/1802722588 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3533520093 0 0) 0x7fe7bc0cf6f0 con 0x7fe7bc15a000 2024-04-20T10:27:49.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.346+0000 7fe7b9600640 1 -- 172.21.15.47:0/1802722588 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe7a0003520 con 0x7fe7bc15a000 2024-04-20T10:27:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.346+0000 7fe7ba000640 1 -- 172.21.15.47:0/1802722588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc0cf210 msgr2=0x7fe7bc145350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.346+0000 7fe7ba000640 1 --2- 172.21.15.47:0/1802722588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc0cf210 0x7fe7bc145350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.346+0000 7fe7ba000640 1 -- 172.21.15.47:0/1802722588 >> v1:172.21.15.47:6789/0 conn(0x7fe7bc15a000 legacy=0x7fe7bc06c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.346+0000 7fe7ba000640 1 -- 172.21.15.47:0/1802722588 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc0cff90 con 0x7fe7bc06c8a0 2024-04-20T10:27:49.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.347+0000 7fe7ba000640 1 --2- 172.21.15.47:0/1802722588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06c8a0 0x7fe7bc15ca80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009d90 tx=0x7fe7ac02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a3cd7e47839991 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:49.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.347+0000 7fe7b9600640 1 -- 172.21.15.47:0/1802722588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac039070 con 0x7fe7bc06c8a0 2024-04-20T10:27:49.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.347+0000 7fe7b9600640 1 -- 172.21.15.47:0/1802722588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7ac02cc50 con 0x7fe7bc06c8a0 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.347+0000 7fe7b9600640 1 -- 172.21.15.47:0/1802722588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac034440 con 0x7fe7bc06c8a0 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 -- 172.21.15.47:0/1802722588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06c8a0 msgr2=0x7fe7bc15ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1802722588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06c8a0 0x7fe7bc15ca80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac009d90 tx=0x7fe7ac02bea0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 -- 172.21.15.47:0/1802722588 shutdown_connections 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1802722588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc0cf210 0x7fe7bc145350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1802722588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc06c8a0 0x7fe7bc15ca80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 -- 172.21.15.47:0/1802722588 >> 172.21.15.47:0/1802722588 conn(0x7fe7bc0704e0 msgr2=0x7fe7bc0708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 -- 172.21.15.47:0/1802722588 shutdown_connections 2024-04-20T10:27:49.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.348+0000 7fe7c1600640 1 -- 172.21.15.47:0/1802722588 wait complete. 2024-04-20T10:27:49.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.350+0000 7fe7c1600640 1 Processor -- start 2024-04-20T10:27:49.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.350+0000 7fe7c1600640 1 -- start start 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7c1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc06c8a0 0x7fe7bc0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7c1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7bc0cf210 0x7fe7bc06e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7c1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc15a000 0x7fe7bc142bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7c1600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc0d09c0 con 0x7fe7bc15a000 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7c1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7bc0d0840 con 0x7fe7bc0cf210 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7c1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7bc145890 con 0x7fe7bc06c8a0 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7bb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc15a000 0x7fe7bc142bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7bb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc15a000 0x7fe7bc142bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47148/0 (socket says 172.21.15.47:47148) 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.351+0000 7fe7bb400640 1 -- 172.21.15.47:0/1384626350 learned_addr learned my addr 172.21.15.47:0/1384626350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:49.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7ba000640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7bc0cf210 0x7fe7bc06e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7baa00640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc06c8a0 0x7fe7bc0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7bb400640 1 -- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc06c8a0 msgr2=0x7fe7bc0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7bb400640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc06c8a0 0x7fe7bc0ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7bb400640 1 -- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7bc0cf210 msgr2=0x7fe7bc06e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7bb400640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7bc0cf210 0x7fe7bc06e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7bb400640 1 -- 172.21.15.47:0/1384626350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7bc0cad90 con 0x7fe7bc15a000 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7ba000640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7bc0cf210 0x7fe7bc06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7baa00640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc06c8a0 0x7fe7bc0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:49.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7bb400640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc15a000 0x7fe7bc142bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e850 tx=0x7fe7b400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:49.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7b2a00640 1 -- 172.21.15.47:0/1384626350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4017070 con 0x7fe7bc15a000 2024-04-20T10:27:49.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7b2a00640 1 -- 172.21.15.47:0/1384626350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe7b400c940 con 0x7fe7bc15a000 2024-04-20T10:27:49.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7bc0cb070 con 0x7fe7bc15a000 2024-04-20T10:27:49.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.352+0000 7fe7b2a00640 1 -- 172.21.15.47:0/1384626350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400cd30 con 0x7fe7bc15a000 2024-04-20T10:27:49.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.353+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7bc0cb630 con 0x7fe7bc15a000 2024-04-20T10:27:49.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.354+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7bc069090 con 0x7fe7bc15a000 2024-04-20T10:27:49.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.355+0000 7fe7b2a00640 1 -- 172.21.15.47:0/1384626350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe7b401a070 con 0x7fe7bc15a000 2024-04-20T10:27:49.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.356+0000 7fe7b2a00640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe79806c8f0 0x7fe79806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.356+0000 7fe7b2a00640 1 -- 172.21.15.47:0/1384626350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe7b408c2a0 con 0x7fe7bc15a000 2024-04-20T10:27:49.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.356+0000 7fe7baa00640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe79806c8f0 0x7fe79806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.357+0000 7fe7baa00640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe79806c8f0 0x7fe79806edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4005880 tx=0x7fe7a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:49.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.360+0000 7fe7b2a00640 1 -- 172.21.15.47:0/1384626350 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe7b405a2f0 con 0x7fe7bc15a000 2024-04-20T10:27:49.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.563+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe7bc160d90 con 0x7fe7bc15a000 2024-04-20T10:27:49.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.563+0000 7fe7b2a00640 1 -- 172.21.15.47:0/1384626350 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fe7b4059bf0 con 0x7fe7bc15a000 2024-04-20T10:27:49.564 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe79806c8f0 msgr2=0x7fe79806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe79806c8f0 0x7fe79806edb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4005880 tx=0x7fe7a4009040 comp rx=0 tx=0).stop 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc15a000 msgr2=0x7fe7bc142bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc15a000 0x7fe7bc142bf0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400e850 tx=0x7fe7b400ed10 comp rx=0 tx=0).stop 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 shutdown_connections 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe79806c8f0 0x7fe79806edb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe7bc15a000 0x7fe7bc142bf0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7bc0cf210 0x7fe7bc06e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 --2- 172.21.15.47:0/1384626350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7bc06c8a0 0x7fe7bc0ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.566+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 >> 172.21.15.47:0/1384626350 conn(0x7fe7bc0704e0 msgr2=0x7fe7bc06c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.567+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 shutdown_connections 2024-04-20T10:27:49.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.567+0000 7fe7c1600640 1 -- 172.21.15.47:0/1384626350 wait complete. 2024-04-20T10:27:49.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:27:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.795+0000 7f2bf9e00640 1 Processor -- start 2024-04-20T10:27:49.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.796+0000 7f2bf9e00640 1 -- start start 2024-04-20T10:27:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.797+0000 7f2bf9e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2bf4154fe0 0x7f2bf413e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.797+0000 7f2bf9e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f2bf40cad70 0x7f2bf40ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.797+0000 7f2bf9e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2bf40ae130 con 0x7f2bf40add20 2024-04-20T10:27:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.797+0000 7f2bf9e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf415e5d0 con 0x7f2bf4154fe0 2024-04-20T10:27:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.797+0000 7f2bf9e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f2bf415e870 con 0x7f2bf40cad70 2024-04-20T10:27:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.798+0000 7f2bf8a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2bf4154fe0 0x7f2bf413e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.798+0000 7f2bf9400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2bf40add20 0x7f2bf415b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41614/0 (socket says 172.21.15.47:41614) 2024-04-20T10:27:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.798+0000 7f2bf9400640 1 -- 172.21.15.47:0/2000831034 learned_addr learned my addr 172.21.15.47:0/2000831034 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:49.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.798+0000 7f2bebe00640 1 --2- 172.21.15.47:0/2000831034 >> v2:172.21.15.175:3300/0 conn(0x7f2bf40cad70 0x7f2bf40ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.798+0000 7f2bf8a00640 1 -- 172.21.15.47:0/2000831034 >> v2:172.21.15.175:3300/0 conn(0x7f2bf40cad70 msgr2=0x7f2bf40ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.798+0000 7f2bf8a00640 1 --2- 172.21.15.47:0/2000831034 >> v2:172.21.15.175:3300/0 conn(0x7f2bf40cad70 0x7f2bf40ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.798+0000 7f2bf8a00640 1 -- 172.21.15.47:0/2000831034 >> v1:172.21.15.47:6789/0 conn(0x7f2bf40add20 legacy=0x7f2bf415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.799+0000 7f2bf8a00640 1 -- 172.21.15.47:0/2000831034 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf415eb50 con 0x7f2bf4154fe0 2024-04-20T10:27:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.799+0000 7f2bf3e00640 1 -- 172.21.15.47:0/2000831034 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2906171982 0 0) 0x7f2bec001490 con 0x7f2bf40add20 2024-04-20T10:27:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.799+0000 7f2bebe00640 1 --2- 172.21.15.47:0/2000831034 >> v2:172.21.15.175:3300/0 conn(0x7f2bf40cad70 0x7f2bf40ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:49.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.800+0000 7f2bf8a00640 1 --2- 172.21.15.47:0/2000831034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf4154fe0 0x7f2bf413e1b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4158bb0 tx=0x7f2be402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8b46cdd97f6a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.800+0000 7f2bf3e00640 1 -- 172.21.15.47:0/2000831034 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4035020 con 0x7f2bf4154fe0 2024-04-20T10:27:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.800+0000 7f2bf3e00640 1 -- 172.21.15.47:0/2000831034 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be4031830 con 0x7f2bf4154fe0 2024-04-20T10:27:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.800+0000 7f2bf3e00640 1 -- 172.21.15.47:0/2000831034 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4031c00 con 0x7f2bf4154fe0 2024-04-20T10:27:49.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 -- 172.21.15.47:0/2000831034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf4154fe0 msgr2=0x7f2bf413e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/2000831034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf4154fe0 0x7f2bf413e1b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2bf4158bb0 tx=0x7f2be402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 -- 172.21.15.47:0/2000831034 shutdown_connections 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/2000831034 >> v2:172.21.15.175:3300/0 conn(0x7f2bf40cad70 0x7f2bf40ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/2000831034 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf4154fe0 0x7f2bf413e1b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 -- 172.21.15.47:0/2000831034 >> 172.21.15.47:0/2000831034 conn(0x7f2bf40cffa0 msgr2=0x7f2bf40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 -- 172.21.15.47:0/2000831034 shutdown_connections 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.801+0000 7f2bf9e00640 1 -- 172.21.15.47:0/2000831034 wait complete. 2024-04-20T10:27:49.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.802+0000 7f2bf9e00640 1 Processor -- start 2024-04-20T10:27:49.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.803+0000 7f2bf9e00640 1 -- start start 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf40add20 0x7f2bf40b5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bf40cad70 0x7f2bf413d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bf4154fe0 0x7f2bf40afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf9e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf415f3f0 con 0x7f2bf40add20 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf415f110 con 0x7f2bf4154fe0 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bf41553c0 con 0x7f2bf40cad70 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf40add20 0x7f2bf40b5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf40add20 0x7f2bf40b5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47156/0 (socket says 172.21.15.47:47156) 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 -- 172.21.15.47:0/3847717194 learned_addr learned my addr 172.21.15.47:0/3847717194 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:49.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bebe00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bf40cad70 0x7f2bf413d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 -- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bf40cad70 msgr2=0x7f2bf413d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf9400640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bf4154fe0 0x7f2bf40afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bf40cad70 0x7f2bf413d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 -- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bf4154fe0 msgr2=0x7f2bf40afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bf4154fe0 0x7f2bf40afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.804+0000 7f2bf8a00640 1 -- 172.21.15.47:0/3847717194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf406dc10 con 0x7f2bf40add20 2024-04-20T10:27:49.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.805+0000 7f2bf8a00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf40add20 0x7f2bf40b5680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009890 tx=0x7f2be40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.805+0000 7f2bf1600640 1 -- 172.21.15.47:0/3847717194 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4040070 con 0x7f2bf40add20 2024-04-20T10:27:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.805+0000 7f2bf1600640 1 -- 172.21.15.47:0/3847717194 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2be403e070 con 0x7f2bf40add20 2024-04-20T10:27:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.805+0000 7f2bf1600640 1 -- 172.21.15.47:0/3847717194 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be4035040 con 0x7f2bf40add20 2024-04-20T10:27:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.805+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf406def0 con 0x7f2bf40add20 2024-04-20T10:27:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.805+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf406e4b0 con 0x7f2bf40add20 2024-04-20T10:27:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.807+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bf4075b30 con 0x7f2bf40add20 2024-04-20T10:27:49.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.808+0000 7f2bf1600640 1 -- 172.21.15.47:0/3847717194 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2be4038020 con 0x7f2bf40add20 2024-04-20T10:27:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.808+0000 7f2bf1600640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2bd006c8f0 0x7f2bd006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:49.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.809+0000 7f2bf1600640 1 -- 172.21.15.47:0/3847717194 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f2be40ad910 con 0x7f2bf40add20 2024-04-20T10:27:49.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.809+0000 7f2bebe00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2bd006c8f0 0x7f2bd006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:49.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.810+0000 7f2bebe00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2bd006c8f0 0x7f2bd006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc006fd0 tx=0x7f2bdc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:49.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:49.814+0000 7f2bf1600640 1 -- 172.21.15.47:0/3847717194 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2be407b960 con 0x7f2bf40add20 2024-04-20T10:27:50.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.017+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2bf406cd00 con 0x7f2bf40add20 2024-04-20T10:27:50.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.018+0000 7f2bf1600640 1 -- 172.21.15.47:0/3847717194 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f2be40317f0 con 0x7f2bf40add20 2024-04-20T10:27:50.018 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2bd006c8f0 msgr2=0x7f2bd006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2bd006c8f0 0x7f2bd006edb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc006fd0 tx=0x7f2bdc008040 comp rx=0 tx=0).stop 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf40add20 msgr2=0x7f2bf40b5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf40add20 0x7f2bf40b5680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f2be4009890 tx=0x7f2be40046c0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 shutdown_connections 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bf4154fe0 0x7f2bf40afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bf40cad70 0x7f2bf413d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2bd006c8f0 0x7f2bd006edb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 --2- 172.21.15.47:0/3847717194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2bf40add20 0x7f2bf40b5680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.020+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 >> 172.21.15.47:0/3847717194 conn(0x7f2bf40cffa0 msgr2=0x7f2bf406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.021+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 shutdown_connections 2024-04-20T10:27:50.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.021+0000 7f2bf9e00640 1 -- 172.21.15.47:0/3847717194 wait complete. 2024-04-20T10:27:50.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:27:50.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.248+0000 7f6ceec00640 1 Processor -- start 2024-04-20T10:27:50.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.248+0000 7f6ceec00640 1 -- start start 2024-04-20T10:27:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6ceec00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ce8161a50 0x7f6ce80b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6ceec00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6ce806c010 0x7f6ce80ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6ceec00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ce80d0980 con 0x7f6ce80d3b20 2024-04-20T10:27:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6ceec00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce80d0c60 con 0x7f6ce8161a50 2024-04-20T10:27:50.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6ceec00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6ce8162ae0 con 0x7f6ce806c010 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6cece00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6ce80d3b20 0x7f6ce806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41626/0 (socket says 172.21.15.47:41626) 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6ced800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ce8161a50 0x7f6ce80b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.250+0000 7f6cece00640 1 -- 172.21.15.47:0/574796541 learned_addr learned my addr 172.21.15.47:0/574796541 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.251+0000 7f6cee200640 1 --2- 172.21.15.47:0/574796541 >> v2:172.21.15.175:3300/0 conn(0x7f6ce806c010 0x7f6ce80ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.251+0000 7f6ced800640 1 -- 172.21.15.47:0/574796541 >> v2:172.21.15.175:3300/0 conn(0x7f6ce806c010 msgr2=0x7f6ce80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.251+0000 7f6ced800640 1 --2- 172.21.15.47:0/574796541 >> v2:172.21.15.175:3300/0 conn(0x7f6ce806c010 0x7f6ce80ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.251+0000 7f6ced800640 1 -- 172.21.15.47:0/574796541 >> v1:172.21.15.47:6789/0 conn(0x7f6ce80d3b20 legacy=0x7f6ce806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.251+0000 7f6ced800640 1 -- 172.21.15.47:0/574796541 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce8162490 con 0x7f6ce8161a50 2024-04-20T10:27:50.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.251+0000 7f6ce7e00640 1 -- 172.21.15.47:0/574796541 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4101915092 0 0) 0x7f6ce80d0980 con 0x7f6ce80d3b20 2024-04-20T10:27:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.252+0000 7f6ced800640 1 --2- 172.21.15.47:0/574796541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80b3180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097a0 tx=0x7f6cdc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c831f10b6d7800 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.252+0000 7f6ce7e00640 1 -- 172.21.15.47:0/574796541 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc035020 con 0x7f6ce8161a50 2024-04-20T10:27:50.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ce7e00640 1 -- 172.21.15.47:0/574796541 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cdc031830 con 0x7f6ce8161a50 2024-04-20T10:27:50.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ce7e00640 1 -- 172.21.15.47:0/574796541 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc031c00 con 0x7f6ce8161a50 2024-04-20T10:27:50.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 -- 172.21.15.47:0/574796541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 msgr2=0x7f6ce80b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 --2- 172.21.15.47:0/574796541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80b3180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc0097a0 tx=0x7f6cdc02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 -- 172.21.15.47:0/574796541 shutdown_connections 2024-04-20T10:27:50.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 --2- 172.21.15.47:0/574796541 >> v2:172.21.15.175:3300/0 conn(0x7f6ce806c010 0x7f6ce80ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 --2- 172.21.15.47:0/574796541 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80b3180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 -- 172.21.15.47:0/574796541 >> 172.21.15.47:0/574796541 conn(0x7f6ce80d0ed0 msgr2=0x7f6ce80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:50.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 -- 172.21.15.47:0/574796541 shutdown_connections 2024-04-20T10:27:50.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.253+0000 7f6ceec00640 1 -- 172.21.15.47:0/574796541 wait complete. 2024-04-20T10:27:50.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.255+0000 7f6ceec00640 1 Processor -- start 2024-04-20T10:27:50.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.255+0000 7f6ceec00640 1 -- start start 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.256+0000 7f6ceec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce806c010 0x7f6ce815c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.256+0000 7f6ceec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.256+0000 7f6ceec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80abb50 0x7f6ce80cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.256+0000 7f6ceec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce80d0600 con 0x7f6ce8161a50 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.256+0000 7f6ceec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce80d0480 con 0x7f6ce80abb50 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.256+0000 7f6ceec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ce8161e30 con 0x7f6ce806c010 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47180/0 (socket says 172.21.15.47:47180) 2024-04-20T10:27:50.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 -- 172.21.15.47:0/1676903452 learned_addr learned my addr 172.21.15.47:0/1676903452 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6ced800640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce806c010 0x7f6ce815c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cee200640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80abb50 0x7f6ce80cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 -- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce806c010 msgr2=0x7f6ce815c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce806c010 0x7f6ce815c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 -- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80abb50 msgr2=0x7f6ce80cd6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80abb50 0x7f6ce80cd6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cece00640 1 -- 172.21.15.47:0/1676903452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce80cae30 con 0x7f6ce8161a50 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6ced800640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce806c010 0x7f6ce815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:50.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.257+0000 7f6cee200640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80abb50 0x7f6ce80cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:50.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.258+0000 7f6cece00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80ab610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c4b0 tx=0x7f6cd400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:50.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.258+0000 7f6ce5600640 1 -- 172.21.15.47:0/1676903452 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4018070 con 0x7f6ce8161a50 2024-04-20T10:27:50.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.258+0000 7f6ce5600640 1 -- 172.21.15.47:0/1676903452 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6cd4007500 con 0x7f6ce8161a50 2024-04-20T10:27:50.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.258+0000 7f6ce5600640 1 -- 172.21.15.47:0/1676903452 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4004a20 con 0x7f6ce8161a50 2024-04-20T10:27:50.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.258+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce80ca2b0 con 0x7f6ce8161a50 2024-04-20T10:27:50.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.258+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce80ca870 con 0x7f6ce8161a50 2024-04-20T10:27:50.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.259+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ce8075b30 con 0x7f6ce8161a50 2024-04-20T10:27:50.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.260+0000 7f6ce5600640 1 -- 172.21.15.47:0/1676903452 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6cd401b070 con 0x7f6ce8161a50 2024-04-20T10:27:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.261+0000 7f6ce5600640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6cc806c8f0 0x7f6cc806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.261+0000 7f6ce5600640 1 -- 172.21.15.47:0/1676903452 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6cd408b130 con 0x7f6ce8161a50 2024-04-20T10:27:50.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.262+0000 7f6ced800640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6cc806c8f0 0x7f6cc806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.263+0000 7f6ced800640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6cc806c8f0 0x7f6cc806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc036000 tx=0x7f6cdc03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:50.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.266+0000 7f6ce5600640 1 -- 172.21.15.47:0/1676903452 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ce80cae30 con 0x7f6ce8161a50 2024-04-20T10:27:50.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.466+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6ce806cd20 con 0x7f6ce8161a50 2024-04-20T10:27:50.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.467+0000 7f6ce5600640 1 -- 172.21.15.47:0/1676903452 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f6cd405f020 con 0x7f6ce8161a50 2024-04-20T10:27:50.467 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:27:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6cc806c8f0 msgr2=0x7f6cc806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6cc806c8f0 0x7f6cc806edb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc036000 tx=0x7f6cdc03c040 comp rx=0 tx=0).stop 2024-04-20T10:27:50.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 msgr2=0x7f6ce80ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80ab610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400c4b0 tx=0x7f6cd400c970 comp rx=0 tx=0).stop 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 shutdown_connections 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ce80abb50 0x7f6ce80cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6cc806c8f0 0x7f6cc806edb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ce8161a50 0x7f6ce80ab610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 --2- 172.21.15.47:0/1676903452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ce806c010 0x7f6ce815c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 >> 172.21.15.47:0/1676903452 conn(0x7f6ce80d0ed0 msgr2=0x7f6ce815f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 shutdown_connections 2024-04-20T10:27:50.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.469+0000 7f6ceec00640 1 -- 172.21.15.47:0/1676903452 wait complete. 2024-04-20T10:27:50.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.697+0000 7f8027200640 1 Processor -- start 2024-04-20T10:27:50.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.697+0000 7f8027200640 1 -- start start 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.698+0000 7f8027200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f802015b400 0x7f8020072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.698+0000 7f8027200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f802015eec0 0x7f802015f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.698+0000 7f8027200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 0x7f802015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.698+0000 7f8027200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f802006d280 con 0x7f802015b400 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.698+0000 7f8027200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f802015fcd0 con 0x7f802015eec0 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.698+0000 7f8027200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f802006ccc0 con 0x7f802015f870 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.698+0000 7f8025e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f802015b400 0x7f8020072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f802015b400 0x7f8020072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47188/0 (socket says 172.21.15.47:47188) 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025e00640 1 -- 172.21.15.47:0/2730202785 learned_addr learned my addr 172.21.15.47:0/2730202785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025400640 1 --2- 172.21.15.47:0/2730202785 >> v2:172.21.15.175:3300/0 conn(0x7f802015eec0 0x7f802015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8026800640 1 --2- 172.21.15.47:0/2730202785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 0x7f802015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025e00640 1 -- 172.21.15.47:0/2730202785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 msgr2=0x7f802015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025e00640 1 --2- 172.21.15.47:0/2730202785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 0x7f802015b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025e00640 1 -- 172.21.15.47:0/2730202785 >> v2:172.21.15.175:3300/0 conn(0x7f802015eec0 msgr2=0x7f802015f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025e00640 1 --2- 172.21.15.47:0/2730202785 >> v2:172.21.15.175:3300/0 conn(0x7f802015eec0 0x7f802015f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.699+0000 7f8025e00640 1 -- 172.21.15.47:0/2730202785 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802006cfa0 con 0x7f802015b400 2024-04-20T10:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.700+0000 7f8025e00640 1 --2- 172.21.15.47:0/2730202785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f8020072d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8020074a40 tx=0x7f801402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70821caf1a4b933c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8024a00640 1 -- 172.21.15.47:0/2730202785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014035020 con 0x7f802015b400 2024-04-20T10:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8024a00640 1 -- 172.21.15.47:0/2730202785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8014031870 con 0x7f802015b400 2024-04-20T10:27:50.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8024a00640 1 -- 172.21.15.47:0/2730202785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014031c40 con 0x7f802015b400 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 -- 172.21.15.47:0/2730202785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 msgr2=0x7f8020072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 --2- 172.21.15.47:0/2730202785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f8020072d00 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f8020074a40 tx=0x7f801402b910 comp rx=0 tx=0).stop 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 -- 172.21.15.47:0/2730202785 shutdown_connections 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 --2- 172.21.15.47:0/2730202785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 0x7f802015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 --2- 172.21.15.47:0/2730202785 >> v2:172.21.15.175:3300/0 conn(0x7f802015eec0 0x7f802015f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 --2- 172.21.15.47:0/2730202785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f8020072d00 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 -- 172.21.15.47:0/2730202785 >> 172.21.15.47:0/2730202785 conn(0x7f80200d0040 msgr2=0x7f80200d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 -- 172.21.15.47:0/2730202785 shutdown_connections 2024-04-20T10:27:50.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.701+0000 7f8027200640 1 -- 172.21.15.47:0/2730202785 wait complete. 2024-04-20T10:27:50.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.703+0000 7f8027200640 1 Processor -- start 2024-04-20T10:27:50.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.703+0000 7f8027200640 1 -- start start 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.704+0000 7f8027200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f802009d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.704+0000 7f8027200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f802015eec0 0x7f802009abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.704+0000 7f8027200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 0x7f802009b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.704+0000 7f8027200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8020073240 con 0x7f802015b400 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.704+0000 7f8027200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f802006e720 con 0x7f802015eec0 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.704+0000 7f8027200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f802015bd20 con 0x7f802015f870 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f802009d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f802009d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47204/0 (socket says 172.21.15.47:47204) 2024-04-20T10:27:50.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 -- 172.21.15.47:0/3226329910 learned_addr learned my addr 172.21.15.47:0/3226329910 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 -- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 msgr2=0x7f802009b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 0x7f802009b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 -- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f802015eec0 msgr2=0x7f802009abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f802015eec0 0x7f802009abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 -- 172.21.15.47:0/3226329910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80200cc2b0 con 0x7f802015b400 2024-04-20T10:27:50.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.705+0000 7f8025e00640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f802009d420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f801402b950 tx=0x7f80140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.706+0000 7f801a000640 1 -- 172.21.15.47:0/3226329910 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014040070 con 0x7f802015b400 2024-04-20T10:27:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.706+0000 7f801a000640 1 -- 172.21.15.47:0/3226329910 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8014030d00 con 0x7f802015b400 2024-04-20T10:27:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.706+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80200cd740 con 0x7f802015b400 2024-04-20T10:27:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.706+0000 7f801a000640 1 -- 172.21.15.47:0/3226329910 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014035040 con 0x7f802015b400 2024-04-20T10:27:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.706+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80200cdd00 con 0x7f802015b400 2024-04-20T10:27:50.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.707+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80200cdf00 con 0x7f802015b400 2024-04-20T10:27:50.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.708+0000 7f801a000640 1 -- 172.21.15.47:0/3226329910 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f801404b060 con 0x7f802015b400 2024-04-20T10:27:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.709+0000 7f801a000640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f800406c8f0 0x7f800406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.709+0000 7f801a000640 1 -- 172.21.15.47:0/3226329910 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f80140adb00 con 0x7f802015b400 2024-04-20T10:27:50.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.709+0000 7f8025400640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f800406c8f0 0x7f800406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:50.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.710+0000 7f8025400640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f800406c8f0 0x7f800406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f802015a8c0 tx=0x7f8010009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:50.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.713+0000 7f801a000640 1 -- 172.21.15.47:0/3226329910 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f80200cdf00 con 0x7f802015b400 2024-04-20T10:27:50.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.932+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f80200b5220 con 0x7f802015b400 2024-04-20T10:27:50.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.933+0000 7f801a000640 1 -- 172.21.15.47:0/3226329910 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f801407bc80 con 0x7f802015b400 2024-04-20T10:27:50.933 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f800406c8f0 msgr2=0x7f800406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f800406c8f0 0x7f800406edb0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f802015a8c0 tx=0x7f8010009500 comp rx=0 tx=0).stop 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 msgr2=0x7f802009d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f802009d420 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f801402b950 tx=0x7f80140046c0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 shutdown_connections 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f802015f870 0x7f802009b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f802015eec0 0x7f802009abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f800406c8f0 0x7f800406edb0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 --2- 172.21.15.47:0/3226329910 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f802015b400 0x7f802009d420 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.936+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 >> 172.21.15.47:0/3226329910 conn(0x7f80200d0040 msgr2=0x7f802006eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.937+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 shutdown_connections 2024-04-20T10:27:50.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:50.937+0000 7f8027200640 1 -- 172.21.15.47:0/3226329910 wait complete. 2024-04-20T10:27:50.955 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-04-20T10:27:50.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:50.955 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-20T10:27:51.034 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi047.stderr:ceph-fuse[27722]: fuse finished with error 0 and tester_r 0 2024-04-20T10:27:51.043 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:27:51.122 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:27:51.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:27:51.122 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-20T10:27:51.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:51.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.407+0000 7fd9df200640 1 Processor -- start 2024-04-20T10:27:51.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.407+0000 7fd9df200640 1 -- start start 2024-04-20T10:27:51.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.408+0000 7fd9df200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9d80d1e00 0x7fd9d80d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.408+0000 7fd9df200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d80d17e0 con 0x7fd9d80d1400 2024-04-20T10:27:51.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.408+0000 7fd9df200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd9d81623c0 con 0x7fd9d815e680 2024-04-20T10:27:51.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.408+0000 7fd9df200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd9d8162fd0 con 0x7fd9d80d1e00 2024-04-20T10:27:51.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9dd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9d80d1e00 0x7fd9d80d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9de800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd9d80d1400 0x7fd9d80cca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41632/0 (socket says 172.21.15.47:41632) 2024-04-20T10:27:51.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9de800640 1 -- 172.21.15.47:0/3468781590 learned_addr learned my addr 172.21.15.47:0/3468781590 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:51.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9dd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd9d80d1e00 0x7fd9d80d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47208/0 (socket says 172.21.15.47:47208) 2024-04-20T10:27:51.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9dd400640 1 -- 172.21.15.47:0/3468781590 >> v1:172.21.15.175:6789/0 conn(0x7fd9d815e680 legacy=0x7fd9d80b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:51.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9dd400640 1 -- 172.21.15.47:0/3468781590 >> v1:172.21.15.47:6789/0 conn(0x7fd9d80d1400 legacy=0x7fd9d80cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9dd400640 1 -- 172.21.15.47:0/3468781590 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d81626a0 con 0x7fd9d80d1e00 2024-04-20T10:27:51.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.409+0000 7fd9dca00640 1 -- 172.21.15.47:0/3468781590 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 436669622 0 0) 0x7fd9d80d17e0 con 0x7fd9d80d1400 2024-04-20T10:27:51.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.410+0000 7fd9dd400640 1 --2- 172.21.15.47:0/3468781590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d80d1e00 0x7fd9d80d0ec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e4f7bafef4a52a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:51.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.411+0000 7fd9dca00640 1 -- 172.21.15.47:0/3468781590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8035020 con 0x7fd9d80d1e00 2024-04-20T10:27:51.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.411+0000 7fd9dca00640 1 -- 172.21.15.47:0/3468781590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9c8031830 con 0x7fd9d80d1e00 2024-04-20T10:27:51.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.411+0000 7fd9dca00640 1 -- 172.21.15.47:0/3468781590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c8031c00 con 0x7fd9d80d1e00 2024-04-20T10:27:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.411+0000 7fd9df200640 1 -- 172.21.15.47:0/3468781590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d80d1e00 msgr2=0x7fd9d80d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.411+0000 7fd9df200640 1 --2- 172.21.15.47:0/3468781590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d80d1e00 0x7fd9d80d0ec0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd9c80097a0 tx=0x7fd9c802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.412+0000 7fd9df200640 1 -- 172.21.15.47:0/3468781590 shutdown_connections 2024-04-20T10:27:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.412+0000 7fd9df200640 1 --2- 172.21.15.47:0/3468781590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d80d1e00 0x7fd9d80d0ec0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.412+0000 7fd9df200640 1 -- 172.21.15.47:0/3468781590 >> 172.21.15.47:0/3468781590 conn(0x7fd9d80ced60 msgr2=0x7fd9d80cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.412+0000 7fd9df200640 1 -- 172.21.15.47:0/3468781590 shutdown_connections 2024-04-20T10:27:51.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.412+0000 7fd9df200640 1 -- 172.21.15.47:0/3468781590 wait complete. 2024-04-20T10:27:51.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.414+0000 7fd9df200640 1 Processor -- start 2024-04-20T10:27:51.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.414+0000 7fd9df200640 1 -- start start 2024-04-20T10:27:51.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9df200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9d80d1400 0x7fd9d814e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9df200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9d80d1e00 0x7fd9d80adae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9df200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d815e680 0x7fd9d814a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9df200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d815fa70 con 0x7fd9d815e680 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9df200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d8162c60 con 0x7fd9d80d1400 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9df200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd9d80cd140 con 0x7fd9d80d1e00 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9de800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d815e680 0x7fd9d814a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9de800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d815e680 0x7fd9d814a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47224/0 (socket says 172.21.15.47:47224) 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9de800640 1 -- 172.21.15.47:0/1341084443 learned_addr learned my addr 172.21.15.47:0/1341084443 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9dde00640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9d80d1400 0x7fd9d814e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.415+0000 7fd9dd400640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9d80d1e00 0x7fd9d80adae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9de800640 1 -- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9d80d1e00 msgr2=0x7fd9d80adae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9de800640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9d80d1e00 0x7fd9d80adae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9de800640 1 -- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9d80d1400 msgr2=0x7fd9d814e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9de800640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9d80d1400 0x7fd9d814e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9de800640 1 -- 172.21.15.47:0/1341084443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d80afc70 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9dd400640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9d80d1e00 0x7fd9d80adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9de800640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d815e680 0x7fd9d814a500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c480 tx=0x7fd9d400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9ce000640 1 -- 172.21.15.47:0/1341084443 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4018070 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9ce000640 1 -- 172.21.15.47:0/1341084443 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd9d4007500 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d80aff50 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9ce000640 1 -- 172.21.15.47:0/1341084443 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9d4004a20 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.416+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d80b0510 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.418+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d8075b30 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.419+0000 7fd9ce000640 1 -- 172.21.15.47:0/1341084443 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd9d401b070 con 0x7fd9d815e680 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.419+0000 7fd9ce000640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd9bc06c8f0 0x7fd9bc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.420+0000 7fd9ce000640 1 -- 172.21.15.47:0/1341084443 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd9d408b1b0 con 0x7fd9d815e680 2024-04-20T10:27:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.420+0000 7fd9dde00640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd9bc06c8f0 0x7fd9bc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.421+0000 7fd9dde00640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd9bc06c8f0 0x7fd9bc06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009f20 tx=0x7fd9d00094d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.424+0000 7fd9ce000640 1 -- 172.21.15.47:0/1341084443 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd9d4059290 con 0x7fd9d815e680 2024-04-20T10:27:51.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.632+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd9d80b54a0 con 0x7fd9d815e680 2024-04-20T10:27:51.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.634+0000 7fd9ce000640 1 -- 172.21.15.47:0/1341084443 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7fd9d4058b90 con 0x7fd9d815e680 2024-04-20T10:27:51.634 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:51.635 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd9bc06c8f0 msgr2=0x7fd9bc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd9bc06c8f0 0x7fd9bc06edb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd9d0009f20 tx=0x7fd9d00094d0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d815e680 msgr2=0x7fd9d814a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d815e680 0x7fd9d814a500 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd9d400c480 tx=0x7fd9d400c940 comp rx=0 tx=0).stop 2024-04-20T10:27:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 shutdown_connections 2024-04-20T10:27:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd9bc06c8f0 0x7fd9bc06edb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd9d815e680 0x7fd9d814a500 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd9d80d1e00 0x7fd9d80adae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 --2- 172.21.15.47:0/1341084443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd9d80d1400 0x7fd9d814e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 >> 172.21.15.47:0/1341084443 conn(0x7fd9d80ced60 msgr2=0x7fd9d81604a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 shutdown_connections 2024-04-20T10:27:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.637+0000 7fd9df200640 1 -- 172.21.15.47:0/1341084443 wait complete. 2024-04-20T10:27:51.657 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.865+0000 7f8b93000640 1 Processor -- start 2024-04-20T10:27:51.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.866+0000 7f8b93000640 1 -- start start 2024-04-20T10:27:51.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.867+0000 7f8b93000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c161d50 0x7f8b8c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.867+0000 7f8b93000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8c06c190 con 0x7f8b8c0b3330 2024-04-20T10:27:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.867+0000 7f8b93000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8b8c06c470 con 0x7f8b8c06cde0 2024-04-20T10:27:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.867+0000 7f8b93000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c162130 con 0x7f8b8c161d50 2024-04-20T10:27:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.868+0000 7f8b91200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8b8c0b3330 0x7f8b8c06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:41640/0 (socket says 172.21.15.47:41640) 2024-04-20T10:27:51.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.868+0000 7f8b91200640 1 -- 172.21.15.47:0/42617569 learned_addr learned my addr 172.21.15.47:0/42617569 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.868+0000 7f8b91c00640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c161d50 0x7f8b8c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.868+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 181890996 0 0) 0x7f8b8c06c190 con 0x7f8b8c0b3330 2024-04-20T10:27:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.868+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b70003520 con 0x7f8b8c0b3330 2024-04-20T10:27:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.868+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3784340783 0 0) 0x7f8b70003520 con 0x7f8b8c0b3330 2024-04-20T10:27:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c161d50 msgr2=0x7f8b8c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c161d50 0x7f8b8c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 >> v1:172.21.15.175:6789/0 conn(0x7f8b8c06cde0 legacy=0x7f8b8c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c0d2e00 con 0x7f8b8c0b3330 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f8b78001690 con 0x7f8b8c0b3330 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 >> v1:172.21.15.47:6789/0 conn(0x7f8b8c0b3330 legacy=0x7f8b8c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700039e0 0x7f8b70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c161d50 0x7f8b700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7000ba70 con 0x7f8b70004800 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b7000bc20 con 0x7f8b700039e0 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b7000be20 con 0x7f8b8c161d50 2024-04-20T10:27:51.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b91200640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b91c00640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700039e0 0x7f8b70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b91200640 1 -- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c161d50 msgr2=0x7f8b700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b91200640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c161d50 0x7f8b700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b91200640 1 -- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700039e0 msgr2=0x7f8b70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:51.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b91200640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b700039e0 0x7f8b70003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.869+0000 7f8b91200640 1 -- 172.21.15.47:0/42617569 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b70003520 con 0x7f8b70004800 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.871+0000 7f8b91200640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b70004c80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0ba2b0 tx=0x7f8b70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d2bbe931143afd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.871+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78007c40 con 0x7f8b70004800 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.871+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b78030070 con 0x7f8b70004800 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.871+0000 7f8b8be00640 1 -- 172.21.15.47:0/42617569 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b78005d10 con 0x7f8b70004800 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.871+0000 7f8b93000640 1 -- 172.21.15.47:0/42617569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 msgr2=0x7f8b70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.871+0000 7f8b93000640 1 --2- 172.21.15.47:0/42617569 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b70004c80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0ba2b0 tx=0x7f8b70005830 comp rx=0 tx=0).stop 2024-04-20T10:27:51.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.871+0000 7f8b92600640 1 -- 172.21.15.47:0/42617569 reap_dead start 2024-04-20T10:27:51.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.872+0000 7f8b93000640 1 -- 172.21.15.47:0/42617569 shutdown_connections 2024-04-20T10:27:51.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.872+0000 7f8b93000640 1 -- 172.21.15.47:0/42617569 >> 172.21.15.47:0/42617569 conn(0x7f8b8c0d4450 msgr2=0x7f8b8c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:51.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.872+0000 7f8b93000640 1 -- 172.21.15.47:0/42617569 shutdown_connections 2024-04-20T10:27:51.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.872+0000 7f8b93000640 1 -- 172.21.15.47:0/42617569 wait complete. 2024-04-20T10:27:51.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.874+0000 7f8b93000640 1 Processor -- start 2024-04-20T10:27:51.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.874+0000 7f8b93000640 1 -- start start 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b93000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b8c0c9f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b93000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b8c0cb520 0x7f8b8c0b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b93000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c0b5f50 0x7f8b8c0c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b93000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c0d38e0 con 0x7f8b70004800 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b93000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b8c0d33c0 con 0x7f8b8c0cb520 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b93000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8c0d3a60 con 0x7f8b8c0b5f50 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b8c0c9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b8c0c9f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47248/0 (socket says 172.21.15.47:47248) 2024-04-20T10:27:51.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 -- 172.21.15.47:0/2528420295 learned_addr learned my addr 172.21.15.47:0/2528420295 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:51.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91200640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b8c0cb520 0x7f8b8c0b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b92600640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c0b5f50 0x7f8b8c0c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 -- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c0b5f50 msgr2=0x7f8b8c0c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c0b5f50 0x7f8b8c0c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 -- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b8c0cb520 msgr2=0x7f8b8c0b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b8c0cb520 0x7f8b8c0b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91c00640 1 -- 172.21.15.47:0/2528420295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b8c0cb9e0 con 0x7f8b70004800 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.875+0000 7f8b91200640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b8c0cb520 0x7f8b8c0b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.876+0000 7f8b91c00640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b8c0c9f10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c010890 tx=0x7f8b7c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.876+0000 7f8b89600640 1 -- 172.21.15.47:0/2528420295 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c00f040 con 0x7f8b70004800 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.876+0000 7f8b89600640 1 -- 172.21.15.47:0/2528420295 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b7c00eb00 con 0x7f8b70004800 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.876+0000 7f8b89600640 1 -- 172.21.15.47:0/2528420295 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b7c01c400 con 0x7f8b70004800 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.876+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b8c0cbcc0 con 0x7f8b70004800 2024-04-20T10:27:51.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.876+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b8c0cc280 con 0x7f8b70004800 2024-04-20T10:27:51.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.878+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b8c0b6360 con 0x7f8b70004800 2024-04-20T10:27:51.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.878+0000 7f8b89600640 1 -- 172.21.15.47:0/2528420295 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b7c01c5a0 con 0x7f8b70004800 2024-04-20T10:27:51.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.879+0000 7f8b89600640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:51.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.880+0000 7f8b89600640 1 -- 172.21.15.47:0/2528420295 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8b7c08c3d0 con 0x7f8b70004800 2024-04-20T10:27:51.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.880+0000 7f8b91200640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:51.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.881+0000 7f8b91200640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b78034000 tx=0x7f8b78007000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:51.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:51.884+0000 7f8b89600640 1 -- 172.21.15.47:0/2528420295 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8b8c0b6360 con 0x7f8b70004800 2024-04-20T10:27:52.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.099+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8b8c15e4d0 con 0x7f8b70004800 2024-04-20T10:27:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.102+0000 7f8b89600640 1 -- 172.21.15.47:0/2528420295 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7f8b7c018030 con 0x7f8b70004800 2024-04-20T10:27:52.102 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:52.103 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b6c06c8f0 msgr2=0x7f8b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8b78034000 tx=0x7f8b78007000 comp rx=0 tx=0).stop 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 msgr2=0x7f8b8c0c9f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b8c0c9f10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8b7c010890 tx=0x7f8b7c010d50 comp rx=0 tx=0).stop 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 shutdown_connections 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b8c0b5f50 0x7f8b8c0c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b8c0cb520 0x7f8b8c0b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b6c06c8f0 0x7f8b6c06edb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 --2- 172.21.15.47:0/2528420295 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b70004800 0x7f8b8c0c9f10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 >> 172.21.15.47:0/2528420295 conn(0x7f8b8c0d4450 msgr2=0x7f8b8c160e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 shutdown_connections 2024-04-20T10:27:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.105+0000 7f8b93000640 1 -- 172.21.15.47:0/2528420295 wait complete. 2024-04-20T10:27:52.126 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T10:27:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.342+0000 7fdf3fc00640 1 Processor -- start 2024-04-20T10:27:52.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.342+0000 7fdf3fc00640 1 -- start start 2024-04-20T10:27:52.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.343+0000 7fdf3fc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdf380d4270 0x7fdf380d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.343+0000 7fdf3fc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf380d4650 0x7fdf380b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf3fc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380b17d0 0x7fdf380d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf3fc00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdf380d40c0 con 0x7fdf380d4650 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf3fc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdf380d3820 con 0x7fdf380d4270 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf3fc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf380d3b00 con 0x7fdf380b17d0 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf37e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf380d4650 0x7fdf380b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf37e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdf380d4650 0x7fdf380b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47250/0 (socket says 172.21.15.47:47250) 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf37e00640 1 -- 172.21.15.47:0/850774739 learned_addr learned my addr 172.21.15.47:0/850774739 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf3d200640 1 --2- 172.21.15.47:0/850774739 >> v2:172.21.15.175:3300/0 conn(0x7fdf380d4270 0x7fdf380d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.344+0000 7fdf3dc00640 1 --2- 172.21.15.47:0/850774739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380b17d0 0x7fdf380d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.345+0000 7fdf37e00640 1 -- 172.21.15.47:0/850774739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380b17d0 msgr2=0x7fdf380d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.345+0000 7fdf37e00640 1 --2- 172.21.15.47:0/850774739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380b17d0 0x7fdf380d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.345+0000 7fdf37e00640 1 -- 172.21.15.47:0/850774739 >> v2:172.21.15.175:3300/0 conn(0x7fdf380d4270 msgr2=0x7fdf380d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:52.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.345+0000 7fdf37e00640 1 --2- 172.21.15.47:0/850774739 >> v2:172.21.15.175:3300/0 conn(0x7fdf380d4270 0x7fdf380d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.345+0000 7fdf37e00640 1 -- 172.21.15.47:0/850774739 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf380d3de0 con 0x7fdf380d4650 2024-04-20T10:27:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.345+0000 7fdf3d200640 1 --2- 172.21.15.47:0/850774739 >> v2:172.21.15.175:3300/0 conn(0x7fdf380d4270 0x7fdf380d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.346+0000 7fdf37e00640 1 --2- 172.21.15.47:0/850774739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380d4650 0x7fdf380b1290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf3808e820 tx=0x7fdf2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558f9a4b37789ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.346+0000 7fdf37400640 1 -- 172.21.15.47:0/850774739 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24035020 con 0x7fdf380d4650 2024-04-20T10:27:52.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.346+0000 7fdf37400640 1 -- 172.21.15.47:0/850774739 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf24031870 con 0x7fdf380d4650 2024-04-20T10:27:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.346+0000 7fdf37400640 1 -- 172.21.15.47:0/850774739 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf24031c40 con 0x7fdf380d4650 2024-04-20T10:27:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 -- 172.21.15.47:0/850774739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380d4650 msgr2=0x7fdf380b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:52.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/850774739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380d4650 0x7fdf380b1290 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdf3808e820 tx=0x7fdf2402b910 comp rx=0 tx=0).stop 2024-04-20T10:27:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 -- 172.21.15.47:0/850774739 shutdown_connections 2024-04-20T10:27:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/850774739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380b17d0 0x7fdf380d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/850774739 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380d4650 0x7fdf380b1290 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/850774739 >> v2:172.21.15.175:3300/0 conn(0x7fdf380d4270 0x7fdf380d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 -- 172.21.15.47:0/850774739 >> 172.21.15.47:0/850774739 conn(0x7fdf380b99c0 msgr2=0x7fdf380b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 -- 172.21.15.47:0/850774739 shutdown_connections 2024-04-20T10:27:52.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.347+0000 7fdf3fc00640 1 -- 172.21.15.47:0/850774739 wait complete. 2024-04-20T10:27:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.349+0000 7fdf3fc00640 1 Processor -- start 2024-04-20T10:27:52.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.349+0000 7fdf3fc00640 1 -- start start 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3fc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380b17d0 0x7fdf380ce480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3fc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf380d4270 0x7fdf380b5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3fc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380d4650 0x7fdf38159bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3fc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdf380d4ae0 con 0x7fdf380b17d0 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3fc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf380d53b0 con 0x7fdf380d4270 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3fc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf380d4c60 con 0x7fdf380d4650 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380b17d0 0x7fdf380ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:52.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380b17d0 0x7fdf380ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47262/0 (socket says 172.21.15.47:47262) 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.350+0000 7fdf3d200640 1 -- 172.21.15.47:0/523681867 learned_addr learned my addr 172.21.15.47:0/523681867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf37e00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf380d4270 0x7fdf380b5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3dc00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380d4650 0x7fdf38159bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3d200640 1 -- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380d4650 msgr2=0x7fdf38159bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3d200640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380d4650 0x7fdf38159bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3d200640 1 -- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf380d4270 msgr2=0x7fdf380b5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3d200640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf380d4270 0x7fdf380b5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3d200640 1 -- 172.21.15.47:0/523681867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf380c9ae0 con 0x7fdf380b17d0 2024-04-20T10:27:52.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3d200640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380b17d0 0x7fdf380ce480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c820 tx=0x7fdf2c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf34c00640 1 -- 172.21.15.47:0/523681867 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c015070 con 0x7fdf380b17d0 2024-04-20T10:27:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf34c00640 1 -- 172.21.15.47:0/523681867 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdf2c007a30 con 0x7fdf380b17d0 2024-04-20T10:27:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.351+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf380c9d80 con 0x7fdf380b17d0 2024-04-20T10:27:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.352+0000 7fdf34c00640 1 -- 172.21.15.47:0/523681867 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c007de0 con 0x7fdf380b17d0 2024-04-20T10:27:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.352+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf380ca340 con 0x7fdf380b17d0 2024-04-20T10:27:52.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.353+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf38068980 con 0x7fdf380b17d0 2024-04-20T10:27:52.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.354+0000 7fdf34c00640 1 -- 172.21.15.47:0/523681867 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdf2c018070 con 0x7fdf380b17d0 2024-04-20T10:27:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.355+0000 7fdf34c00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdf1806c8f0 0x7fdf1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.355+0000 7fdf34c00640 1 -- 172.21.15.47:0/523681867 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fdf2c08bd60 con 0x7fdf380b17d0 2024-04-20T10:27:52.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.356+0000 7fdf37e00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdf1806c8f0 0x7fdf1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:52.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.357+0000 7fdf37e00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdf1806c8f0 0x7fdf1806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf24036000 tx=0x7fdf24008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:52.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.360+0000 7fdf34c00640 1 -- 172.21.15.47:0/523681867 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdf2c059db0 con 0x7fdf380b17d0 2024-04-20T10:27:52.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:52.602+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fdf380b3c80 con 0x7fdf380b17d0 2024-04-20T10:27:53.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.179+0000 7fdf34c00640 1 -- 172.21.15.47:0/523681867 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fdf2c0596b0 con 0x7fdf380b17d0 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdf1806c8f0 msgr2=0x7fdf1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdf1806c8f0 0x7fdf1806edb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fdf24036000 tx=0x7fdf24008030 comp rx=0 tx=0).stop 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380b17d0 msgr2=0x7fdf380ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380b17d0 0x7fdf380ce480 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c00c820 tx=0x7fdf2c00cce0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 shutdown_connections 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf380d4650 0x7fdf38159bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf380d4270 0x7fdf380b5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdf1806c8f0 0x7fdf1806edb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 --2- 172.21.15.47:0/523681867 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdf380b17d0 0x7fdf380ce480 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.182+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 >> 172.21.15.47:0/523681867 conn(0x7fdf380b99c0 msgr2=0x7fdf380b2d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.183+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 shutdown_connections 2024-04-20T10:27:53.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.183+0000 7fdf3fc00640 1 -- 172.21.15.47:0/523681867 wait complete. 2024-04-20T10:27:53.205 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-20T10:27:53.205 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T10:27:53.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.415+0000 7f7691200640 1 Processor -- start 2024-04-20T10:27:53.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.415+0000 7f7691200640 1 -- start start 2024-04-20T10:27:53.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.417+0000 7f7691200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c162700 0x7f768c162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:53.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.417+0000 7f7691200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f768c0b6c20 0x7f768c15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:53.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.417+0000 7f7691200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768c06f2b0 con 0x7f768c0b3510 2024-04-20T10:27:53.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.417+0000 7f7691200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f768c0b6900 con 0x7f768c0b6c20 2024-04-20T10:27:53.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.417+0000 7f7691200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f768c0b6620 con 0x7f768c162700 2024-04-20T10:27:53.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.418+0000 7f7689e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c162700 0x7f768c162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:53.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.418+0000 7f7689e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c162700 0x7f768c162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:48724/0 (socket says 172.21.15.47:48724) 2024-04-20T10:27:53.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.418+0000 7f7689e00640 1 -- 172.21.15.47:0/2634059642 learned_addr learned my addr 172.21.15.47:0/2634059642 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:53.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.418+0000 7f7689400640 1 --2- 172.21.15.47:0/2634059642 >> v2:172.21.15.175:3300/0 conn(0x7f768c0b6c20 0x7f768c15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:53.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.419+0000 7f7689e00640 1 -- 172.21.15.47:0/2634059642 >> v2:172.21.15.175:3300/0 conn(0x7f768c0b6c20 msgr2=0x7f768c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:53.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.419+0000 7f7689e00640 1 --2- 172.21.15.47:0/2634059642 >> v2:172.21.15.175:3300/0 conn(0x7f768c0b6c20 0x7f768c15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.419+0000 7f7689e00640 1 -- 172.21.15.47:0/2634059642 >> v1:172.21.15.175:6789/0 conn(0x7f768c0b3510 legacy=0x7f768c0d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:53.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.419+0000 7f7689e00640 1 -- 172.21.15.47:0/2634059642 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c0b6160 con 0x7f768c162700 2024-04-20T10:27:53.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.420+0000 7f7689e00640 1 --2- 172.21.15.47:0/2634059642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c162700 0x7f768c162ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f768c0b67a0 tx=0x7f767c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f5fdd513ff6ca23e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:53.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.420+0000 7f7688a00640 1 -- 172.21.15.47:0/2634059642 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c0317c0 con 0x7f768c162700 2024-04-20T10:27:53.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.420+0000 7f7688a00640 1 -- 172.21.15.47:0/2634059642 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c037070 con 0x7f768c162700 2024-04-20T10:27:53.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.420+0000 7f7688a00640 1 -- 172.21.15.47:0/2634059642 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f767c031b80 con 0x7f768c162700 2024-04-20T10:27:53.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 -- 172.21.15.47:0/2634059642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c162700 msgr2=0x7f768c162ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:53.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 --2- 172.21.15.47:0/2634059642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c162700 0x7f768c162ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f768c0b67a0 tx=0x7f767c02b860 comp rx=0 tx=0).stop 2024-04-20T10:27:53.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 -- 172.21.15.47:0/2634059642 shutdown_connections 2024-04-20T10:27:53.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 --2- 172.21.15.47:0/2634059642 >> v2:172.21.15.175:3300/0 conn(0x7f768c0b6c20 0x7f768c15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 --2- 172.21.15.47:0/2634059642 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c162700 0x7f768c162ae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 -- 172.21.15.47:0/2634059642 >> 172.21.15.47:0/2634059642 conn(0x7f768c09d910 msgr2=0x7f768c0b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:53.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 -- 172.21.15.47:0/2634059642 shutdown_connections 2024-04-20T10:27:53.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.421+0000 7f7691200640 1 -- 172.21.15.47:0/2634059642 wait complete. 2024-04-20T10:27:53.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.423+0000 7f7691200640 1 Processor -- start 2024-04-20T10:27:53.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.423+0000 7f7691200640 1 -- start start 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f7691200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c0b3510 0x7f768c150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f7691200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f768c0b6c20 0x7f768c0cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f7691200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f768c162700 0x7f768c0cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f7691200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f768c0d08a0 con 0x7f768c162700 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f7691200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f768c0d0720 con 0x7f768c0b6c20 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f7691200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f768c0d0a20 con 0x7f768c0b3510 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f768a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f768c162700 0x7f768c0cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f768a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f768c162700 0x7f768c0cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:47270/0 (socket says 172.21.15.47:47270) 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.424+0000 7f768a800640 1 -- 172.21.15.47:0/4071348476 learned_addr learned my addr 172.21.15.47:0/4071348476 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:53.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f7689e00640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c0b3510 0x7f768c150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:53.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f7689400640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f768c0b6c20 0x7f768c0cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:53.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f768a800640 1 -- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c0b3510 msgr2=0x7f768c150910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:53.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f768a800640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c0b3510 0x7f768c150910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f768a800640 1 -- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f768c0b6c20 msgr2=0x7f768c0cbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:53.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f768a800640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f768c0b6c20 0x7f768c0cbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:53.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f768a800640 1 -- 172.21.15.47:0/4071348476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768c0cca70 con 0x7f768c162700 2024-04-20T10:27:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.425+0000 7f768a800640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f768c162700 0x7f768c0cc530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f768400c7a0 tx=0x7f768400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.426+0000 7f7682000640 1 -- 172.21.15.47:0/4071348476 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7684003b70 con 0x7f768c162700 2024-04-20T10:27:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.426+0000 7f7682000640 1 -- 172.21.15.47:0/4071348476 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7684003d30 con 0x7f768c162700 2024-04-20T10:27:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.426+0000 7f7682000640 1 -- 172.21.15.47:0/4071348476 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f768400aa90 con 0x7f768c162700 2024-04-20T10:27:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.426+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f768c0ccd10 con 0x7f768c162700 2024-04-20T10:27:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.426+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f768c0cd2d0 con 0x7f768c162700 2024-04-20T10:27:53.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.427+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f768c068980 con 0x7f768c162700 2024-04-20T10:27:53.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.428+0000 7f7682000640 1 -- 172.21.15.47:0/4071348476 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f768400ac30 con 0x7f768c162700 2024-04-20T10:27:53.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.429+0000 7f7682000640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f766c06c920 0x7f766c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:53.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.430+0000 7f7682000640 1 -- 172.21.15.47:0/4071348476 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f768408be30 con 0x7f768c162700 2024-04-20T10:27:53.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.430+0000 7f7689e00640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f766c06c920 0x7f766c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:53.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.431+0000 7f7689e00640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f766c06c920 0x7f766c06ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f767c036000 tx=0x7f767c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:53.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.434+0000 7f7682000640 1 -- 172.21.15.47:0/4071348476 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7684059e80 con 0x7f768c162700 2024-04-20T10:27:53.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:53.682+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f768c069270 con 0x7f768c162700 2024-04-20T10:27:54.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.182+0000 7f7682000640 1 -- 172.21.15.47:0/4071348476 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f7684059780 con 0x7f768c162700 2024-04-20T10:27:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.185+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f766c06c920 msgr2=0x7f766c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f766c06c920 0x7f766c06ede0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f767c036000 tx=0x7f767c03d040 comp rx=0 tx=0).stop 2024-04-20T10:27:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f768c162700 msgr2=0x7f768c0cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f768c162700 0x7f768c0cc530 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f768400c7a0 tx=0x7f768400cc60 comp rx=0 tx=0).stop 2024-04-20T10:27:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 shutdown_connections 2024-04-20T10:27:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f766c06c920 0x7f766c06ede0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f768c162700 0x7f768c0cc530 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f768c0b6c20 0x7f768c0cbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 --2- 172.21.15.47:0/4071348476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f768c0b3510 0x7f768c150910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 >> 172.21.15.47:0/4071348476 conn(0x7f768c09d910 msgr2=0x7f768c0d0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 shutdown_connections 2024-04-20T10:27:54.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.186+0000 7f7691200640 1 -- 172.21.15.47:0/4071348476 wait complete. 2024-04-20T10:27:54.212 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T10:27:54.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:27:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.414+0000 7fb1bf200640 1 Processor -- start 2024-04-20T10:27:54.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.415+0000 7fb1bf200640 1 -- start start 2024-04-20T10:27:54.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.416+0000 7fb1bf200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb1c0163b60 0x7fb1c0071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.416+0000 7fb1bf200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c006c310 con 0x7fb1c0163070 2024-04-20T10:27:54.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.416+0000 7fb1bf200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb1c006c5f0 con 0x7fb1c01651d0 2024-04-20T10:27:54.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.416+0000 7fb1bf200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb1c0073cf0 con 0x7fb1c0163b60 2024-04-20T10:27:54.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.417+0000 7fb1bde00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb1c0163070 0x7fb1c00735e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33580/0 (socket says 172.21.15.47:33580) 2024-04-20T10:27:54.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.417+0000 7fb1bde00640 1 -- 172.21.15.47:0/196405278 learned_addr learned my addr 172.21.15.47:0/196405278 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.417+0000 7fb1be800640 1 --2- 172.21.15.47:0/196405278 >> v2:172.21.15.175:3300/0 conn(0x7fb1c0163b60 0x7fb1c0071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.417+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2310376738 0 0) 0x7fb1c006c310 con 0x7fb1c0163070 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.417+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb1a4003520 con 0x7fb1c0163070 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3085791341 0 0) 0x7fb1a4003520 con 0x7fb1c0163070 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 >> v2:172.21.15.175:3300/0 conn(0x7fb1c0163b60 msgr2=0x7fb1c0071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 --2- 172.21.15.47:0/196405278 >> v2:172.21.15.175:3300/0 conn(0x7fb1c0163b60 0x7fb1c0071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 >> v1:172.21.15.175:6789/0 conn(0x7fb1c01651d0 legacy=0x7fb1c0163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c006c8d0 con 0x7fb1c0163070 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fb1b0001670 con 0x7fb1c0163070 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 >> v1:172.21.15.47:6789/0 conn(0x7fb1c0163070 legacy=0x7fb1c00735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a4003a00 0x7fb1a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a4004800 0x7fb1a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c0073cf0 con 0x7fb1a4005210 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a400c160 con 0x7fb1a4003a00 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a400c360 con 0x7fb1a4004800 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1be800640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.418+0000 7fb1bde00640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a4003a00 0x7fb1a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.419+0000 7fb1b5400640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a4004800 0x7fb1a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.419+0000 7fb1be800640 1 -- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a4004800 msgr2=0x7fb1a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.419+0000 7fb1be800640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1a4004800 0x7fb1a4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.419+0000 7fb1be800640 1 -- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a4003a00 msgr2=0x7fb1a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.419+0000 7fb1be800640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1a4003a00 0x7fb1a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.419+0000 7fb1be800640 1 -- 172.21.15.47:0/196405278 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a4003520 con 0x7fb1a4005210 2024-04-20T10:27:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.420+0000 7fb1be800640 1 -- 172.21.15.47:0/196405278 reap_dead start 2024-04-20T10:27:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.420+0000 7fb1be800640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1a400b9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009f90 tx=0x7fb1a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30680bcf3c566fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.420+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8009c10 con 0x7fb1a4005210 2024-04-20T10:27:54.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.420+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b802cbf0 con 0x7fb1a4005210 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.420+0000 7fb1bd400640 1 -- 172.21.15.47:0/196405278 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8003990 con 0x7fb1a4005210 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.421+0000 7fb1bf200640 1 -- 172.21.15.47:0/196405278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 msgr2=0x7fb1a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.421+0000 7fb1bf200640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1a400b9c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8009f90 tx=0x7fb1a4005830 comp rx=0 tx=0).stop 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.421+0000 7fb1bf200640 1 -- 172.21.15.47:0/196405278 shutdown_connections 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.421+0000 7fb1bf200640 1 --2- 172.21.15.47:0/196405278 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1a400b9c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.421+0000 7fb1bf200640 1 -- 172.21.15.47:0/196405278 >> 172.21.15.47:0/196405278 conn(0x7fb1c00d0040 msgr2=0x7fb1c00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.421+0000 7fb1bf200640 1 -- 172.21.15.47:0/196405278 shutdown_connections 2024-04-20T10:27:54.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.421+0000 7fb1bf200640 1 -- 172.21.15.47:0/196405278 wait complete. 2024-04-20T10:27:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.423+0000 7fb1bf200640 1 Processor -- start 2024-04-20T10:27:54.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.423+0000 7fb1bf200640 1 -- start start 2024-04-20T10:27:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1c015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c0154ad0 0x7fb1c00b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1c00b4fb0 0x7fb1c01598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bf200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c01662a0 con 0x7fb1a4005210 2024-04-20T10:27:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c006ce90 con 0x7fb1c00b4fb0 2024-04-20T10:27:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1c0071d90 con 0x7fb1c0154ad0 2024-04-20T10:27:54.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1c015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1c015f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55722/0 (socket says 172.21.15.47:55722) 2024-04-20T10:27:54.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.424+0000 7fb1bde00640 1 -- 172.21.15.47:0/2259875486 learned_addr learned my addr 172.21.15.47:0/2259875486 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1be800640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1c00b4fb0 0x7fb1c01598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bde00640 1 -- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c0154ad0 msgr2=0x7fb1c00b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bde00640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c0154ad0 0x7fb1c00b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bde00640 1 -- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1c00b4fb0 msgr2=0x7fb1c01598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bde00640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1c00b4fb0 0x7fb1c01598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bde00640 1 -- 172.21.15.47:0/2259875486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c01fbfc0 con 0x7fb1a4005210 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1be800640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1c00b4fb0 0x7fb1c01598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:54.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bde00640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1c015f040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000e7b0 tx=0x7fb1b000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1b6a00640 1 -- 172.21.15.47:0/2259875486 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000c660 con 0x7fb1a4005210 2024-04-20T10:27:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1b6a00640 1 -- 172.21.15.47:0/2259875486 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb1b0018070 con 0x7fb1a4005210 2024-04-20T10:27:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1b6a00640 1 -- 172.21.15.47:0/2259875486 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1b000ca20 con 0x7fb1a4005210 2024-04-20T10:27:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c009cb40 con 0x7fb1a4005210 2024-04-20T10:27:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.425+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c009d100 con 0x7fb1a4005210 2024-04-20T10:27:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.427+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c0075b60 con 0x7fb1a4005210 2024-04-20T10:27:54.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.428+0000 7fb1b6a00640 1 -- 172.21.15.47:0/2259875486 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb1b000cbc0 con 0x7fb1a4005210 2024-04-20T10:27:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.428+0000 7fb1b6a00640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb19c06c920 0x7fb19c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.429+0000 7fb1b6a00640 1 -- 172.21.15.47:0/2259875486 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb1b008b980 con 0x7fb1a4005210 2024-04-20T10:27:54.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.429+0000 7fb1b5400640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb19c06c920 0x7fb19c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.430+0000 7fb1b5400640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb19c06c920 0x7fb19c06ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8000c00 tx=0x7fb1a8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:54.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.433+0000 7fb1b6a00640 1 -- 172.21.15.47:0/2259875486 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb1b00599d0 con 0x7fb1a4005210 2024-04-20T10:27:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.683+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb1c00b10e0 con 0x7fb1a4005210 2024-04-20T10:27:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.684+0000 7fb1b6a00640 1 -- 172.21.15.47:0/2259875486 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7fb1b00592d0 con 0x7fb1a4005210 2024-04-20T10:27:54.685 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:54.685 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:27:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:27:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb19c06c920 msgr2=0x7fb19c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb19c06c920 0x7fb19c06ede0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8000c00 tx=0x7fb1a8009040 comp rx=0 tx=0).stop 2024-04-20T10:27:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 msgr2=0x7fb1c015f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1c015f040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fb1b000e7b0 tx=0x7fb1b000ec70 comp rx=0 tx=0).stop 2024-04-20T10:27:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 shutdown_connections 2024-04-20T10:27:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1c00b4fb0 0x7fb1c01598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1c0154ad0 0x7fb1c00b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb19c06c920 0x7fb19c06ede0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 --2- 172.21.15.47:0/2259875486 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1a4005210 0x7fb1c015f040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 >> 172.21.15.47:0/2259875486 conn(0x7fb1c00d0040 msgr2=0x7fb1c00afd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 shutdown_connections 2024-04-20T10:27:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.687+0000 7fb1bf200640 1 -- 172.21.15.47:0/2259875486 wait complete. 2024-04-20T10:27:54.705 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.909+0000 7f31f3200640 1 Processor -- start 2024-04-20T10:27:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.909+0000 7f31f3200640 1 -- start start 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.910+0000 7f31f3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40b2f70 0x7f31f40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.910+0000 7f31f3200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31f406b900 0x7f31f406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.910+0000 7f31f3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31f4071a80 con 0x7f31f40adf60 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.910+0000 7f31f3200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f31f406c270 con 0x7f31f406b900 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.910+0000 7f31f3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31f40ae3c0 con 0x7f31f40b2f70 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.911+0000 7f31f2800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31f406b900 0x7f31f406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.911+0000 7f31f1e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40b2f70 0x7f31f40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.911+0000 7f31f2800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31f406b900 0x7f31f406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40592/0 (socket says 172.21.15.47:40592) 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.911+0000 7f31f2800640 1 -- 172.21.15.47:0/499876335 learned_addr learned my addr 172.21.15.47:0/499876335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.911+0000 7f31f2800640 1 -- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40b2f70 msgr2=0x7f31f40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.911+0000 7f31f2800640 1 --2- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40b2f70 0x7f31f40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.912+0000 7f31f2800640 1 -- 172.21.15.47:0/499876335 >> v1:172.21.15.175:6789/0 conn(0x7f31f40adf60 legacy=0x7f31f415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.912+0000 7f31f2800640 1 -- 172.21.15.47:0/499876335 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f4071cf0 con 0x7f31f406b900 2024-04-20T10:27:54.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.912+0000 7f31f1e00640 1 --2- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40b2f70 0x7f31f40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.913+0000 7f31f2800640 1 --2- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f406b900 0x7f31f406bd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009e50 tx=0x7f31ec02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c82c3c877fc6337c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.913+0000 7f31f1400640 1 -- 172.21.15.47:0/499876335 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec02cb00 con 0x7f31f406b900 2024-04-20T10:27:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f1400640 1 -- 172.21.15.47:0/499876335 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31ec02cca0 con 0x7f31f406b900 2024-04-20T10:27:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f1400640 1 -- 172.21.15.47:0/499876335 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31ec034610 con 0x7f31f406b900 2024-04-20T10:27:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 -- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f406b900 msgr2=0x7f31f406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 --2- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f406b900 0x7f31f406bd30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f31ec009e50 tx=0x7f31ec02bf60 comp rx=0 tx=0).stop 2024-04-20T10:27:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 -- 172.21.15.47:0/499876335 shutdown_connections 2024-04-20T10:27:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 --2- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f406b900 0x7f31f406bd30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 --2- 172.21.15.47:0/499876335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40b2f70 0x7f31f40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 -- 172.21.15.47:0/499876335 >> 172.21.15.47:0/499876335 conn(0x7f31f40d0040 msgr2=0x7f31f40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 -- 172.21.15.47:0/499876335 shutdown_connections 2024-04-20T10:27:54.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.914+0000 7f31f3200640 1 -- 172.21.15.47:0/499876335 wait complete. 2024-04-20T10:27:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.916+0000 7f31f3200640 1 Processor -- start 2024-04-20T10:27:54.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.916+0000 7f31f3200640 1 -- start start 2024-04-20T10:27:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.917+0000 7f31f3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f406b900 0x7f31f4152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.917+0000 7f31f3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40adf60 0x7f31f409d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.917+0000 7f31f3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40b2f70 0x7f31f415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.917+0000 7f31f3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31f40736c0 con 0x7f31f406b900 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.917+0000 7f31f3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31f40739c0 con 0x7f31f40b2f70 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.917+0000 7f31f3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31f415fcc0 con 0x7f31f40adf60 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f406b900 0x7f31f4152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f406b900 0x7f31f4152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55734/0 (socket says 172.21.15.47:55734) 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 -- 172.21.15.47:0/879914350 learned_addr learned my addr 172.21.15.47:0/879914350 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31e9400640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40adf60 0x7f31f409d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f2800640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40b2f70 0x7f31f415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 -- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40adf60 msgr2=0x7f31f409d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40adf60 0x7f31f409d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 -- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40b2f70 msgr2=0x7f31f415e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40b2f70 0x7f31f415e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 -- 172.21.15.47:0/879914350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31f409be10 con 0x7f31f406b900 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f2800640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40b2f70 0x7f31f415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:54.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.918+0000 7f31f1e00640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f406b900 0x7f31f4152730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e960 tx=0x7f31e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.919+0000 7f31eaa00640 1 -- 172.21.15.47:0/879914350 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e4017070 con 0x7f31f406b900 2024-04-20T10:27:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.919+0000 7f31eaa00640 1 -- 172.21.15.47:0/879914350 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31e400ca00 con 0x7f31f406b900 2024-04-20T10:27:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.919+0000 7f31eaa00640 1 -- 172.21.15.47:0/879914350 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31e400cdd0 con 0x7f31f406b900 2024-04-20T10:27:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.919+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31f409c0f0 con 0x7f31f406b900 2024-04-20T10:27:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.919+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31f4152c70 con 0x7f31f406b900 2024-04-20T10:27:54.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.920+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31f4075b60 con 0x7f31f406b900 2024-04-20T10:27:54.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.921+0000 7f31eaa00640 1 -- 172.21.15.47:0/879914350 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31e401a070 con 0x7f31f406b900 2024-04-20T10:27:54.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.922+0000 7f31eaa00640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f31d006c8f0 0x7f31d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:54.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.922+0000 7f31eaa00640 1 -- 172.21.15.47:0/879914350 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f31e408c1a0 con 0x7f31f406b900 2024-04-20T10:27:54.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.923+0000 7f31e9400640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f31d006c8f0 0x7f31d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:54.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.924+0000 7f31e9400640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f31d006c8f0 0x7f31d006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31f41594b0 tx=0x7f31dc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:54.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:54.927+0000 7f31eaa00640 1 -- 172.21.15.47:0/879914350 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31e405a1f0 con 0x7f31f406b900 2024-04-20T10:27:55.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.134+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31f406d220 con 0x7f31f406b900 2024-04-20T10:27:55.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.137+0000 7f31eaa00640 1 -- 172.21.15.47:0/879914350 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7f31e4059af0 con 0x7f31f406b900 2024-04-20T10:27:55.138 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:55.138 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f31d006c8f0 msgr2=0x7f31d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f31d006c8f0 0x7f31d006edb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f31f41594b0 tx=0x7f31dc009040 comp rx=0 tx=0).stop 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f406b900 msgr2=0x7f31f4152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f406b900 0x7f31f4152730 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f31e400e960 tx=0x7f31e400ee20 comp rx=0 tx=0).stop 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 shutdown_connections 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31f40b2f70 0x7f31f415e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31f40adf60 0x7f31f409d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f31d006c8f0 0x7f31d006edb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 --2- 172.21.15.47:0/879914350 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31f406b900 0x7f31f4152730 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.140+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 >> 172.21.15.47:0/879914350 conn(0x7f31f40d0040 msgr2=0x7f31f40b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:55.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.141+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 shutdown_connections 2024-04-20T10:27:55.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.141+0000 7f31f3200640 1 -- 172.21.15.47:0/879914350 wait complete. 2024-04-20T10:27:55.160 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T10:27:55.160 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:27:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.369+0000 7f06f6a00640 1 Processor -- start 2024-04-20T10:27:55.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.369+0000 7f06f6a00640 1 -- start start 2024-04-20T10:27:55.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.371+0000 7f06f6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f06f0159e10 0x7f06f0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.371+0000 7f06f6a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f0160e50 con 0x7f06f006da90 2024-04-20T10:27:55.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.371+0000 7f06f6a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f06f015cc90 con 0x7f06f015a1f0 2024-04-20T10:27:55.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.371+0000 7f06f6a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f06f0072900 con 0x7f06f0159e10 2024-04-20T10:27:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.371+0000 7f06f6000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f06f006da90 0x7f06f0160740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33590/0 (socket says 172.21.15.47:33590) 2024-04-20T10:27:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.371+0000 7f06f6000640 1 -- 172.21.15.47:0/1706384388 learned_addr learned my addr 172.21.15.47:0/1706384388 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.371+0000 7f06f5600640 1 --2- 172.21.15.47:0/1706384388 >> v2:172.21.15.47:3300/0 conn(0x7f06f0159e10 0x7f06f0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.372+0000 7f06f5600640 1 -- 172.21.15.47:0/1706384388 >> v1:172.21.15.175:6789/0 conn(0x7f06f015a1f0 legacy=0x7f06f0160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.372+0000 7f06f5600640 1 -- 172.21.15.47:0/1706384388 >> v1:172.21.15.47:6789/0 conn(0x7f06f006da90 legacy=0x7f06f0160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.372+0000 7f06f5600640 1 -- 172.21.15.47:0/1706384388 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f015cf70 con 0x7f06f0159e10 2024-04-20T10:27:55.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.372+0000 7f06efe00640 1 -- 172.21.15.47:0/1706384388 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3515038343 0 0) 0x7f06f0160e50 con 0x7f06f006da90 2024-04-20T10:27:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.373+0000 7f06f5600640 1 --2- 172.21.15.47:0/1706384388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f0071a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06e40097a0 tx=0x7f06e402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ad5346cdb7bb4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.373+0000 7f06efe00640 1 -- 172.21.15.47:0/1706384388 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4035020 con 0x7f06f0159e10 2024-04-20T10:27:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.373+0000 7f06efe00640 1 -- 172.21.15.47:0/1706384388 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06e4031830 con 0x7f06f0159e10 2024-04-20T10:27:55.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.373+0000 7f06efe00640 1 -- 172.21.15.47:0/1706384388 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06e4031c00 con 0x7f06f0159e10 2024-04-20T10:27:55.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.374+0000 7f06f6a00640 1 -- 172.21.15.47:0/1706384388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 msgr2=0x7f06f0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.374+0000 7f06f6a00640 1 --2- 172.21.15.47:0/1706384388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f0071a80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f06e40097a0 tx=0x7f06e402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.374+0000 7f06f6a00640 1 -- 172.21.15.47:0/1706384388 shutdown_connections 2024-04-20T10:27:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.374+0000 7f06f6a00640 1 --2- 172.21.15.47:0/1706384388 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f0071a80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.374+0000 7f06f6a00640 1 -- 172.21.15.47:0/1706384388 >> 172.21.15.47:0/1706384388 conn(0x7f06f00d0040 msgr2=0x7f06f00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.374+0000 7f06f6a00640 1 -- 172.21.15.47:0/1706384388 shutdown_connections 2024-04-20T10:27:55.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.374+0000 7f06f6a00640 1 -- 172.21.15.47:0/1706384388 wait complete. 2024-04-20T10:27:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.376+0000 7f06f6a00640 1 Processor -- start 2024-04-20T10:27:55.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.376+0000 7f06f6a00640 1 -- start start 2024-04-20T10:27:55.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f006da90 0x7f06f00ca420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f00b4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06f015a1f0 0x7f06f00b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f6a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f06f015d530 con 0x7f06f0159e10 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f6a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f06f015ca40 con 0x7f06f015a1f0 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f6a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06f015d810 con 0x7f06f006da90 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f00b4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f00b4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55754/0 (socket says 172.21.15.47:55754) 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f4c00640 1 -- 172.21.15.47:0/3757414896 learned_addr learned my addr 172.21.15.47:0/3757414896 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f6000640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06f015a1f0 0x7f06f00b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.377+0000 7f06f5600640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f006da90 0x7f06f00ca420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f4c00640 1 -- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f006da90 msgr2=0x7f06f00ca420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f4c00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f006da90 0x7f06f00ca420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f4c00640 1 -- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06f015a1f0 msgr2=0x7f06f00b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f4c00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06f015a1f0 0x7f06f00b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f4c00640 1 -- 172.21.15.47:0/3757414896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06f01f7770 con 0x7f06f0159e10 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f6000640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06f015a1f0 0x7f06f00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f5600640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f006da90 0x7f06f00ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f4c00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f00b4270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00b3c0 tx=0x7f06dc00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06ed600640 1 -- 172.21.15.47:0/3757414896 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc011830 con 0x7f06f0159e10 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06ed600640 1 -- 172.21.15.47:0/3757414896 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06dc016070 con 0x7f06f0159e10 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06f01f7a50 con 0x7f06f0159e10 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06ed600640 1 -- 172.21.15.47:0/3757414896 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06dc011c50 con 0x7f06f0159e10 2024-04-20T10:27:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.378+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06f01f8010 con 0x7f06f0159e10 2024-04-20T10:27:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.380+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06f00742b0 con 0x7f06f0159e10 2024-04-20T10:27:55.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.381+0000 7f06ed600640 1 -- 172.21.15.47:0/3757414896 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f06dc01b460 con 0x7f06f0159e10 2024-04-20T10:27:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.382+0000 7f06ed600640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f06d006c8f0 0x7f06d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.382+0000 7f06ed600640 1 -- 172.21.15.47:0/3757414896 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f06dc08bc20 con 0x7f06f0159e10 2024-04-20T10:27:55.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.383+0000 7f06f5600640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f06d006c8f0 0x7f06d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.384+0000 7f06f5600640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f06d006c8f0 0x7f06d006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f06e4036000 tx=0x7f06e4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:55.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.387+0000 7f06ed600640 1 -- 172.21.15.47:0/3757414896 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f06f00742b0 con 0x7f06f0159e10 2024-04-20T10:27:55.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.587+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f06f00cc460 con 0x7f06d006c8f0 2024-04-20T10:27:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.619+0000 7f06ed600640 1 -- 172.21.15.47:0/3757414896 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332566 (secure 0 0 0) 0x7f06f00cc460 con 0x7f06d006c8f0 2024-04-20T10:27:55.624 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:27:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f06d006c8f0 msgr2=0x7f06d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f06d006c8f0 0x7f06d006edb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f06e4036000 tx=0x7f06e4008030 comp rx=0 tx=0).stop 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 msgr2=0x7f06f00b4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f00b4270 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f06dc00b3c0 tx=0x7f06dc00b880 comp rx=0 tx=0).stop 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 shutdown_connections 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06f015a1f0 0x7f06f00b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f06d006c8f0 0x7f06d006edb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f06f0159e10 0x7f06f00b4270 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 --2- 172.21.15.47:0/3757414896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06f006da90 0x7f06f00ca420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 >> 172.21.15.47:0/3757414896 conn(0x7f06f00d0040 msgr2=0x7f06f006c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 shutdown_connections 2024-04-20T10:27:55.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.625+0000 7f06f6a00640 1 -- 172.21.15.47:0/3757414896 wait complete. 2024-04-20T10:27:55.645 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-20T10:27:55.071410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227932,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746788,"statfs":{"total":773094113280,"available":772860710912,"internally_reserved":0,"allocated":3514368,"data_stored":2072155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005176"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392064+0000","last_change":"2024-04-20T10:27:32.090082+0000","last_active":"2024-04-20T10:27:38.392064+0000","last_peered":"2024-04-20T10:27:38.392064+0000","last_clean":"2024-04-20T10:27:38.392064+0000","last_became_active":"2024-04-20T10:27:32.089819+0000","last_became_peered":"2024-04-20T10:27:32.089819+0000","last_unstale":"2024-04-20T10:27:38.392064+0000","last_undegraded":"2024-04-20T10:27:38.392064+0000","last_fullsized":"2024-04-20T10:27:38.392064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:16.568087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112567+0000","last_change":"2024-04-20T10:27:33.015363+0000","last_active":"2024-04-20T10:27:38.112567+0000","last_peered":"2024-04-20T10:27:38.112567+0000","last_clean":"2024-04-20T10:27:38.112567+0000","last_became_active":"2024-04-20T10:27:33.013421+0000","last_became_peered":"2024-04-20T10:27:33.013421+0000","last_unstale":"2024-04-20T10:27:38.112567+0000","last_undegraded":"2024-04-20T10:27:38.112567+0000","last_fullsized":"2024-04-20T10:27:38.112567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391215+0000","last_change":"2024-04-20T10:27:37.132637+0000","last_active":"2024-04-20T10:27:38.391215+0000","last_peered":"2024-04-20T10:27:38.391215+0000","last_clean":"2024-04-20T10:27:38.391215+0000","last_became_active":"2024-04-20T10:27:37.132147+0000","last_became_peered":"2024-04-20T10:27:37.132147+0000","last_unstale":"2024-04-20T10:27:38.391215+0000","last_undegraded":"2024-04-20T10:27:38.391215+0000","last_fullsized":"2024-04-20T10:27:38.391215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:13.081027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112843+0000","last_change":"2024-04-20T10:27:32.093624+0000","last_active":"2024-04-20T10:27:38.112843+0000","last_peered":"2024-04-20T10:27:38.112843+0000","last_clean":"2024-04-20T10:27:38.112843+0000","last_became_active":"2024-04-20T10:27:32.093349+0000","last_became_peered":"2024-04-20T10:27:32.093349+0000","last_unstale":"2024-04-20T10:27:38.112843+0000","last_undegraded":"2024-04-20T10:27:38.112843+0000","last_fullsized":"2024-04-20T10:27:38.112843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113178+0000","last_change":"2024-04-20T10:27:33.019078+0000","last_active":"2024-04-20T10:27:38.113178+0000","last_peered":"2024-04-20T10:27:38.113178+0000","last_clean":"2024-04-20T10:27:38.113178+0000","last_became_active":"2024-04-20T10:27:33.018362+0000","last_became_peered":"2024-04-20T10:27:33.018362+0000","last_unstale":"2024-04-20T10:27:38.113178+0000","last_undegraded":"2024-04-20T10:27:38.113178+0000","last_fullsized":"2024-04-20T10:27:38.113178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390250+0000","last_change":"2024-04-20T10:27:37.118407+0000","last_active":"2024-04-20T10:27:38.390250+0000","last_peered":"2024-04-20T10:27:38.390250+0000","last_clean":"2024-04-20T10:27:38.390250+0000","last_became_active":"2024-04-20T10:27:37.117850+0000","last_became_peered":"2024-04-20T10:27:37.117850+0000","last_unstale":"2024-04-20T10:27:38.390250+0000","last_undegraded":"2024-04-20T10:27:38.390250+0000","last_fullsized":"2024-04-20T10:27:38.390250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:36.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388629+0000","last_change":"2024-04-20T10:27:32.080920+0000","last_active":"2024-04-20T10:27:38.388629+0000","last_peered":"2024-04-20T10:27:38.388629+0000","last_clean":"2024-04-20T10:27:38.388629+0000","last_became_active":"2024-04-20T10:27:32.080614+0000","last_became_peered":"2024-04-20T10:27:32.080614+0000","last_unstale":"2024-04-20T10:27:38.388629+0000","last_undegraded":"2024-04-20T10:27:38.388629+0000","last_fullsized":"2024-04-20T10:27:38.388629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:51.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392440+0000","last_change":"2024-04-20T10:27:33.006318+0000","last_active":"2024-04-20T10:27:38.392440+0000","last_peered":"2024-04-20T10:27:38.392440+0000","last_clean":"2024-04-20T10:27:38.392440+0000","last_became_active":"2024-04-20T10:27:33.006040+0000","last_became_peered":"2024-04-20T10:27:33.006040+0000","last_unstale":"2024-04-20T10:27:38.392440+0000","last_undegraded":"2024-04-20T10:27:38.392440+0000","last_fullsized":"2024-04-20T10:27:38.392440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391743+0000","last_change":"2024-04-20T10:27:37.123688+0000","last_active":"2024-04-20T10:27:38.391743+0000","last_peered":"2024-04-20T10:27:38.391743+0000","last_clean":"2024-04-20T10:27:38.391743+0000","last_became_active":"2024-04-20T10:27:37.123316+0000","last_became_peered":"2024-04-20T10:27:37.123316+0000","last_unstale":"2024-04-20T10:27:38.391743+0000","last_undegraded":"2024-04-20T10:27:38.391743+0000","last_fullsized":"2024-04-20T10:27:38.391743+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388939+0000","last_change":"2024-04-20T10:27:32.082282+0000","last_active":"2024-04-20T10:27:38.388939+0000","last_peered":"2024-04-20T10:27:38.388939+0000","last_clean":"2024-04-20T10:27:38.388939+0000","last_became_active":"2024-04-20T10:27:32.081943+0000","last_became_peered":"2024-04-20T10:27:32.081943+0000","last_unstale":"2024-04-20T10:27:38.388939+0000","last_undegraded":"2024-04-20T10:27:38.388939+0000","last_fullsized":"2024-04-20T10:27:38.388939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:12.394383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108312+0000","last_change":"2024-04-20T10:27:33.003700+0000","last_active":"2024-04-20T10:27:38.108312+0000","last_peered":"2024-04-20T10:27:38.108312+0000","last_clean":"2024-04-20T10:27:38.108312+0000","last_became_active":"2024-04-20T10:27:33.003443+0000","last_became_peered":"2024-04-20T10:27:33.003443+0000","last_unstale":"2024-04-20T10:27:38.108312+0000","last_undegraded":"2024-04-20T10:27:38.108312+0000","last_fullsized":"2024-04-20T10:27:38.108312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:33.760650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390259+0000","last_change":"2024-04-20T10:27:37.127775+0000","last_active":"2024-04-20T10:27:38.390259+0000","last_peered":"2024-04-20T10:27:38.390259+0000","last_clean":"2024-04-20T10:27:38.390259+0000","last_became_active":"2024-04-20T10:27:37.126837+0000","last_became_peered":"2024-04-20T10:27:37.126837+0000","last_unstale":"2024-04-20T10:27:38.390259+0000","last_undegraded":"2024-04-20T10:27:38.390259+0000","last_fullsized":"2024-04-20T10:27:38.390259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:31.369897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106458+0000","last_change":"2024-04-20T10:27:33.004497+0000","last_active":"2024-04-20T10:27:38.106458+0000","last_peered":"2024-04-20T10:27:38.106458+0000","last_clean":"2024-04-20T10:27:38.106458+0000","last_became_active":"2024-04-20T10:27:33.004228+0000","last_became_peered":"2024-04-20T10:27:33.004228+0000","last_unstale":"2024-04-20T10:27:38.106458+0000","last_undegraded":"2024-04-20T10:27:38.106458+0000","last_fullsized":"2024-04-20T10:27:38.106458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:04.936311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106698+0000","last_change":"2024-04-20T10:27:32.069803+0000","last_active":"2024-04-20T10:27:38.106698+0000","last_peered":"2024-04-20T10:27:38.106698+0000","last_clean":"2024-04-20T10:27:38.106698+0000","last_became_active":"2024-04-20T10:27:32.069542+0000","last_became_peered":"2024-04-20T10:27:32.069542+0000","last_unstale":"2024-04-20T10:27:38.106698+0000","last_undegraded":"2024-04-20T10:27:38.106698+0000","last_fullsized":"2024-04-20T10:27:38.106698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107502+0000","last_change":"2024-04-20T10:27:37.122386+0000","last_active":"2024-04-20T10:27:38.107502+0000","last_peered":"2024-04-20T10:27:38.107502+0000","last_clean":"2024-04-20T10:27:38.107502+0000","last_became_active":"2024-04-20T10:27:37.121943+0000","last_became_peered":"2024-04-20T10:27:37.121943+0000","last_unstale":"2024-04-20T10:27:38.107502+0000","last_undegraded":"2024-04-20T10:27:38.107502+0000","last_fullsized":"2024-04-20T10:27:38.107502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:53:31.071316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391786+0000","last_change":"2024-04-20T10:27:33.015242+0000","last_active":"2024-04-20T10:27:38.391786+0000","last_peered":"2024-04-20T10:27:38.391786+0000","last_clean":"2024-04-20T10:27:38.391786+0000","last_became_active":"2024-04-20T10:27:33.014003+0000","last_became_peered":"2024-04-20T10:27:33.014003+0000","last_unstale":"2024-04-20T10:27:38.391786+0000","last_undegraded":"2024-04-20T10:27:38.391786+0000","last_fullsized":"2024-04-20T10:27:38.391786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110379+0000","last_change":"2024-04-20T10:27:32.087085+0000","last_active":"2024-04-20T10:27:38.110379+0000","last_peered":"2024-04-20T10:27:38.110379+0000","last_clean":"2024-04-20T10:27:38.110379+0000","last_became_active":"2024-04-20T10:27:32.086777+0000","last_became_peered":"2024-04-20T10:27:32.086777+0000","last_unstale":"2024-04-20T10:27:38.110379+0000","last_undegraded":"2024-04-20T10:27:38.110379+0000","last_fullsized":"2024-04-20T10:27:38.110379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388751+0000","last_change":"2024-04-20T10:27:37.123374+0000","last_active":"2024-04-20T10:27:38.388751+0000","last_peered":"2024-04-20T10:27:38.388751+0000","last_clean":"2024-04-20T10:27:38.388751+0000","last_became_active":"2024-04-20T10:27:37.118855+0000","last_became_peered":"2024-04-20T10:27:37.118855+0000","last_unstale":"2024-04-20T10:27:38.388751+0000","last_undegraded":"2024-04-20T10:27:38.388751+0000","last_fullsized":"2024-04-20T10:27:38.388751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389358+0000","last_change":"2024-04-20T10:27:33.013100+0000","last_active":"2024-04-20T10:27:38.389358+0000","last_peered":"2024-04-20T10:27:38.389358+0000","last_clean":"2024-04-20T10:27:38.389358+0000","last_became_active":"2024-04-20T10:27:33.012720+0000","last_became_peered":"2024-04-20T10:27:33.012720+0000","last_unstale":"2024-04-20T10:27:38.389358+0000","last_undegraded":"2024-04-20T10:27:38.389358+0000","last_fullsized":"2024-04-20T10:27:38.389358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:52.950066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390484+0000","last_change":"2024-04-20T10:27:32.082330+0000","last_active":"2024-04-20T10:27:38.390484+0000","last_peered":"2024-04-20T10:27:38.390484+0000","last_clean":"2024-04-20T10:27:38.390484+0000","last_became_active":"2024-04-20T10:27:32.080602+0000","last_became_peered":"2024-04-20T10:27:32.080602+0000","last_unstale":"2024-04-20T10:27:38.390484+0000","last_undegraded":"2024-04-20T10:27:38.390484+0000","last_fullsized":"2024-04-20T10:27:38.390484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:59.067061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392568+0000","last_change":"2024-04-20T10:27:37.125949+0000","last_active":"2024-04-20T10:27:38.392568+0000","last_peered":"2024-04-20T10:27:38.392568+0000","last_clean":"2024-04-20T10:27:38.392568+0000","last_became_active":"2024-04-20T10:27:37.125481+0000","last_became_peered":"2024-04-20T10:27:37.125481+0000","last_unstale":"2024-04-20T10:27:38.392568+0000","last_undegraded":"2024-04-20T10:27:38.392568+0000","last_fullsized":"2024-04-20T10:27:38.392568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:15.992852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105786+0000","last_change":"2024-04-20T10:27:33.015612+0000","last_active":"2024-04-20T10:27:38.105786+0000","last_peered":"2024-04-20T10:27:38.105786+0000","last_clean":"2024-04-20T10:27:38.105786+0000","last_became_active":"2024-04-20T10:27:33.014370+0000","last_became_peered":"2024-04-20T10:27:33.014370+0000","last_unstale":"2024-04-20T10:27:38.105786+0000","last_undegraded":"2024-04-20T10:27:38.105786+0000","last_fullsized":"2024-04-20T10:27:38.105786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387559+0000","last_change":"2024-04-20T10:27:32.068081+0000","last_active":"2024-04-20T10:27:38.387559+0000","last_peered":"2024-04-20T10:27:38.387559+0000","last_clean":"2024-04-20T10:27:38.387559+0000","last_became_active":"2024-04-20T10:27:32.067727+0000","last_became_peered":"2024-04-20T10:27:32.067727+0000","last_unstale":"2024-04-20T10:27:38.387559+0000","last_undegraded":"2024-04-20T10:27:38.387559+0000","last_fullsized":"2024-04-20T10:27:38.387559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:59.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392244+0000","last_change":"2024-04-20T10:27:37.119884+0000","last_active":"2024-04-20T10:27:38.392244+0000","last_peered":"2024-04-20T10:27:38.392244+0000","last_clean":"2024-04-20T10:27:38.392244+0000","last_became_active":"2024-04-20T10:27:37.119461+0000","last_became_peered":"2024-04-20T10:27:37.119461+0000","last_unstale":"2024-04-20T10:27:38.392244+0000","last_undegraded":"2024-04-20T10:27:38.392244+0000","last_fullsized":"2024-04-20T10:27:38.392244+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391977+0000","last_change":"2024-04-20T10:27:33.009139+0000","last_active":"2024-04-20T10:27:38.391977+0000","last_peered":"2024-04-20T10:27:38.391977+0000","last_clean":"2024-04-20T10:27:38.391977+0000","last_became_active":"2024-04-20T10:27:33.008830+0000","last_became_peered":"2024-04-20T10:27:33.008830+0000","last_unstale":"2024-04-20T10:27:38.391977+0000","last_undegraded":"2024-04-20T10:27:38.391977+0000","last_fullsized":"2024-04-20T10:27:38.391977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:06.157535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108669+0000","last_change":"2024-04-20T10:27:32.086356+0000","last_active":"2024-04-20T10:27:38.108669+0000","last_peered":"2024-04-20T10:27:38.108669+0000","last_clean":"2024-04-20T10:27:38.108669+0000","last_became_active":"2024-04-20T10:27:32.077640+0000","last_became_peered":"2024-04-20T10:27:32.077640+0000","last_unstale":"2024-04-20T10:27:38.108669+0000","last_undegraded":"2024-04-20T10:27:38.108669+0000","last_fullsized":"2024-04-20T10:27:38.108669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:44.469214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390797+0000","last_change":"2024-04-20T10:27:37.124270+0000","last_active":"2024-04-20T10:27:38.390797+0000","last_peered":"2024-04-20T10:27:38.390797+0000","last_clean":"2024-04-20T10:27:38.390797+0000","last_became_active":"2024-04-20T10:27:37.123953+0000","last_became_peered":"2024-04-20T10:27:37.123953+0000","last_unstale":"2024-04-20T10:27:38.390797+0000","last_undegraded":"2024-04-20T10:27:38.390797+0000","last_fullsized":"2024-04-20T10:27:38.390797+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:27.841628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390487+0000","last_change":"2024-04-20T10:27:33.017825+0000","last_active":"2024-04-20T10:27:38.390487+0000","last_peered":"2024-04-20T10:27:38.390487+0000","last_clean":"2024-04-20T10:27:38.390487+0000","last_became_active":"2024-04-20T10:27:33.015635+0000","last_became_peered":"2024-04-20T10:27:33.015635+0000","last_unstale":"2024-04-20T10:27:38.390487+0000","last_undegraded":"2024-04-20T10:27:38.390487+0000","last_fullsized":"2024-04-20T10:27:38.390487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:58.914910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392232+0000","last_change":"2024-04-20T10:27:32.082908+0000","last_active":"2024-04-20T10:27:38.392232+0000","last_peered":"2024-04-20T10:27:38.392232+0000","last_clean":"2024-04-20T10:27:38.392232+0000","last_became_active":"2024-04-20T10:27:32.082507+0000","last_became_peered":"2024-04-20T10:27:32.082507+0000","last_unstale":"2024-04-20T10:27:38.392232+0000","last_undegraded":"2024-04-20T10:27:38.392232+0000","last_fullsized":"2024-04-20T10:27:38.392232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:52.804365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390908+0000","last_change":"2024-04-20T10:27:37.126093+0000","last_active":"2024-04-20T10:27:38.390908+0000","last_peered":"2024-04-20T10:27:38.390908+0000","last_clean":"2024-04-20T10:27:38.390908+0000","last_became_active":"2024-04-20T10:27:37.125681+0000","last_became_peered":"2024-04-20T10:27:37.125681+0000","last_unstale":"2024-04-20T10:27:38.390908+0000","last_undegraded":"2024-04-20T10:27:38.390908+0000","last_fullsized":"2024-04-20T10:27:38.390908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:13:36.622877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109415+0000","last_change":"2024-04-20T10:27:33.011662+0000","last_active":"2024-04-20T10:27:38.109415+0000","last_peered":"2024-04-20T10:27:38.109415+0000","last_clean":"2024-04-20T10:27:38.109415+0000","last_became_active":"2024-04-20T10:27:33.010842+0000","last_became_peered":"2024-04-20T10:27:33.010842+0000","last_unstale":"2024-04-20T10:27:38.109415+0000","last_undegraded":"2024-04-20T10:27:38.109415+0000","last_fullsized":"2024-04-20T10:27:38.109415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108664+0000","last_change":"2024-04-20T10:27:32.083502+0000","last_active":"2024-04-20T10:27:38.108664+0000","last_peered":"2024-04-20T10:27:38.108664+0000","last_clean":"2024-04-20T10:27:38.108664+0000","last_became_active":"2024-04-20T10:27:32.083217+0000","last_became_peered":"2024-04-20T10:27:32.083217+0000","last_unstale":"2024-04-20T10:27:38.108664+0000","last_undegraded":"2024-04-20T10:27:38.108664+0000","last_fullsized":"2024-04-20T10:27:38.108664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:41.953724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109193+0000","last_change":"2024-04-20T10:27:37.110995+0000","last_active":"2024-04-20T10:27:38.109193+0000","last_peered":"2024-04-20T10:27:38.109193+0000","last_clean":"2024-04-20T10:27:38.109193+0000","last_became_active":"2024-04-20T10:27:37.110619+0000","last_became_peered":"2024-04-20T10:27:37.110619+0000","last_unstale":"2024-04-20T10:27:38.109193+0000","last_undegraded":"2024-04-20T10:27:38.109193+0000","last_fullsized":"2024-04-20T10:27:38.109193+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:33.173033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391490+0000","last_change":"2024-04-20T10:27:32.997361+0000","last_active":"2024-04-20T10:27:38.391490+0000","last_peered":"2024-04-20T10:27:38.391490+0000","last_clean":"2024-04-20T10:27:38.391490+0000","last_became_active":"2024-04-20T10:27:32.996366+0000","last_became_peered":"2024-04-20T10:27:32.996366+0000","last_unstale":"2024-04-20T10:27:38.391490+0000","last_undegraded":"2024-04-20T10:27:38.391490+0000","last_fullsized":"2024-04-20T10:27:38.391490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110460+0000","last_change":"2024-04-20T10:27:32.087311+0000","last_active":"2024-04-20T10:27:38.110460+0000","last_peered":"2024-04-20T10:27:38.110460+0000","last_clean":"2024-04-20T10:27:38.110460+0000","last_became_active":"2024-04-20T10:27:32.086959+0000","last_became_peered":"2024-04-20T10:27:32.086959+0000","last_unstale":"2024-04-20T10:27:38.110460+0000","last_undegraded":"2024-04-20T10:27:38.110460+0000","last_fullsized":"2024-04-20T10:27:38.110460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110080+0000","last_change":"2024-04-20T10:27:37.121913+0000","last_active":"2024-04-20T10:27:38.110080+0000","last_peered":"2024-04-20T10:27:38.110080+0000","last_clean":"2024-04-20T10:27:38.110080+0000","last_became_active":"2024-04-20T10:27:37.121491+0000","last_became_peered":"2024-04-20T10:27:37.121491+0000","last_unstale":"2024-04-20T10:27:38.110080+0000","last_undegraded":"2024-04-20T10:27:38.110080+0000","last_fullsized":"2024-04-20T10:27:38.110080+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:28.925537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388610+0000","last_change":"2024-04-20T10:27:33.004607+0000","last_active":"2024-04-20T10:27:38.388610+0000","last_peered":"2024-04-20T10:27:38.388610+0000","last_clean":"2024-04-20T10:27:38.388610+0000","last_became_active":"2024-04-20T10:27:33.004299+0000","last_became_peered":"2024-04-20T10:27:33.004299+0000","last_unstale":"2024-04-20T10:27:38.388610+0000","last_undegraded":"2024-04-20T10:27:38.388610+0000","last_fullsized":"2024-04-20T10:27:38.388610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107052+0000","last_change":"2024-04-20T10:27:32.082735+0000","last_active":"2024-04-20T10:27:38.107052+0000","last_peered":"2024-04-20T10:27:38.107052+0000","last_clean":"2024-04-20T10:27:38.107052+0000","last_became_active":"2024-04-20T10:27:32.079695+0000","last_became_peered":"2024-04-20T10:27:32.079695+0000","last_unstale":"2024-04-20T10:27:38.107052+0000","last_undegraded":"2024-04-20T10:27:38.107052+0000","last_fullsized":"2024-04-20T10:27:38.107052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:07.962987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109079+0000","last_change":"2024-04-20T10:27:37.131952+0000","last_active":"2024-04-20T10:27:38.109079+0000","last_peered":"2024-04-20T10:27:38.109079+0000","last_clean":"2024-04-20T10:27:38.109079+0000","last_became_active":"2024-04-20T10:27:37.130721+0000","last_became_peered":"2024-04-20T10:27:37.130721+0000","last_unstale":"2024-04-20T10:27:38.109079+0000","last_undegraded":"2024-04-20T10:27:38.109079+0000","last_fullsized":"2024-04-20T10:27:38.109079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107777+0000","last_change":"2024-04-20T10:27:33.013537+0000","last_active":"2024-04-20T10:27:38.107777+0000","last_peered":"2024-04-20T10:27:38.107777+0000","last_clean":"2024-04-20T10:27:38.107777+0000","last_became_active":"2024-04-20T10:27:33.013223+0000","last_became_peered":"2024-04-20T10:27:33.013223+0000","last_unstale":"2024-04-20T10:27:38.107777+0000","last_undegraded":"2024-04-20T10:27:38.107777+0000","last_fullsized":"2024-04-20T10:27:38.107777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391320+0000","last_change":"2024-04-20T10:27:32.086422+0000","last_active":"2024-04-20T10:27:38.391320+0000","last_peered":"2024-04-20T10:27:38.391320+0000","last_clean":"2024-04-20T10:27:38.391320+0000","last_became_active":"2024-04-20T10:27:32.085353+0000","last_became_peered":"2024-04-20T10:27:32.085353+0000","last_unstale":"2024-04-20T10:27:38.391320+0000","last_undegraded":"2024-04-20T10:27:38.391320+0000","last_fullsized":"2024-04-20T10:27:38.391320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112264+0000","last_change":"2024-04-20T10:27:37.113304+0000","last_active":"2024-04-20T10:27:38.112264+0000","last_peered":"2024-04-20T10:27:38.112264+0000","last_clean":"2024-04-20T10:27:38.112264+0000","last_became_active":"2024-04-20T10:27:37.112828+0000","last_became_peered":"2024-04-20T10:27:37.112828+0000","last_unstale":"2024-04-20T10:27:38.112264+0000","last_undegraded":"2024-04-20T10:27:38.112264+0000","last_fullsized":"2024-04-20T10:27:38.112264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:19.178189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108899+0000","last_change":"2024-04-20T10:27:32.989233+0000","last_active":"2024-04-20T10:27:38.108899+0000","last_peered":"2024-04-20T10:27:38.108899+0000","last_clean":"2024-04-20T10:27:38.108899+0000","last_became_active":"2024-04-20T10:27:32.988092+0000","last_became_peered":"2024-04-20T10:27:32.988092+0000","last_unstale":"2024-04-20T10:27:38.108899+0000","last_undegraded":"2024-04-20T10:27:38.108899+0000","last_fullsized":"2024-04-20T10:27:38.108899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:11.889218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106243+0000","last_change":"2024-04-20T10:27:32.092257+0000","last_active":"2024-04-20T10:27:38.106243+0000","last_peered":"2024-04-20T10:27:38.106243+0000","last_clean":"2024-04-20T10:27:38.106243+0000","last_became_active":"2024-04-20T10:27:32.091944+0000","last_became_peered":"2024-04-20T10:27:32.091944+0000","last_unstale":"2024-04-20T10:27:38.106243+0000","last_undegraded":"2024-04-20T10:27:38.106243+0000","last_fullsized":"2024-04-20T10:27:38.106243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:56.139394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112022+0000","last_change":"2024-04-20T10:27:37.124420+0000","last_active":"2024-04-20T10:27:38.112022+0000","last_peered":"2024-04-20T10:27:38.112022+0000","last_clean":"2024-04-20T10:27:38.112022+0000","last_became_active":"2024-04-20T10:27:37.124048+0000","last_became_peered":"2024-04-20T10:27:37.124048+0000","last_unstale":"2024-04-20T10:27:38.112022+0000","last_undegraded":"2024-04-20T10:27:38.112022+0000","last_fullsized":"2024-04-20T10:27:38.112022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:37.864284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390325+0000","last_change":"2024-04-20T10:27:33.016860+0000","last_active":"2024-04-20T10:27:38.390325+0000","last_peered":"2024-04-20T10:27:38.390325+0000","last_clean":"2024-04-20T10:27:38.390325+0000","last_became_active":"2024-04-20T10:27:33.016323+0000","last_became_peered":"2024-04-20T10:27:33.016323+0000","last_unstale":"2024-04-20T10:27:38.390325+0000","last_undegraded":"2024-04-20T10:27:38.390325+0000","last_fullsized":"2024-04-20T10:27:38.390325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:11:39.053198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107037+0000","last_change":"2024-04-20T10:27:32.069209+0000","last_active":"2024-04-20T10:27:38.107037+0000","last_peered":"2024-04-20T10:27:38.107037+0000","last_clean":"2024-04-20T10:27:38.107037+0000","last_became_active":"2024-04-20T10:27:32.068711+0000","last_became_peered":"2024-04-20T10:27:32.068711+0000","last_unstale":"2024-04-20T10:27:38.107037+0000","last_undegraded":"2024-04-20T10:27:38.107037+0000","last_fullsized":"2024-04-20T10:27:38.107037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:55.256665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392446+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:27:38.392446+0000","last_peered":"2024-04-20T10:27:38.392446+0000","last_clean":"2024-04-20T10:27:38.392446+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:27:38.392446+0000","last_undegraded":"2024-04-20T10:27:38.392446+0000","last_fullsized":"2024-04-20T10:27:38.392446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108582+0000","last_change":"2024-04-20T10:27:37.103228+0000","last_active":"2024-04-20T10:27:38.108582+0000","last_peered":"2024-04-20T10:27:38.108582+0000","last_clean":"2024-04-20T10:27:38.108582+0000","last_became_active":"2024-04-20T10:27:37.102768+0000","last_became_peered":"2024-04-20T10:27:37.102768+0000","last_unstale":"2024-04-20T10:27:38.108582+0000","last_undegraded":"2024-04-20T10:27:38.108582+0000","last_fullsized":"2024-04-20T10:27:38.108582+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:13.639017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108024+0000","last_change":"2024-04-20T10:27:33.014266+0000","last_active":"2024-04-20T10:27:38.108024+0000","last_peered":"2024-04-20T10:27:38.108024+0000","last_clean":"2024-04-20T10:27:38.108024+0000","last_became_active":"2024-04-20T10:27:33.013935+0000","last_became_peered":"2024-04-20T10:27:33.013935+0000","last_unstale":"2024-04-20T10:27:38.108024+0000","last_undegraded":"2024-04-20T10:27:38.108024+0000","last_fullsized":"2024-04-20T10:27:38.108024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:03.292296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109594+0000","last_change":"2024-04-20T10:27:32.078144+0000","last_active":"2024-04-20T10:27:38.109594+0000","last_peered":"2024-04-20T10:27:38.109594+0000","last_clean":"2024-04-20T10:27:38.109594+0000","last_became_active":"2024-04-20T10:27:32.075588+0000","last_became_peered":"2024-04-20T10:27:32.075588+0000","last_unstale":"2024-04-20T10:27:38.109594+0000","last_undegraded":"2024-04-20T10:27:38.109594+0000","last_fullsized":"2024-04-20T10:27:38.109594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390435+0000","last_change":"2024-04-20T10:27:37.128340+0000","last_active":"2024-04-20T10:27:38.390435+0000","last_peered":"2024-04-20T10:27:38.390435+0000","last_clean":"2024-04-20T10:27:38.390435+0000","last_became_active":"2024-04-20T10:27:37.127609+0000","last_became_peered":"2024-04-20T10:27:37.127609+0000","last_unstale":"2024-04-20T10:27:38.390435+0000","last_undegraded":"2024-04-20T10:27:38.390435+0000","last_fullsized":"2024-04-20T10:27:38.390435+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:41.767940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388918+0000","last_change":"2024-04-20T10:27:33.007624+0000","last_active":"2024-04-20T10:27:38.388918+0000","last_peered":"2024-04-20T10:27:38.388918+0000","last_clean":"2024-04-20T10:27:38.388918+0000","last_became_active":"2024-04-20T10:27:33.007319+0000","last_became_peered":"2024-04-20T10:27:33.007319+0000","last_unstale":"2024-04-20T10:27:38.388918+0000","last_undegraded":"2024-04-20T10:27:38.388918+0000","last_fullsized":"2024-04-20T10:27:38.388918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112959+0000","last_change":"2024-04-20T10:27:32.087470+0000","last_active":"2024-04-20T10:27:38.112959+0000","last_peered":"2024-04-20T10:27:38.112959+0000","last_clean":"2024-04-20T10:27:38.112959+0000","last_became_active":"2024-04-20T10:27:32.086045+0000","last_became_peered":"2024-04-20T10:27:32.086045+0000","last_unstale":"2024-04-20T10:27:38.112959+0000","last_undegraded":"2024-04-20T10:27:38.112959+0000","last_fullsized":"2024-04-20T10:27:38.112959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111234+0000","last_change":"2024-04-20T10:27:37.118303+0000","last_active":"2024-04-20T10:27:38.111234+0000","last_peered":"2024-04-20T10:27:38.111234+0000","last_clean":"2024-04-20T10:27:38.111234+0000","last_became_active":"2024-04-20T10:27:37.117875+0000","last_became_peered":"2024-04-20T10:27:37.117875+0000","last_unstale":"2024-04-20T10:27:38.111234+0000","last_undegraded":"2024-04-20T10:27:38.111234+0000","last_fullsized":"2024-04-20T10:27:38.111234+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389469+0000","last_change":"2024-04-20T10:27:33.021039+0000","last_active":"2024-04-20T10:27:38.389469+0000","last_peered":"2024-04-20T10:27:38.389469+0000","last_clean":"2024-04-20T10:27:38.389469+0000","last_became_active":"2024-04-20T10:27:33.020767+0000","last_became_peered":"2024-04-20T10:27:33.020767+0000","last_unstale":"2024-04-20T10:27:38.389469+0000","last_undegraded":"2024-04-20T10:27:38.389469+0000","last_fullsized":"2024-04-20T10:27:38.389469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392137+0000","last_change":"2024-04-20T10:27:32.091910+0000","last_active":"2024-04-20T10:27:38.392137+0000","last_peered":"2024-04-20T10:27:38.392137+0000","last_clean":"2024-04-20T10:27:38.392137+0000","last_became_active":"2024-04-20T10:27:32.091500+0000","last_became_peered":"2024-04-20T10:27:32.091500+0000","last_unstale":"2024-04-20T10:27:38.392137+0000","last_undegraded":"2024-04-20T10:27:38.392137+0000","last_fullsized":"2024-04-20T10:27:38.392137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:53:02.468594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105575+0000","last_change":"2024-04-20T10:27:37.119655+0000","last_active":"2024-04-20T10:27:38.105575+0000","last_peered":"2024-04-20T10:27:38.105575+0000","last_clean":"2024-04-20T10:27:38.105575+0000","last_became_active":"2024-04-20T10:27:37.119274+0000","last_became_peered":"2024-04-20T10:27:37.119274+0000","last_unstale":"2024-04-20T10:27:38.105575+0000","last_undegraded":"2024-04-20T10:27:38.105575+0000","last_fullsized":"2024-04-20T10:27:38.105575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:56:33.147822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392306+0000","last_change":"2024-04-20T10:27:33.003557+0000","last_active":"2024-04-20T10:27:38.392306+0000","last_peered":"2024-04-20T10:27:38.392306+0000","last_clean":"2024-04-20T10:27:38.392306+0000","last_became_active":"2024-04-20T10:27:33.001763+0000","last_became_peered":"2024-04-20T10:27:33.001763+0000","last_unstale":"2024-04-20T10:27:38.392306+0000","last_undegraded":"2024-04-20T10:27:38.392306+0000","last_fullsized":"2024-04-20T10:27:38.392306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:45.483619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389540+0000","last_change":"2024-04-20T10:27:32.090042+0000","last_active":"2024-04-20T10:27:38.389540+0000","last_peered":"2024-04-20T10:27:38.389540+0000","last_clean":"2024-04-20T10:27:38.389540+0000","last_became_active":"2024-04-20T10:27:32.089694+0000","last_became_peered":"2024-04-20T10:27:32.089694+0000","last_unstale":"2024-04-20T10:27:38.389540+0000","last_undegraded":"2024-04-20T10:27:38.389540+0000","last_fullsized":"2024-04-20T10:27:38.389540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393350+0000","last_change":"2024-04-20T10:27:37.125234+0000","last_active":"2024-04-20T10:27:38.393350+0000","last_peered":"2024-04-20T10:27:38.393350+0000","last_clean":"2024-04-20T10:27:38.393350+0000","last_became_active":"2024-04-20T10:27:37.124838+0000","last_became_peered":"2024-04-20T10:27:37.124838+0000","last_unstale":"2024-04-20T10:27:38.393350+0000","last_undegraded":"2024-04-20T10:27:38.393350+0000","last_fullsized":"2024-04-20T10:27:38.393350+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:47.071764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105563+0000","last_change":"2024-04-20T10:27:33.013769+0000","last_active":"2024-04-20T10:27:38.105563+0000","last_peered":"2024-04-20T10:27:38.105563+0000","last_clean":"2024-04-20T10:27:38.105563+0000","last_became_active":"2024-04-20T10:27:33.005662+0000","last_became_peered":"2024-04-20T10:27:33.005662+0000","last_unstale":"2024-04-20T10:27:38.105563+0000","last_undegraded":"2024-04-20T10:27:38.105563+0000","last_fullsized":"2024-04-20T10:27:38.105563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389725+0000","last_change":"2024-04-20T10:27:32.080478+0000","last_active":"2024-04-20T10:27:38.389725+0000","last_peered":"2024-04-20T10:27:38.389725+0000","last_clean":"2024-04-20T10:27:38.389725+0000","last_became_active":"2024-04-20T10:27:32.080148+0000","last_became_peered":"2024-04-20T10:27:32.080148+0000","last_unstale":"2024-04-20T10:27:38.389725+0000","last_undegraded":"2024-04-20T10:27:38.389725+0000","last_fullsized":"2024-04-20T10:27:38.389725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:17.004529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112793+0000","last_change":"2024-04-20T10:27:37.126177+0000","last_active":"2024-04-20T10:27:38.112793+0000","last_peered":"2024-04-20T10:27:38.112793+0000","last_clean":"2024-04-20T10:27:38.112793+0000","last_became_active":"2024-04-20T10:27:37.125862+0000","last_became_peered":"2024-04-20T10:27:37.125862+0000","last_unstale":"2024-04-20T10:27:38.112793+0000","last_undegraded":"2024-04-20T10:27:38.112793+0000","last_fullsized":"2024-04-20T10:27:38.112793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:07.395930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391792+0000","last_change":"2024-04-20T10:27:33.016106+0000","last_active":"2024-04-20T10:27:38.391792+0000","last_peered":"2024-04-20T10:27:38.391792+0000","last_clean":"2024-04-20T10:27:38.391792+0000","last_became_active":"2024-04-20T10:27:33.015787+0000","last_became_peered":"2024-04-20T10:27:33.015787+0000","last_unstale":"2024-04-20T10:27:38.391792+0000","last_undegraded":"2024-04-20T10:27:38.391792+0000","last_fullsized":"2024-04-20T10:27:38.391792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388035+0000","last_change":"2024-04-20T10:27:32.075511+0000","last_active":"2024-04-20T10:27:38.388035+0000","last_peered":"2024-04-20T10:27:38.388035+0000","last_clean":"2024-04-20T10:27:38.388035+0000","last_became_active":"2024-04-20T10:27:32.072523+0000","last_became_peered":"2024-04-20T10:27:32.072523+0000","last_unstale":"2024-04-20T10:27:38.388035+0000","last_undegraded":"2024-04-20T10:27:38.388035+0000","last_fullsized":"2024-04-20T10:27:38.388035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113117+0000","last_change":"2024-04-20T10:27:37.121568+0000","last_active":"2024-04-20T10:27:38.113117+0000","last_peered":"2024-04-20T10:27:38.113117+0000","last_clean":"2024-04-20T10:27:38.113117+0000","last_became_active":"2024-04-20T10:27:37.121105+0000","last_became_peered":"2024-04-20T10:27:37.121105+0000","last_unstale":"2024-04-20T10:27:38.113117+0000","last_undegraded":"2024-04-20T10:27:38.113117+0000","last_fullsized":"2024-04-20T10:27:38.113117+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:20.615032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107307+0000","last_change":"2024-04-20T10:27:33.008601+0000","last_active":"2024-04-20T10:27:38.107307+0000","last_peered":"2024-04-20T10:27:38.107307+0000","last_clean":"2024-04-20T10:27:38.107307+0000","last_became_active":"2024-04-20T10:27:33.008317+0000","last_became_peered":"2024-04-20T10:27:33.008317+0000","last_unstale":"2024-04-20T10:27:38.107307+0000","last_undegraded":"2024-04-20T10:27:38.107307+0000","last_fullsized":"2024-04-20T10:27:38.107307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:55.576815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106751+0000","last_change":"2024-04-20T10:27:32.081278+0000","last_active":"2024-04-20T10:27:38.106751+0000","last_peered":"2024-04-20T10:27:38.106751+0000","last_clean":"2024-04-20T10:27:38.106751+0000","last_became_active":"2024-04-20T10:27:32.080466+0000","last_became_peered":"2024-04-20T10:27:32.080466+0000","last_unstale":"2024-04-20T10:27:38.106751+0000","last_undegraded":"2024-04-20T10:27:38.106751+0000","last_fullsized":"2024-04-20T10:27:38.106751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:51.199449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109133+0000","last_change":"2024-04-20T10:27:37.132802+0000","last_active":"2024-04-20T10:27:38.109133+0000","last_peered":"2024-04-20T10:27:38.109133+0000","last_clean":"2024-04-20T10:27:38.109133+0000","last_became_active":"2024-04-20T10:27:37.129193+0000","last_became_peered":"2024-04-20T10:27:37.129193+0000","last_unstale":"2024-04-20T10:27:38.109133+0000","last_undegraded":"2024-04-20T10:27:38.109133+0000","last_fullsized":"2024-04-20T10:27:38.109133+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110211+0000","last_change":"2024-04-20T10:27:33.010100+0000","last_active":"2024-04-20T10:27:38.110211+0000","last_peered":"2024-04-20T10:27:38.110211+0000","last_clean":"2024-04-20T10:27:38.110211+0000","last_became_active":"2024-04-20T10:27:33.009703+0000","last_became_peered":"2024-04-20T10:27:33.009703+0000","last_unstale":"2024-04-20T10:27:38.110211+0000","last_undegraded":"2024-04-20T10:27:38.110211+0000","last_fullsized":"2024-04-20T10:27:38.110211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391899+0000","last_change":"2024-04-20T10:27:32.078724+0000","last_active":"2024-04-20T10:27:38.391899+0000","last_peered":"2024-04-20T10:27:38.391899+0000","last_clean":"2024-04-20T10:27:38.391899+0000","last_became_active":"2024-04-20T10:27:32.078093+0000","last_became_peered":"2024-04-20T10:27:32.078093+0000","last_unstale":"2024-04-20T10:27:38.391899+0000","last_undegraded":"2024-04-20T10:27:38.391899+0000","last_fullsized":"2024-04-20T10:27:38.391899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107857+0000","last_change":"2024-04-20T10:27:37.121822+0000","last_active":"2024-04-20T10:27:38.107857+0000","last_peered":"2024-04-20T10:27:38.107857+0000","last_clean":"2024-04-20T10:27:38.107857+0000","last_became_active":"2024-04-20T10:27:37.121423+0000","last_became_peered":"2024-04-20T10:27:37.121423+0000","last_unstale":"2024-04-20T10:27:38.107857+0000","last_undegraded":"2024-04-20T10:27:38.107857+0000","last_fullsized":"2024-04-20T10:27:38.107857+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:21:35.974697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388424+0000","last_change":"2024-04-20T10:27:33.001752+0000","last_active":"2024-04-20T10:27:38.388424+0000","last_peered":"2024-04-20T10:27:38.388424+0000","last_clean":"2024-04-20T10:27:38.388424+0000","last_became_active":"2024-04-20T10:27:33.001201+0000","last_became_peered":"2024-04-20T10:27:33.001201+0000","last_unstale":"2024-04-20T10:27:38.388424+0000","last_undegraded":"2024-04-20T10:27:38.388424+0000","last_fullsized":"2024-04-20T10:27:38.388424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:44.536794+0000","last_change":"2024-04-20T10:27:32.080328+0000","last_active":"2024-04-20T10:27:44.536794+0000","last_peered":"2024-04-20T10:27:44.536794+0000","last_clean":"2024-04-20T10:27:44.536794+0000","last_became_active":"2024-04-20T10:27:32.080011+0000","last_became_peered":"2024-04-20T10:27:32.080011+0000","last_unstale":"2024-04-20T10:27:44.536794+0000","last_undegraded":"2024-04-20T10:27:44.536794+0000","last_fullsized":"2024-04-20T10:27:44.536794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:22:14.097676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389991+0000","last_change":"2024-04-20T10:27:37.126061+0000","last_active":"2024-04-20T10:27:38.389991+0000","last_peered":"2024-04-20T10:27:38.389991+0000","last_clean":"2024-04-20T10:27:38.389991+0000","last_became_active":"2024-04-20T10:27:37.121461+0000","last_became_peered":"2024-04-20T10:27:37.121461+0000","last_unstale":"2024-04-20T10:27:38.389991+0000","last_undegraded":"2024-04-20T10:27:38.389991+0000","last_fullsized":"2024-04-20T10:27:38.389991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:13.289328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107516+0000","last_change":"2024-04-20T10:27:33.007572+0000","last_active":"2024-04-20T10:27:38.107516+0000","last_peered":"2024-04-20T10:27:38.107516+0000","last_clean":"2024-04-20T10:27:38.107516+0000","last_became_active":"2024-04-20T10:27:33.007206+0000","last_became_peered":"2024-04-20T10:27:33.007206+0000","last_unstale":"2024-04-20T10:27:38.107516+0000","last_undegraded":"2024-04-20T10:27:38.107516+0000","last_fullsized":"2024-04-20T10:27:38.107516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:21.846828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393065+0000","last_change":"2024-04-20T10:27:32.080090+0000","last_active":"2024-04-20T10:27:38.393065+0000","last_peered":"2024-04-20T10:27:38.393065+0000","last_clean":"2024-04-20T10:27:38.393065+0000","last_became_active":"2024-04-20T10:27:32.079763+0000","last_became_peered":"2024-04-20T10:27:32.079763+0000","last_unstale":"2024-04-20T10:27:38.393065+0000","last_undegraded":"2024-04-20T10:27:38.393065+0000","last_fullsized":"2024-04-20T10:27:38.393065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390646+0000","last_change":"2024-04-20T10:27:37.113593+0000","last_active":"2024-04-20T10:27:38.390646+0000","last_peered":"2024-04-20T10:27:38.390646+0000","last_clean":"2024-04-20T10:27:38.390646+0000","last_became_active":"2024-04-20T10:27:37.113016+0000","last_became_peered":"2024-04-20T10:27:37.113016+0000","last_unstale":"2024-04-20T10:27:38.390646+0000","last_undegraded":"2024-04-20T10:27:38.390646+0000","last_fullsized":"2024-04-20T10:27:38.390646+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:27.394166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113229+0000","last_change":"2024-04-20T10:27:33.020099+0000","last_active":"2024-04-20T10:27:38.113229+0000","last_peered":"2024-04-20T10:27:38.113229+0000","last_clean":"2024-04-20T10:27:38.113229+0000","last_became_active":"2024-04-20T10:27:33.018743+0000","last_became_peered":"2024-04-20T10:27:33.018743+0000","last_unstale":"2024-04-20T10:27:38.113229+0000","last_undegraded":"2024-04-20T10:27:38.113229+0000","last_fullsized":"2024-04-20T10:27:38.113229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389020+0000","last_change":"2024-04-20T10:27:32.081599+0000","last_active":"2024-04-20T10:27:38.389020+0000","last_peered":"2024-04-20T10:27:38.389020+0000","last_clean":"2024-04-20T10:27:38.389020+0000","last_became_active":"2024-04-20T10:27:32.081288+0000","last_became_peered":"2024-04-20T10:27:32.081288+0000","last_unstale":"2024-04-20T10:27:38.389020+0000","last_undegraded":"2024-04-20T10:27:38.389020+0000","last_fullsized":"2024-04-20T10:27:38.389020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391304+0000","last_change":"2024-04-20T10:27:37.134883+0000","last_active":"2024-04-20T10:27:38.391304+0000","last_peered":"2024-04-20T10:27:38.391304+0000","last_clean":"2024-04-20T10:27:38.391304+0000","last_became_active":"2024-04-20T10:27:37.134221+0000","last_became_peered":"2024-04-20T10:27:37.134221+0000","last_unstale":"2024-04-20T10:27:38.391304+0000","last_undegraded":"2024-04-20T10:27:38.391304+0000","last_fullsized":"2024-04-20T10:27:38.391304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:10.327991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392241+0000","last_change":"2024-04-20T10:27:33.006819+0000","last_active":"2024-04-20T10:27:38.392241+0000","last_peered":"2024-04-20T10:27:38.392241+0000","last_clean":"2024-04-20T10:27:38.392241+0000","last_became_active":"2024-04-20T10:27:33.005657+0000","last_became_peered":"2024-04-20T10:27:33.005657+0000","last_unstale":"2024-04-20T10:27:38.392241+0000","last_undegraded":"2024-04-20T10:27:38.392241+0000","last_fullsized":"2024-04-20T10:27:38.392241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391630+0000","last_change":"2024-04-20T10:27:32.083786+0000","last_active":"2024-04-20T10:27:38.391630+0000","last_peered":"2024-04-20T10:27:38.391630+0000","last_clean":"2024-04-20T10:27:38.391630+0000","last_became_active":"2024-04-20T10:27:32.083256+0000","last_became_peered":"2024-04-20T10:27:32.083256+0000","last_unstale":"2024-04-20T10:27:38.391630+0000","last_undegraded":"2024-04-20T10:27:38.391630+0000","last_fullsized":"2024-04-20T10:27:38.391630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:11.023664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390075+0000","last_change":"2024-04-20T10:27:37.131051+0000","last_active":"2024-04-20T10:27:38.390075+0000","last_peered":"2024-04-20T10:27:38.390075+0000","last_clean":"2024-04-20T10:27:38.390075+0000","last_became_active":"2024-04-20T10:27:37.130723+0000","last_became_peered":"2024-04-20T10:27:37.130723+0000","last_unstale":"2024-04-20T10:27:38.390075+0000","last_undegraded":"2024-04-20T10:27:38.390075+0000","last_fullsized":"2024-04-20T10:27:38.390075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:28.980316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391821+0000","last_change":"2024-04-20T10:27:37.124930+0000","last_active":"2024-04-20T10:27:38.391821+0000","last_peered":"2024-04-20T10:27:38.391821+0000","last_clean":"2024-04-20T10:27:38.391821+0000","last_became_active":"2024-04-20T10:27:37.123089+0000","last_became_peered":"2024-04-20T10:27:37.123089+0000","last_unstale":"2024-04-20T10:27:38.391821+0000","last_undegraded":"2024-04-20T10:27:38.391821+0000","last_fullsized":"2024-04-20T10:27:38.391821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:05.113291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390231+0000","last_change":"2024-04-20T10:27:33.020003+0000","last_active":"2024-04-20T10:27:38.390231+0000","last_peered":"2024-04-20T10:27:38.390231+0000","last_clean":"2024-04-20T10:27:38.390231+0000","last_became_active":"2024-04-20T10:27:33.019709+0000","last_became_peered":"2024-04-20T10:27:33.019709+0000","last_unstale":"2024-04-20T10:27:38.390231+0000","last_undegraded":"2024-04-20T10:27:38.390231+0000","last_fullsized":"2024-04-20T10:27:38.390231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:06.663893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106161+0000","last_change":"2024-04-20T10:27:32.070067+0000","last_active":"2024-04-20T10:27:38.106161+0000","last_peered":"2024-04-20T10:27:38.106161+0000","last_clean":"2024-04-20T10:27:38.106161+0000","last_became_active":"2024-04-20T10:27:32.069761+0000","last_became_peered":"2024-04-20T10:27:32.069761+0000","last_unstale":"2024-04-20T10:27:38.106161+0000","last_undegraded":"2024-04-20T10:27:38.106161+0000","last_fullsized":"2024-04-20T10:27:38.106161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108148+0000","last_change":"2024-04-20T10:27:37.125295+0000","last_active":"2024-04-20T10:27:38.108148+0000","last_peered":"2024-04-20T10:27:38.108148+0000","last_clean":"2024-04-20T10:27:38.108148+0000","last_became_active":"2024-04-20T10:27:37.124922+0000","last_became_peered":"2024-04-20T10:27:37.124922+0000","last_unstale":"2024-04-20T10:27:38.108148+0000","last_undegraded":"2024-04-20T10:27:38.108148+0000","last_fullsized":"2024-04-20T10:27:38.108148+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:45.803688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108085+0000","last_change":"2024-04-20T10:27:33.009010+0000","last_active":"2024-04-20T10:27:38.108085+0000","last_peered":"2024-04-20T10:27:38.108085+0000","last_clean":"2024-04-20T10:27:38.108085+0000","last_became_active":"2024-04-20T10:27:33.008701+0000","last_became_peered":"2024-04-20T10:27:33.008701+0000","last_unstale":"2024-04-20T10:27:38.108085+0000","last_undegraded":"2024-04-20T10:27:38.108085+0000","last_fullsized":"2024-04-20T10:27:38.108085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109186+0000","last_change":"2024-04-20T10:27:32.079095+0000","last_active":"2024-04-20T10:27:38.109186+0000","last_peered":"2024-04-20T10:27:38.109186+0000","last_clean":"2024-04-20T10:27:38.109186+0000","last_became_active":"2024-04-20T10:27:32.074687+0000","last_became_peered":"2024-04-20T10:27:32.074687+0000","last_unstale":"2024-04-20T10:27:38.109186+0000","last_undegraded":"2024-04-20T10:27:38.109186+0000","last_fullsized":"2024-04-20T10:27:38.109186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106749+0000","last_change":"2024-04-20T10:27:32.080751+0000","last_active":"2024-04-20T10:27:38.106749+0000","last_peered":"2024-04-20T10:27:38.106749+0000","last_clean":"2024-04-20T10:27:38.106749+0000","last_became_active":"2024-04-20T10:27:32.078604+0000","last_became_peered":"2024-04-20T10:27:32.078604+0000","last_unstale":"2024-04-20T10:27:38.106749+0000","last_undegraded":"2024-04-20T10:27:38.106749+0000","last_fullsized":"2024-04-20T10:27:38.106749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:10.874813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390089+0000","last_change":"2024-04-20T10:27:33.024156+0000","last_active":"2024-04-20T10:27:38.390089+0000","last_peered":"2024-04-20T10:27:38.390089+0000","last_clean":"2024-04-20T10:27:38.390089+0000","last_became_active":"2024-04-20T10:27:33.023639+0000","last_became_peered":"2024-04-20T10:27:33.023639+0000","last_unstale":"2024-04-20T10:27:38.390089+0000","last_undegraded":"2024-04-20T10:27:38.390089+0000","last_fullsized":"2024-04-20T10:27:38.390089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:45.132088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391353+0000","last_change":"2024-04-20T10:27:37.136584+0000","last_active":"2024-04-20T10:27:38.391353+0000","last_peered":"2024-04-20T10:27:38.391353+0000","last_clean":"2024-04-20T10:27:38.391353+0000","last_became_active":"2024-04-20T10:27:37.135997+0000","last_became_peered":"2024-04-20T10:27:37.135997+0000","last_unstale":"2024-04-20T10:27:38.391353+0000","last_undegraded":"2024-04-20T10:27:38.391353+0000","last_fullsized":"2024-04-20T10:27:38.391353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:27.651374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111655+0000","last_change":"2024-04-20T10:27:32.090018+0000","last_active":"2024-04-20T10:27:38.111655+0000","last_peered":"2024-04-20T10:27:38.111655+0000","last_clean":"2024-04-20T10:27:38.111655+0000","last_became_active":"2024-04-20T10:27:32.089144+0000","last_became_peered":"2024-04-20T10:27:32.089144+0000","last_unstale":"2024-04-20T10:27:38.111655+0000","last_undegraded":"2024-04-20T10:27:38.111655+0000","last_fullsized":"2024-04-20T10:27:38.111655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390744+0000","last_change":"2024-04-20T10:27:33.010778+0000","last_active":"2024-04-20T10:27:38.390744+0000","last_peered":"2024-04-20T10:27:38.390744+0000","last_clean":"2024-04-20T10:27:38.390744+0000","last_became_active":"2024-04-20T10:27:33.010403+0000","last_became_peered":"2024-04-20T10:27:33.010403+0000","last_unstale":"2024-04-20T10:27:38.390744+0000","last_undegraded":"2024-04-20T10:27:38.390744+0000","last_fullsized":"2024-04-20T10:27:38.390744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393172+0000","last_change":"2024-04-20T10:27:37.134702+0000","last_active":"2024-04-20T10:27:38.393172+0000","last_peered":"2024-04-20T10:27:38.393172+0000","last_clean":"2024-04-20T10:27:38.393172+0000","last_became_active":"2024-04-20T10:27:37.134178+0000","last_became_peered":"2024-04-20T10:27:37.134178+0000","last_unstale":"2024-04-20T10:27:38.393172+0000","last_undegraded":"2024-04-20T10:27:38.393172+0000","last_fullsized":"2024-04-20T10:27:38.393172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:48.722800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390536+0000","last_change":"2024-04-20T10:27:32.998168+0000","last_active":"2024-04-20T10:27:38.390536+0000","last_peered":"2024-04-20T10:27:38.390536+0000","last_clean":"2024-04-20T10:27:38.390536+0000","last_became_active":"2024-04-20T10:27:32.996843+0000","last_became_peered":"2024-04-20T10:27:32.996843+0000","last_unstale":"2024-04-20T10:27:38.390536+0000","last_undegraded":"2024-04-20T10:27:38.390536+0000","last_fullsized":"2024-04-20T10:27:38.390536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:13.408788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391052+0000","last_change":"2024-04-20T10:27:33.020874+0000","last_active":"2024-04-20T10:27:38.391052+0000","last_peered":"2024-04-20T10:27:38.391052+0000","last_clean":"2024-04-20T10:27:38.391052+0000","last_became_active":"2024-04-20T10:27:33.016375+0000","last_became_peered":"2024-04-20T10:27:33.016375+0000","last_unstale":"2024-04-20T10:27:38.391052+0000","last_undegraded":"2024-04-20T10:27:38.391052+0000","last_fullsized":"2024-04-20T10:27:38.391052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105932+0000","last_change":"2024-04-20T10:27:33.011331+0000","last_active":"2024-04-20T10:27:38.105932+0000","last_peered":"2024-04-20T10:27:38.105932+0000","last_clean":"2024-04-20T10:27:38.105932+0000","last_became_active":"2024-04-20T10:27:33.010956+0000","last_became_peered":"2024-04-20T10:27:33.010956+0000","last_unstale":"2024-04-20T10:27:38.105932+0000","last_undegraded":"2024-04-20T10:27:38.105932+0000","last_fullsized":"2024-04-20T10:27:38.105932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:16.511398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111379+0000","last_change":"2024-04-20T10:27:33.009386+0000","last_active":"2024-04-20T10:27:38.111379+0000","last_peered":"2024-04-20T10:27:38.111379+0000","last_clean":"2024-04-20T10:27:38.111379+0000","last_became_active":"2024-04-20T10:27:33.008596+0000","last_became_peered":"2024-04-20T10:27:33.008596+0000","last_unstale":"2024-04-20T10:27:38.111379+0000","last_undegraded":"2024-04-20T10:27:38.111379+0000","last_fullsized":"2024-04-20T10:27:38.111379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109220+0000","last_change":"2024-04-20T10:27:33.001780+0000","last_active":"2024-04-20T10:27:38.109220+0000","last_peered":"2024-04-20T10:27:38.109220+0000","last_clean":"2024-04-20T10:27:38.109220+0000","last_became_active":"2024-04-20T10:27:33.000852+0000","last_became_peered":"2024-04-20T10:27:33.000852+0000","last_unstale":"2024-04-20T10:27:38.109220+0000","last_undegraded":"2024-04-20T10:27:38.109220+0000","last_fullsized":"2024-04-20T10:27:38.109220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:52.213855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111488+0000","last_change":"2024-04-20T10:27:33.010234+0000","last_active":"2024-04-20T10:27:38.111488+0000","last_peered":"2024-04-20T10:27:38.111488+0000","last_clean":"2024-04-20T10:27:38.111488+0000","last_became_active":"2024-04-20T10:27:33.009934+0000","last_became_peered":"2024-04-20T10:27:33.009934+0000","last_unstale":"2024-04-20T10:27:38.111488+0000","last_undegraded":"2024-04-20T10:27:38.111488+0000","last_fullsized":"2024-04-20T10:27:38.111488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392171+0000","last_change":"2024-04-20T10:27:33.006041+0000","last_active":"2024-04-20T10:27:38.392171+0000","last_peered":"2024-04-20T10:27:38.392171+0000","last_clean":"2024-04-20T10:27:38.392171+0000","last_became_active":"2024-04-20T10:27:33.005259+0000","last_became_peered":"2024-04-20T10:27:33.005259+0000","last_unstale":"2024-04-20T10:27:38.392171+0000","last_undegraded":"2024-04-20T10:27:38.392171+0000","last_fullsized":"2024-04-20T10:27:38.392171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112860+0000","last_change":"2024-04-20T10:27:33.015530+0000","last_active":"2024-04-20T10:27:38.112860+0000","last_peered":"2024-04-20T10:27:38.112860+0000","last_clean":"2024-04-20T10:27:38.112860+0000","last_became_active":"2024-04-20T10:27:33.015240+0000","last_became_peered":"2024-04-20T10:27:33.015240+0000","last_unstale":"2024-04-20T10:27:38.112860+0000","last_undegraded":"2024-04-20T10:27:38.112860+0000","last_fullsized":"2024-04-20T10:27:38.112860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105792+0000","last_change":"2024-04-20T10:27:33.015916+0000","last_active":"2024-04-20T10:27:38.105792+0000","last_peered":"2024-04-20T10:27:38.105792+0000","last_clean":"2024-04-20T10:27:38.105792+0000","last_became_active":"2024-04-20T10:27:33.015585+0000","last_became_peered":"2024-04-20T10:27:33.015585+0000","last_unstale":"2024-04-20T10:27:38.105792+0000","last_undegraded":"2024-04-20T10:27:38.105792+0000","last_fullsized":"2024-04-20T10:27:38.105792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112167+0000","last_change":"2024-04-20T10:27:33.014307+0000","last_active":"2024-04-20T10:27:38.112167+0000","last_peered":"2024-04-20T10:27:38.112167+0000","last_clean":"2024-04-20T10:27:38.112167+0000","last_became_active":"2024-04-20T10:27:33.013891+0000","last_became_peered":"2024-04-20T10:27:33.013891+0000","last_unstale":"2024-04-20T10:27:38.112167+0000","last_undegraded":"2024-04-20T10:27:38.112167+0000","last_fullsized":"2024-04-20T10:27:38.112167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:32.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390622+0000","last_change":"2024-04-20T10:27:33.010713+0000","last_active":"2024-04-20T10:27:38.390622+0000","last_peered":"2024-04-20T10:27:38.390622+0000","last_clean":"2024-04-20T10:27:38.390622+0000","last_became_active":"2024-04-20T10:27:33.009349+0000","last_became_peered":"2024-04-20T10:27:33.009349+0000","last_unstale":"2024-04-20T10:27:38.390622+0000","last_undegraded":"2024-04-20T10:27:38.390622+0000","last_fullsized":"2024-04-20T10:27:38.390622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389798+0000","last_change":"2024-04-20T10:27:33.018481+0000","last_active":"2024-04-20T10:27:38.389798+0000","last_peered":"2024-04-20T10:27:38.389798+0000","last_clean":"2024-04-20T10:27:38.389798+0000","last_became_active":"2024-04-20T10:27:33.015328+0000","last_became_peered":"2024-04-20T10:27:33.015328+0000","last_unstale":"2024-04-20T10:27:38.389798+0000","last_undegraded":"2024-04-20T10:27:38.389798+0000","last_fullsized":"2024-04-20T10:27:38.389798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:43:48.722150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388831+0000","last_change":"2024-04-20T10:27:33.006877+0000","last_active":"2024-04-20T10:27:38.388831+0000","last_peered":"2024-04-20T10:27:38.388831+0000","last_clean":"2024-04-20T10:27:38.388831+0000","last_became_active":"2024-04-20T10:27:33.006143+0000","last_became_peered":"2024-04-20T10:27:33.006143+0000","last_unstale":"2024-04-20T10:27:38.388831+0000","last_undegraded":"2024-04-20T10:27:38.388831+0000","last_fullsized":"2024-04-20T10:27:38.388831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108600+0000","last_change":"2024-04-20T10:27:33.016666+0000","last_active":"2024-04-20T10:27:38.108600+0000","last_peered":"2024-04-20T10:27:38.108600+0000","last_clean":"2024-04-20T10:27:38.108600+0000","last_became_active":"2024-04-20T10:27:33.016268+0000","last_became_peered":"2024-04-20T10:27:33.016268+0000","last_unstale":"2024-04-20T10:27:38.108600+0000","last_undegraded":"2024-04-20T10:27:38.108600+0000","last_fullsized":"2024-04-20T10:27:38.108600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:07.386901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392668+0000","last_change":"2024-04-20T10:27:33.014742+0000","last_active":"2024-04-20T10:27:38.392668+0000","last_peered":"2024-04-20T10:27:38.392668+0000","last_clean":"2024-04-20T10:27:38.392668+0000","last_became_active":"2024-04-20T10:27:33.014443+0000","last_became_peered":"2024-04-20T10:27:33.014443+0000","last_unstale":"2024-04-20T10:27:38.392668+0000","last_undegraded":"2024-04-20T10:27:38.392668+0000","last_fullsized":"2024-04-20T10:27:38.392668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108659+0000","last_change":"2024-04-20T10:27:33.002993+0000","last_active":"2024-04-20T10:27:38.108659+0000","last_peered":"2024-04-20T10:27:38.108659+0000","last_clean":"2024-04-20T10:27:38.108659+0000","last_became_active":"2024-04-20T10:27:33.002685+0000","last_became_peered":"2024-04-20T10:27:33.002685+0000","last_unstale":"2024-04-20T10:27:38.108659+0000","last_undegraded":"2024-04-20T10:27:38.108659+0000","last_fullsized":"2024-04-20T10:27:38.108659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:38.509857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105782+0000","last_change":"2024-04-20T10:27:33.014659+0000","last_active":"2024-04-20T10:27:38.105782+0000","last_peered":"2024-04-20T10:27:38.105782+0000","last_clean":"2024-04-20T10:27:38.105782+0000","last_became_active":"2024-04-20T10:27:33.014188+0000","last_became_peered":"2024-04-20T10:27:33.014188+0000","last_unstale":"2024-04-20T10:27:38.105782+0000","last_undegraded":"2024-04-20T10:27:38.105782+0000","last_fullsized":"2024-04-20T10:27:38.105782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107794+0000","last_change":"2024-04-20T10:27:33.011041+0000","last_active":"2024-04-20T10:27:38.107794+0000","last_peered":"2024-04-20T10:27:38.107794+0000","last_clean":"2024-04-20T10:27:38.107794+0000","last_became_active":"2024-04-20T10:27:33.010751+0000","last_became_peered":"2024-04-20T10:27:33.010751+0000","last_unstale":"2024-04-20T10:27:38.107794+0000","last_undegraded":"2024-04-20T10:27:38.107794+0000","last_fullsized":"2024-04-20T10:27:38.107794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110132+0000","last_change":"2024-04-20T10:27:33.005787+0000","last_active":"2024-04-20T10:27:38.110132+0000","last_peered":"2024-04-20T10:27:38.110132+0000","last_clean":"2024-04-20T10:27:38.110132+0000","last_became_active":"2024-04-20T10:27:33.004692+0000","last_became_peered":"2024-04-20T10:27:33.004692+0000","last_unstale":"2024-04-20T10:27:38.110132+0000","last_undegraded":"2024-04-20T10:27:38.110132+0000","last_fullsized":"2024-04-20T10:27:38.110132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:08.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112010+0000","last_change":"2024-04-20T10:27:33.010711+0000","last_active":"2024-04-20T10:27:38.112010+0000","last_peered":"2024-04-20T10:27:38.112010+0000","last_clean":"2024-04-20T10:27:38.112010+0000","last_became_active":"2024-04-20T10:27:33.010282+0000","last_became_peered":"2024-04-20T10:27:33.010282+0000","last_unstale":"2024-04-20T10:27:38.112010+0000","last_undegraded":"2024-04-20T10:27:38.112010+0000","last_fullsized":"2024-04-20T10:27:38.112010+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:06.224357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388721+0000","last_change":"2024-04-20T10:27:33.008425+0000","last_active":"2024-04-20T10:27:38.388721+0000","last_peered":"2024-04-20T10:27:38.388721+0000","last_clean":"2024-04-20T10:27:38.388721+0000","last_became_active":"2024-04-20T10:27:33.006532+0000","last_became_peered":"2024-04-20T10:27:33.006532+0000","last_unstale":"2024-04-20T10:27:38.388721+0000","last_undegraded":"2024-04-20T10:27:38.388721+0000","last_fullsized":"2024-04-20T10:27:38.388721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388300+0000","last_change":"2024-04-20T10:27:33.018484+0000","last_active":"2024-04-20T10:27:38.388300+0000","last_peered":"2024-04-20T10:27:38.388300+0000","last_clean":"2024-04-20T10:27:38.388300+0000","last_became_active":"2024-04-20T10:27:33.018257+0000","last_became_peered":"2024-04-20T10:27:33.018257+0000","last_unstale":"2024-04-20T10:27:38.388300+0000","last_undegraded":"2024-04-20T10:27:38.388300+0000","last_fullsized":"2024-04-20T10:27:38.388300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388715+0000","last_change":"2024-04-20T10:27:33.022995+0000","last_active":"2024-04-20T10:27:38.388715+0000","last_peered":"2024-04-20T10:27:38.388715+0000","last_clean":"2024-04-20T10:27:38.388715+0000","last_became_active":"2024-04-20T10:27:33.017740+0000","last_became_peered":"2024-04-20T10:27:33.017740+0000","last_unstale":"2024-04-20T10:27:38.388715+0000","last_undegraded":"2024-04-20T10:27:38.388715+0000","last_fullsized":"2024-04-20T10:27:38.388715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391590+0000","last_change":"2024-04-20T10:27:33.008312+0000","last_active":"2024-04-20T10:27:38.391590+0000","last_peered":"2024-04-20T10:27:38.391590+0000","last_clean":"2024-04-20T10:27:38.391590+0000","last_became_active":"2024-04-20T10:27:33.002597+0000","last_became_peered":"2024-04-20T10:27:33.002597+0000","last_unstale":"2024-04-20T10:27:38.391590+0000","last_undegraded":"2024-04-20T10:27:38.391590+0000","last_fullsized":"2024-04-20T10:27:38.391590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107489+0000","last_change":"2024-04-20T10:27:33.010109+0000","last_active":"2024-04-20T10:27:38.107489+0000","last_peered":"2024-04-20T10:27:38.107489+0000","last_clean":"2024-04-20T10:27:38.107489+0000","last_became_active":"2024-04-20T10:27:33.009838+0000","last_became_peered":"2024-04-20T10:27:33.009838+0000","last_unstale":"2024-04-20T10:27:38.107489+0000","last_undegraded":"2024-04-20T10:27:38.107489+0000","last_fullsized":"2024-04-20T10:27:38.107489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387903+0000","last_change":"2024-04-20T10:27:33.024641+0000","last_active":"2024-04-20T10:27:38.387903+0000","last_peered":"2024-04-20T10:27:38.387903+0000","last_clean":"2024-04-20T10:27:38.387903+0000","last_became_active":"2024-04-20T10:27:33.024410+0000","last_became_peered":"2024-04-20T10:27:33.024410+0000","last_unstale":"2024-04-20T10:27:38.387903+0000","last_undegraded":"2024-04-20T10:27:38.387903+0000","last_fullsized":"2024-04-20T10:27:38.387903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391138+0000","last_change":"2024-04-20T10:27:33.025226+0000","last_active":"2024-04-20T10:27:38.391138+0000","last_peered":"2024-04-20T10:27:38.391138+0000","last_clean":"2024-04-20T10:27:38.391138+0000","last_became_active":"2024-04-20T10:27:33.024311+0000","last_became_peered":"2024-04-20T10:27:33.024311+0000","last_unstale":"2024-04-20T10:27:38.391138+0000","last_undegraded":"2024-04-20T10:27:38.391138+0000","last_fullsized":"2024-04-20T10:27:38.391138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389207+0000","last_change":"2024-04-20T10:27:33.022586+0000","last_active":"2024-04-20T10:27:38.389207+0000","last_peered":"2024-04-20T10:27:38.389207+0000","last_clean":"2024-04-20T10:27:38.389207+0000","last_became_active":"2024-04-20T10:27:33.022293+0000","last_became_peered":"2024-04-20T10:27:33.022293+0000","last_unstale":"2024-04-20T10:27:38.389207+0000","last_undegraded":"2024-04-20T10:27:38.389207+0000","last_fullsized":"2024-04-20T10:27:38.389207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390946+0000","last_change":"2024-04-20T10:27:33.019615+0000","last_active":"2024-04-20T10:27:38.390946+0000","last_peered":"2024-04-20T10:27:38.390946+0000","last_clean":"2024-04-20T10:27:38.390946+0000","last_became_active":"2024-04-20T10:27:33.019288+0000","last_became_peered":"2024-04-20T10:27:33.019288+0000","last_unstale":"2024-04-20T10:27:38.390946+0000","last_undegraded":"2024-04-20T10:27:38.390946+0000","last_fullsized":"2024-04-20T10:27:38.390946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111457+0000","last_change":"2024-04-20T10:27:33.019815+0000","last_active":"2024-04-20T10:27:38.111457+0000","last_peered":"2024-04-20T10:27:38.111457+0000","last_clean":"2024-04-20T10:27:38.111457+0000","last_became_active":"2024-04-20T10:27:33.019369+0000","last_became_peered":"2024-04-20T10:27:33.019369+0000","last_unstale":"2024-04-20T10:27:38.111457+0000","last_undegraded":"2024-04-20T10:27:38.111457+0000","last_fullsized":"2024-04-20T10:27:38.111457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112358+0000","last_change":"2024-04-20T10:27:33.015773+0000","last_active":"2024-04-20T10:27:38.112358+0000","last_peered":"2024-04-20T10:27:38.112358+0000","last_clean":"2024-04-20T10:27:38.112358+0000","last_became_active":"2024-04-20T10:27:33.015430+0000","last_became_peered":"2024-04-20T10:27:33.015430+0000","last_unstale":"2024-04-20T10:27:38.112358+0000","last_undegraded":"2024-04-20T10:27:38.112358+0000","last_fullsized":"2024-04-20T10:27:38.112358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389282+0000","last_change":"2024-04-20T10:27:33.024105+0000","last_active":"2024-04-20T10:27:38.389282+0000","last_peered":"2024-04-20T10:27:38.389282+0000","last_clean":"2024-04-20T10:27:38.389282+0000","last_became_active":"2024-04-20T10:27:33.023418+0000","last_became_peered":"2024-04-20T10:27:33.023418+0000","last_unstale":"2024-04-20T10:27:38.389282+0000","last_undegraded":"2024-04-20T10:27:38.389282+0000","last_fullsized":"2024-04-20T10:27:38.389282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":110867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":700723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":114799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:27:55.651 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:27:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.903+0000 7ff857e00640 1 Processor -- start 2024-04-20T10:27:55.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.903+0000 7ff857e00640 1 -- start start 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff857e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff85815a1f0 0x7ff858072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff857e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff858072190 0x7ff858073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff857e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff85815cc90 con 0x7ff858159e10 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff857e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff858072900 con 0x7ff85815a1f0 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff857e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff85815ca40 con 0x7ff858072190 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff84fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff85815a1f0 0x7ff858072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff856a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff858159e10 0x7ff858071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33598/0 (socket says 172.21.15.47:33598) 2024-04-20T10:27:55.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff856a00640 1 -- 172.21.15.47:0/2707361921 learned_addr learned my addr 172.21.15.47:0/2707361921 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.905+0000 7ff84fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff85815a1f0 0x7ff858072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55770/0 (socket says 172.21.15.47:55770) 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.906+0000 7ff857400640 1 --2- 172.21.15.47:0/2707361921 >> v2:172.21.15.175:3300/0 conn(0x7ff858072190 0x7ff858073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.906+0000 7ff84fe00640 1 -- 172.21.15.47:0/2707361921 >> v2:172.21.15.175:3300/0 conn(0x7ff858072190 msgr2=0x7ff858073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.906+0000 7ff84fe00640 1 --2- 172.21.15.47:0/2707361921 >> v2:172.21.15.175:3300/0 conn(0x7ff858072190 0x7ff858073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.906+0000 7ff84fe00640 1 -- 172.21.15.47:0/2707361921 >> v1:172.21.15.47:6789/0 conn(0x7ff858159e10 legacy=0x7ff858071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.906+0000 7ff84fe00640 1 -- 172.21.15.47:0/2707361921 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff85815cf70 con 0x7ff85815a1f0 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.906+0000 7ff856000640 1 -- 172.21.15.47:0/2707361921 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10947982 0 0) 0x7ff85815cc90 con 0x7ff858159e10 2024-04-20T10:27:55.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.907+0000 7ff84fe00640 1 --2- 172.21.15.47:0/2707361921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85815a1f0 0x7ff858072ee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff85815daf0 tx=0x7ff84002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d68f7d5ff2a5ca53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:55.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.907+0000 7ff856000640 1 -- 172.21.15.47:0/2707361921 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840035020 con 0x7ff85815a1f0 2024-04-20T10:27:55.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff856000640 1 -- 172.21.15.47:0/2707361921 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff840031830 con 0x7ff85815a1f0 2024-04-20T10:27:55.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff856000640 1 -- 172.21.15.47:0/2707361921 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff840031c00 con 0x7ff85815a1f0 2024-04-20T10:27:55.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 -- 172.21.15.47:0/2707361921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85815a1f0 msgr2=0x7ff858072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 --2- 172.21.15.47:0/2707361921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85815a1f0 0x7ff858072ee0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff85815daf0 tx=0x7ff84002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 -- 172.21.15.47:0/2707361921 shutdown_connections 2024-04-20T10:27:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 --2- 172.21.15.47:0/2707361921 >> v2:172.21.15.175:3300/0 conn(0x7ff858072190 0x7ff858073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 --2- 172.21.15.47:0/2707361921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff85815a1f0 0x7ff858072ee0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 -- 172.21.15.47:0/2707361921 >> 172.21.15.47:0/2707361921 conn(0x7ff8580d0040 msgr2=0x7ff8580d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 -- 172.21.15.47:0/2707361921 shutdown_connections 2024-04-20T10:27:55.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.908+0000 7ff857e00640 1 -- 172.21.15.47:0/2707361921 wait complete. 2024-04-20T10:27:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.910+0000 7ff857e00640 1 Processor -- start 2024-04-20T10:27:55.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.910+0000 7ff857e00640 1 -- start start 2024-04-20T10:27:55.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff857e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff858072190 0x7ff8580b4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff857e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff858159e10 0x7ff85815be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff857e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85815a1f0 0x7ff858145c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff857e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff85815d810 con 0x7ff858072190 2024-04-20T10:27:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff857e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff85815d530 con 0x7ff858159e10 2024-04-20T10:27:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff857e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff85806da90 con 0x7ff85815a1f0 2024-04-20T10:27:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff856a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff858072190 0x7ff8580b4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff856a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff858072190 0x7ff8580b4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55772/0 (socket says 172.21.15.47:55772) 2024-04-20T10:27:55.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.911+0000 7ff856a00640 1 -- 172.21.15.47:0/3373539010 learned_addr learned my addr 172.21.15.47:0/3373539010 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff857400640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85815a1f0 0x7ff858145c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff84fe00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff858159e10 0x7ff85815be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff856a00640 1 -- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85815a1f0 msgr2=0x7ff858145c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff856a00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85815a1f0 0x7ff858145c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff856a00640 1 -- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff858159e10 msgr2=0x7ff85815be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff856a00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff858159e10 0x7ff85815be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff856a00640 1 -- 172.21.15.47:0/3373539010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8580b3be0 con 0x7ff858072190 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff84fe00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff858159e10 0x7ff85815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:55.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff857400640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85815a1f0 0x7ff858145c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff856a00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff858072190 0x7ff8580b4c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c4d0 tx=0x7ff84800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.912+0000 7ff84ea00640 1 -- 172.21.15.47:0/3373539010 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848018070 con 0x7ff858072190 2024-04-20T10:27:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.913+0000 7ff84ea00640 1 -- 172.21.15.47:0/3373539010 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff848007500 con 0x7ff858072190 2024-04-20T10:27:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.913+0000 7ff84ea00640 1 -- 172.21.15.47:0/3373539010 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff848004a20 con 0x7ff858072190 2024-04-20T10:27:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.913+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8580b3ec0 con 0x7ff858072190 2024-04-20T10:27:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.913+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8580b4480 con 0x7ff858072190 2024-04-20T10:27:55.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.914+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff858075b60 con 0x7ff858072190 2024-04-20T10:27:55.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.915+0000 7ff84ea00640 1 -- 172.21.15.47:0/3373539010 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff848004bc0 con 0x7ff858072190 2024-04-20T10:27:55.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.916+0000 7ff84ea00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff83406c920 0x7ff83406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:55.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.917+0000 7ff84ea00640 1 -- 172.21.15.47:0/3373539010 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff84808b140 con 0x7ff858072190 2024-04-20T10:27:55.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.917+0000 7ff84fe00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff83406c920 0x7ff83406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:55.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.918+0000 7ff84fe00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff83406c920 0x7ff83406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff840036000 tx=0x7ff840008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:55.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:55.921+0000 7ff84ea00640 1 -- 172.21.15.47:0/3373539010 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff848059220 con 0x7ff858072190 2024-04-20T10:27:56.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.133+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8580afe90 con 0x7ff83406c920 2024-04-20T10:27:56.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.158+0000 7ff84ea00640 1 -- 172.21.15.47:0/3373539010 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332566 (secure 0 0 0) 0x7ff8580afe90 con 0x7ff83406c920 2024-04-20T10:27:56.164 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:56.165 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:27:56.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff83406c920 msgr2=0x7ff83406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff83406c920 0x7ff83406ede0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff840036000 tx=0x7ff840008030 comp rx=0 tx=0).stop 2024-04-20T10:27:56.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff858072190 msgr2=0x7ff8580b4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff858072190 0x7ff8580b4c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c4d0 tx=0x7ff84800c990 comp rx=0 tx=0).stop 2024-04-20T10:27:56.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 shutdown_connections 2024-04-20T10:27:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff85815a1f0 0x7ff858145c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff858159e10 0x7ff85815be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff83406c920 0x7ff83406ede0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 --2- 172.21.15.47:0/3373539010 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff858072190 0x7ff8580b4c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 >> 172.21.15.47:0/3373539010 conn(0x7ff8580d0040 msgr2=0x7ff858071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 shutdown_connections 2024-04-20T10:27:56.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.164+0000 7ff857e00640 1 -- 172.21.15.47:0/3373539010 wait complete. 2024-04-20T10:27:56.185 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2024-04-20T10:27:55.071410+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594867,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227932,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746788,"statfs":{"total":773094113280,"available":772860710912,"internally_reserved":0,"allocated":3514368,"data_stored":2072155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2089,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005176"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392064+0000","last_change":"2024-04-20T10:27:32.090082+0000","last_active":"2024-04-20T10:27:38.392064+0000","last_peered":"2024-04-20T10:27:38.392064+0000","last_clean":"2024-04-20T10:27:38.392064+0000","last_became_active":"2024-04-20T10:27:32.089819+0000","last_became_peered":"2024-04-20T10:27:32.089819+0000","last_unstale":"2024-04-20T10:27:38.392064+0000","last_undegraded":"2024-04-20T10:27:38.392064+0000","last_fullsized":"2024-04-20T10:27:38.392064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:16.568087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112567+0000","last_change":"2024-04-20T10:27:33.015363+0000","last_active":"2024-04-20T10:27:38.112567+0000","last_peered":"2024-04-20T10:27:38.112567+0000","last_clean":"2024-04-20T10:27:38.112567+0000","last_became_active":"2024-04-20T10:27:33.013421+0000","last_became_peered":"2024-04-20T10:27:33.013421+0000","last_unstale":"2024-04-20T10:27:38.112567+0000","last_undegraded":"2024-04-20T10:27:38.112567+0000","last_fullsized":"2024-04-20T10:27:38.112567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391215+0000","last_change":"2024-04-20T10:27:37.132637+0000","last_active":"2024-04-20T10:27:38.391215+0000","last_peered":"2024-04-20T10:27:38.391215+0000","last_clean":"2024-04-20T10:27:38.391215+0000","last_became_active":"2024-04-20T10:27:37.132147+0000","last_became_peered":"2024-04-20T10:27:37.132147+0000","last_unstale":"2024-04-20T10:27:38.391215+0000","last_undegraded":"2024-04-20T10:27:38.391215+0000","last_fullsized":"2024-04-20T10:27:38.391215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:13.081027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112843+0000","last_change":"2024-04-20T10:27:32.093624+0000","last_active":"2024-04-20T10:27:38.112843+0000","last_peered":"2024-04-20T10:27:38.112843+0000","last_clean":"2024-04-20T10:27:38.112843+0000","last_became_active":"2024-04-20T10:27:32.093349+0000","last_became_peered":"2024-04-20T10:27:32.093349+0000","last_unstale":"2024-04-20T10:27:38.112843+0000","last_undegraded":"2024-04-20T10:27:38.112843+0000","last_fullsized":"2024-04-20T10:27:38.112843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113178+0000","last_change":"2024-04-20T10:27:33.019078+0000","last_active":"2024-04-20T10:27:38.113178+0000","last_peered":"2024-04-20T10:27:38.113178+0000","last_clean":"2024-04-20T10:27:38.113178+0000","last_became_active":"2024-04-20T10:27:33.018362+0000","last_became_peered":"2024-04-20T10:27:33.018362+0000","last_unstale":"2024-04-20T10:27:38.113178+0000","last_undegraded":"2024-04-20T10:27:38.113178+0000","last_fullsized":"2024-04-20T10:27:38.113178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390250+0000","last_change":"2024-04-20T10:27:37.118407+0000","last_active":"2024-04-20T10:27:38.390250+0000","last_peered":"2024-04-20T10:27:38.390250+0000","last_clean":"2024-04-20T10:27:38.390250+0000","last_became_active":"2024-04-20T10:27:37.117850+0000","last_became_peered":"2024-04-20T10:27:37.117850+0000","last_unstale":"2024-04-20T10:27:38.390250+0000","last_undegraded":"2024-04-20T10:27:38.390250+0000","last_fullsized":"2024-04-20T10:27:38.390250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:36.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388629+0000","last_change":"2024-04-20T10:27:32.080920+0000","last_active":"2024-04-20T10:27:38.388629+0000","last_peered":"2024-04-20T10:27:38.388629+0000","last_clean":"2024-04-20T10:27:38.388629+0000","last_became_active":"2024-04-20T10:27:32.080614+0000","last_became_peered":"2024-04-20T10:27:32.080614+0000","last_unstale":"2024-04-20T10:27:38.388629+0000","last_undegraded":"2024-04-20T10:27:38.388629+0000","last_fullsized":"2024-04-20T10:27:38.388629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:51.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392440+0000","last_change":"2024-04-20T10:27:33.006318+0000","last_active":"2024-04-20T10:27:38.392440+0000","last_peered":"2024-04-20T10:27:38.392440+0000","last_clean":"2024-04-20T10:27:38.392440+0000","last_became_active":"2024-04-20T10:27:33.006040+0000","last_became_peered":"2024-04-20T10:27:33.006040+0000","last_unstale":"2024-04-20T10:27:38.392440+0000","last_undegraded":"2024-04-20T10:27:38.392440+0000","last_fullsized":"2024-04-20T10:27:38.392440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391743+0000","last_change":"2024-04-20T10:27:37.123688+0000","last_active":"2024-04-20T10:27:38.391743+0000","last_peered":"2024-04-20T10:27:38.391743+0000","last_clean":"2024-04-20T10:27:38.391743+0000","last_became_active":"2024-04-20T10:27:37.123316+0000","last_became_peered":"2024-04-20T10:27:37.123316+0000","last_unstale":"2024-04-20T10:27:38.391743+0000","last_undegraded":"2024-04-20T10:27:38.391743+0000","last_fullsized":"2024-04-20T10:27:38.391743+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388939+0000","last_change":"2024-04-20T10:27:32.082282+0000","last_active":"2024-04-20T10:27:38.388939+0000","last_peered":"2024-04-20T10:27:38.388939+0000","last_clean":"2024-04-20T10:27:38.388939+0000","last_became_active":"2024-04-20T10:27:32.081943+0000","last_became_peered":"2024-04-20T10:27:32.081943+0000","last_unstale":"2024-04-20T10:27:38.388939+0000","last_undegraded":"2024-04-20T10:27:38.388939+0000","last_fullsized":"2024-04-20T10:27:38.388939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:12.394383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108312+0000","last_change":"2024-04-20T10:27:33.003700+0000","last_active":"2024-04-20T10:27:38.108312+0000","last_peered":"2024-04-20T10:27:38.108312+0000","last_clean":"2024-04-20T10:27:38.108312+0000","last_became_active":"2024-04-20T10:27:33.003443+0000","last_became_peered":"2024-04-20T10:27:33.003443+0000","last_unstale":"2024-04-20T10:27:38.108312+0000","last_undegraded":"2024-04-20T10:27:38.108312+0000","last_fullsized":"2024-04-20T10:27:38.108312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:33.760650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390259+0000","last_change":"2024-04-20T10:27:37.127775+0000","last_active":"2024-04-20T10:27:38.390259+0000","last_peered":"2024-04-20T10:27:38.390259+0000","last_clean":"2024-04-20T10:27:38.390259+0000","last_became_active":"2024-04-20T10:27:37.126837+0000","last_became_peered":"2024-04-20T10:27:37.126837+0000","last_unstale":"2024-04-20T10:27:38.390259+0000","last_undegraded":"2024-04-20T10:27:38.390259+0000","last_fullsized":"2024-04-20T10:27:38.390259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:31.369897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106458+0000","last_change":"2024-04-20T10:27:33.004497+0000","last_active":"2024-04-20T10:27:38.106458+0000","last_peered":"2024-04-20T10:27:38.106458+0000","last_clean":"2024-04-20T10:27:38.106458+0000","last_became_active":"2024-04-20T10:27:33.004228+0000","last_became_peered":"2024-04-20T10:27:33.004228+0000","last_unstale":"2024-04-20T10:27:38.106458+0000","last_undegraded":"2024-04-20T10:27:38.106458+0000","last_fullsized":"2024-04-20T10:27:38.106458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:04.936311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106698+0000","last_change":"2024-04-20T10:27:32.069803+0000","last_active":"2024-04-20T10:27:38.106698+0000","last_peered":"2024-04-20T10:27:38.106698+0000","last_clean":"2024-04-20T10:27:38.106698+0000","last_became_active":"2024-04-20T10:27:32.069542+0000","last_became_peered":"2024-04-20T10:27:32.069542+0000","last_unstale":"2024-04-20T10:27:38.106698+0000","last_undegraded":"2024-04-20T10:27:38.106698+0000","last_fullsized":"2024-04-20T10:27:38.106698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107502+0000","last_change":"2024-04-20T10:27:37.122386+0000","last_active":"2024-04-20T10:27:38.107502+0000","last_peered":"2024-04-20T10:27:38.107502+0000","last_clean":"2024-04-20T10:27:38.107502+0000","last_became_active":"2024-04-20T10:27:37.121943+0000","last_became_peered":"2024-04-20T10:27:37.121943+0000","last_unstale":"2024-04-20T10:27:38.107502+0000","last_undegraded":"2024-04-20T10:27:38.107502+0000","last_fullsized":"2024-04-20T10:27:38.107502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:53:31.071316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391786+0000","last_change":"2024-04-20T10:27:33.015242+0000","last_active":"2024-04-20T10:27:38.391786+0000","last_peered":"2024-04-20T10:27:38.391786+0000","last_clean":"2024-04-20T10:27:38.391786+0000","last_became_active":"2024-04-20T10:27:33.014003+0000","last_became_peered":"2024-04-20T10:27:33.014003+0000","last_unstale":"2024-04-20T10:27:38.391786+0000","last_undegraded":"2024-04-20T10:27:38.391786+0000","last_fullsized":"2024-04-20T10:27:38.391786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110379+0000","last_change":"2024-04-20T10:27:32.087085+0000","last_active":"2024-04-20T10:27:38.110379+0000","last_peered":"2024-04-20T10:27:38.110379+0000","last_clean":"2024-04-20T10:27:38.110379+0000","last_became_active":"2024-04-20T10:27:32.086777+0000","last_became_peered":"2024-04-20T10:27:32.086777+0000","last_unstale":"2024-04-20T10:27:38.110379+0000","last_undegraded":"2024-04-20T10:27:38.110379+0000","last_fullsized":"2024-04-20T10:27:38.110379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388751+0000","last_change":"2024-04-20T10:27:37.123374+0000","last_active":"2024-04-20T10:27:38.388751+0000","last_peered":"2024-04-20T10:27:38.388751+0000","last_clean":"2024-04-20T10:27:38.388751+0000","last_became_active":"2024-04-20T10:27:37.118855+0000","last_became_peered":"2024-04-20T10:27:37.118855+0000","last_unstale":"2024-04-20T10:27:38.388751+0000","last_undegraded":"2024-04-20T10:27:38.388751+0000","last_fullsized":"2024-04-20T10:27:38.388751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389358+0000","last_change":"2024-04-20T10:27:33.013100+0000","last_active":"2024-04-20T10:27:38.389358+0000","last_peered":"2024-04-20T10:27:38.389358+0000","last_clean":"2024-04-20T10:27:38.389358+0000","last_became_active":"2024-04-20T10:27:33.012720+0000","last_became_peered":"2024-04-20T10:27:33.012720+0000","last_unstale":"2024-04-20T10:27:38.389358+0000","last_undegraded":"2024-04-20T10:27:38.389358+0000","last_fullsized":"2024-04-20T10:27:38.389358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:52.950066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390484+0000","last_change":"2024-04-20T10:27:32.082330+0000","last_active":"2024-04-20T10:27:38.390484+0000","last_peered":"2024-04-20T10:27:38.390484+0000","last_clean":"2024-04-20T10:27:38.390484+0000","last_became_active":"2024-04-20T10:27:32.080602+0000","last_became_peered":"2024-04-20T10:27:32.080602+0000","last_unstale":"2024-04-20T10:27:38.390484+0000","last_undegraded":"2024-04-20T10:27:38.390484+0000","last_fullsized":"2024-04-20T10:27:38.390484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:59.067061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392568+0000","last_change":"2024-04-20T10:27:37.125949+0000","last_active":"2024-04-20T10:27:38.392568+0000","last_peered":"2024-04-20T10:27:38.392568+0000","last_clean":"2024-04-20T10:27:38.392568+0000","last_became_active":"2024-04-20T10:27:37.125481+0000","last_became_peered":"2024-04-20T10:27:37.125481+0000","last_unstale":"2024-04-20T10:27:38.392568+0000","last_undegraded":"2024-04-20T10:27:38.392568+0000","last_fullsized":"2024-04-20T10:27:38.392568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:15.992852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105786+0000","last_change":"2024-04-20T10:27:33.015612+0000","last_active":"2024-04-20T10:27:38.105786+0000","last_peered":"2024-04-20T10:27:38.105786+0000","last_clean":"2024-04-20T10:27:38.105786+0000","last_became_active":"2024-04-20T10:27:33.014370+0000","last_became_peered":"2024-04-20T10:27:33.014370+0000","last_unstale":"2024-04-20T10:27:38.105786+0000","last_undegraded":"2024-04-20T10:27:38.105786+0000","last_fullsized":"2024-04-20T10:27:38.105786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387559+0000","last_change":"2024-04-20T10:27:32.068081+0000","last_active":"2024-04-20T10:27:38.387559+0000","last_peered":"2024-04-20T10:27:38.387559+0000","last_clean":"2024-04-20T10:27:38.387559+0000","last_became_active":"2024-04-20T10:27:32.067727+0000","last_became_peered":"2024-04-20T10:27:32.067727+0000","last_unstale":"2024-04-20T10:27:38.387559+0000","last_undegraded":"2024-04-20T10:27:38.387559+0000","last_fullsized":"2024-04-20T10:27:38.387559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:59.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392244+0000","last_change":"2024-04-20T10:27:37.119884+0000","last_active":"2024-04-20T10:27:38.392244+0000","last_peered":"2024-04-20T10:27:38.392244+0000","last_clean":"2024-04-20T10:27:38.392244+0000","last_became_active":"2024-04-20T10:27:37.119461+0000","last_became_peered":"2024-04-20T10:27:37.119461+0000","last_unstale":"2024-04-20T10:27:38.392244+0000","last_undegraded":"2024-04-20T10:27:38.392244+0000","last_fullsized":"2024-04-20T10:27:38.392244+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391977+0000","last_change":"2024-04-20T10:27:33.009139+0000","last_active":"2024-04-20T10:27:38.391977+0000","last_peered":"2024-04-20T10:27:38.391977+0000","last_clean":"2024-04-20T10:27:38.391977+0000","last_became_active":"2024-04-20T10:27:33.008830+0000","last_became_peered":"2024-04-20T10:27:33.008830+0000","last_unstale":"2024-04-20T10:27:38.391977+0000","last_undegraded":"2024-04-20T10:27:38.391977+0000","last_fullsized":"2024-04-20T10:27:38.391977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:06.157535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108669+0000","last_change":"2024-04-20T10:27:32.086356+0000","last_active":"2024-04-20T10:27:38.108669+0000","last_peered":"2024-04-20T10:27:38.108669+0000","last_clean":"2024-04-20T10:27:38.108669+0000","last_became_active":"2024-04-20T10:27:32.077640+0000","last_became_peered":"2024-04-20T10:27:32.077640+0000","last_unstale":"2024-04-20T10:27:38.108669+0000","last_undegraded":"2024-04-20T10:27:38.108669+0000","last_fullsized":"2024-04-20T10:27:38.108669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:44.469214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390797+0000","last_change":"2024-04-20T10:27:37.124270+0000","last_active":"2024-04-20T10:27:38.390797+0000","last_peered":"2024-04-20T10:27:38.390797+0000","last_clean":"2024-04-20T10:27:38.390797+0000","last_became_active":"2024-04-20T10:27:37.123953+0000","last_became_peered":"2024-04-20T10:27:37.123953+0000","last_unstale":"2024-04-20T10:27:38.390797+0000","last_undegraded":"2024-04-20T10:27:38.390797+0000","last_fullsized":"2024-04-20T10:27:38.390797+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:27.841628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390487+0000","last_change":"2024-04-20T10:27:33.017825+0000","last_active":"2024-04-20T10:27:38.390487+0000","last_peered":"2024-04-20T10:27:38.390487+0000","last_clean":"2024-04-20T10:27:38.390487+0000","last_became_active":"2024-04-20T10:27:33.015635+0000","last_became_peered":"2024-04-20T10:27:33.015635+0000","last_unstale":"2024-04-20T10:27:38.390487+0000","last_undegraded":"2024-04-20T10:27:38.390487+0000","last_fullsized":"2024-04-20T10:27:38.390487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:58.914910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392232+0000","last_change":"2024-04-20T10:27:32.082908+0000","last_active":"2024-04-20T10:27:38.392232+0000","last_peered":"2024-04-20T10:27:38.392232+0000","last_clean":"2024-04-20T10:27:38.392232+0000","last_became_active":"2024-04-20T10:27:32.082507+0000","last_became_peered":"2024-04-20T10:27:32.082507+0000","last_unstale":"2024-04-20T10:27:38.392232+0000","last_undegraded":"2024-04-20T10:27:38.392232+0000","last_fullsized":"2024-04-20T10:27:38.392232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:52.804365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390908+0000","last_change":"2024-04-20T10:27:37.126093+0000","last_active":"2024-04-20T10:27:38.390908+0000","last_peered":"2024-04-20T10:27:38.390908+0000","last_clean":"2024-04-20T10:27:38.390908+0000","last_became_active":"2024-04-20T10:27:37.125681+0000","last_became_peered":"2024-04-20T10:27:37.125681+0000","last_unstale":"2024-04-20T10:27:38.390908+0000","last_undegraded":"2024-04-20T10:27:38.390908+0000","last_fullsized":"2024-04-20T10:27:38.390908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:13:36.622877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109415+0000","last_change":"2024-04-20T10:27:33.011662+0000","last_active":"2024-04-20T10:27:38.109415+0000","last_peered":"2024-04-20T10:27:38.109415+0000","last_clean":"2024-04-20T10:27:38.109415+0000","last_became_active":"2024-04-20T10:27:33.010842+0000","last_became_peered":"2024-04-20T10:27:33.010842+0000","last_unstale":"2024-04-20T10:27:38.109415+0000","last_undegraded":"2024-04-20T10:27:38.109415+0000","last_fullsized":"2024-04-20T10:27:38.109415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108664+0000","last_change":"2024-04-20T10:27:32.083502+0000","last_active":"2024-04-20T10:27:38.108664+0000","last_peered":"2024-04-20T10:27:38.108664+0000","last_clean":"2024-04-20T10:27:38.108664+0000","last_became_active":"2024-04-20T10:27:32.083217+0000","last_became_peered":"2024-04-20T10:27:32.083217+0000","last_unstale":"2024-04-20T10:27:38.108664+0000","last_undegraded":"2024-04-20T10:27:38.108664+0000","last_fullsized":"2024-04-20T10:27:38.108664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:41.953724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109193+0000","last_change":"2024-04-20T10:27:37.110995+0000","last_active":"2024-04-20T10:27:38.109193+0000","last_peered":"2024-04-20T10:27:38.109193+0000","last_clean":"2024-04-20T10:27:38.109193+0000","last_became_active":"2024-04-20T10:27:37.110619+0000","last_became_peered":"2024-04-20T10:27:37.110619+0000","last_unstale":"2024-04-20T10:27:38.109193+0000","last_undegraded":"2024-04-20T10:27:38.109193+0000","last_fullsized":"2024-04-20T10:27:38.109193+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:33.173033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391490+0000","last_change":"2024-04-20T10:27:32.997361+0000","last_active":"2024-04-20T10:27:38.391490+0000","last_peered":"2024-04-20T10:27:38.391490+0000","last_clean":"2024-04-20T10:27:38.391490+0000","last_became_active":"2024-04-20T10:27:32.996366+0000","last_became_peered":"2024-04-20T10:27:32.996366+0000","last_unstale":"2024-04-20T10:27:38.391490+0000","last_undegraded":"2024-04-20T10:27:38.391490+0000","last_fullsized":"2024-04-20T10:27:38.391490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110460+0000","last_change":"2024-04-20T10:27:32.087311+0000","last_active":"2024-04-20T10:27:38.110460+0000","last_peered":"2024-04-20T10:27:38.110460+0000","last_clean":"2024-04-20T10:27:38.110460+0000","last_became_active":"2024-04-20T10:27:32.086959+0000","last_became_peered":"2024-04-20T10:27:32.086959+0000","last_unstale":"2024-04-20T10:27:38.110460+0000","last_undegraded":"2024-04-20T10:27:38.110460+0000","last_fullsized":"2024-04-20T10:27:38.110460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110080+0000","last_change":"2024-04-20T10:27:37.121913+0000","last_active":"2024-04-20T10:27:38.110080+0000","last_peered":"2024-04-20T10:27:38.110080+0000","last_clean":"2024-04-20T10:27:38.110080+0000","last_became_active":"2024-04-20T10:27:37.121491+0000","last_became_peered":"2024-04-20T10:27:37.121491+0000","last_unstale":"2024-04-20T10:27:38.110080+0000","last_undegraded":"2024-04-20T10:27:38.110080+0000","last_fullsized":"2024-04-20T10:27:38.110080+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:28.925537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388610+0000","last_change":"2024-04-20T10:27:33.004607+0000","last_active":"2024-04-20T10:27:38.388610+0000","last_peered":"2024-04-20T10:27:38.388610+0000","last_clean":"2024-04-20T10:27:38.388610+0000","last_became_active":"2024-04-20T10:27:33.004299+0000","last_became_peered":"2024-04-20T10:27:33.004299+0000","last_unstale":"2024-04-20T10:27:38.388610+0000","last_undegraded":"2024-04-20T10:27:38.388610+0000","last_fullsized":"2024-04-20T10:27:38.388610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107052+0000","last_change":"2024-04-20T10:27:32.082735+0000","last_active":"2024-04-20T10:27:38.107052+0000","last_peered":"2024-04-20T10:27:38.107052+0000","last_clean":"2024-04-20T10:27:38.107052+0000","last_became_active":"2024-04-20T10:27:32.079695+0000","last_became_peered":"2024-04-20T10:27:32.079695+0000","last_unstale":"2024-04-20T10:27:38.107052+0000","last_undegraded":"2024-04-20T10:27:38.107052+0000","last_fullsized":"2024-04-20T10:27:38.107052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:07.962987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109079+0000","last_change":"2024-04-20T10:27:37.131952+0000","last_active":"2024-04-20T10:27:38.109079+0000","last_peered":"2024-04-20T10:27:38.109079+0000","last_clean":"2024-04-20T10:27:38.109079+0000","last_became_active":"2024-04-20T10:27:37.130721+0000","last_became_peered":"2024-04-20T10:27:37.130721+0000","last_unstale":"2024-04-20T10:27:38.109079+0000","last_undegraded":"2024-04-20T10:27:38.109079+0000","last_fullsized":"2024-04-20T10:27:38.109079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107777+0000","last_change":"2024-04-20T10:27:33.013537+0000","last_active":"2024-04-20T10:27:38.107777+0000","last_peered":"2024-04-20T10:27:38.107777+0000","last_clean":"2024-04-20T10:27:38.107777+0000","last_became_active":"2024-04-20T10:27:33.013223+0000","last_became_peered":"2024-04-20T10:27:33.013223+0000","last_unstale":"2024-04-20T10:27:38.107777+0000","last_undegraded":"2024-04-20T10:27:38.107777+0000","last_fullsized":"2024-04-20T10:27:38.107777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391320+0000","last_change":"2024-04-20T10:27:32.086422+0000","last_active":"2024-04-20T10:27:38.391320+0000","last_peered":"2024-04-20T10:27:38.391320+0000","last_clean":"2024-04-20T10:27:38.391320+0000","last_became_active":"2024-04-20T10:27:32.085353+0000","last_became_peered":"2024-04-20T10:27:32.085353+0000","last_unstale":"2024-04-20T10:27:38.391320+0000","last_undegraded":"2024-04-20T10:27:38.391320+0000","last_fullsized":"2024-04-20T10:27:38.391320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112264+0000","last_change":"2024-04-20T10:27:37.113304+0000","last_active":"2024-04-20T10:27:38.112264+0000","last_peered":"2024-04-20T10:27:38.112264+0000","last_clean":"2024-04-20T10:27:38.112264+0000","last_became_active":"2024-04-20T10:27:37.112828+0000","last_became_peered":"2024-04-20T10:27:37.112828+0000","last_unstale":"2024-04-20T10:27:38.112264+0000","last_undegraded":"2024-04-20T10:27:38.112264+0000","last_fullsized":"2024-04-20T10:27:38.112264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:19.178189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108899+0000","last_change":"2024-04-20T10:27:32.989233+0000","last_active":"2024-04-20T10:27:38.108899+0000","last_peered":"2024-04-20T10:27:38.108899+0000","last_clean":"2024-04-20T10:27:38.108899+0000","last_became_active":"2024-04-20T10:27:32.988092+0000","last_became_peered":"2024-04-20T10:27:32.988092+0000","last_unstale":"2024-04-20T10:27:38.108899+0000","last_undegraded":"2024-04-20T10:27:38.108899+0000","last_fullsized":"2024-04-20T10:27:38.108899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:11.889218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106243+0000","last_change":"2024-04-20T10:27:32.092257+0000","last_active":"2024-04-20T10:27:38.106243+0000","last_peered":"2024-04-20T10:27:38.106243+0000","last_clean":"2024-04-20T10:27:38.106243+0000","last_became_active":"2024-04-20T10:27:32.091944+0000","last_became_peered":"2024-04-20T10:27:32.091944+0000","last_unstale":"2024-04-20T10:27:38.106243+0000","last_undegraded":"2024-04-20T10:27:38.106243+0000","last_fullsized":"2024-04-20T10:27:38.106243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:56.139394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112022+0000","last_change":"2024-04-20T10:27:37.124420+0000","last_active":"2024-04-20T10:27:38.112022+0000","last_peered":"2024-04-20T10:27:38.112022+0000","last_clean":"2024-04-20T10:27:38.112022+0000","last_became_active":"2024-04-20T10:27:37.124048+0000","last_became_peered":"2024-04-20T10:27:37.124048+0000","last_unstale":"2024-04-20T10:27:38.112022+0000","last_undegraded":"2024-04-20T10:27:38.112022+0000","last_fullsized":"2024-04-20T10:27:38.112022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:37.864284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390325+0000","last_change":"2024-04-20T10:27:33.016860+0000","last_active":"2024-04-20T10:27:38.390325+0000","last_peered":"2024-04-20T10:27:38.390325+0000","last_clean":"2024-04-20T10:27:38.390325+0000","last_became_active":"2024-04-20T10:27:33.016323+0000","last_became_peered":"2024-04-20T10:27:33.016323+0000","last_unstale":"2024-04-20T10:27:38.390325+0000","last_undegraded":"2024-04-20T10:27:38.390325+0000","last_fullsized":"2024-04-20T10:27:38.390325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:11:39.053198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107037+0000","last_change":"2024-04-20T10:27:32.069209+0000","last_active":"2024-04-20T10:27:38.107037+0000","last_peered":"2024-04-20T10:27:38.107037+0000","last_clean":"2024-04-20T10:27:38.107037+0000","last_became_active":"2024-04-20T10:27:32.068711+0000","last_became_peered":"2024-04-20T10:27:32.068711+0000","last_unstale":"2024-04-20T10:27:38.107037+0000","last_undegraded":"2024-04-20T10:27:38.107037+0000","last_fullsized":"2024-04-20T10:27:38.107037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:55.256665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392446+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:27:38.392446+0000","last_peered":"2024-04-20T10:27:38.392446+0000","last_clean":"2024-04-20T10:27:38.392446+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:27:38.392446+0000","last_undegraded":"2024-04-20T10:27:38.392446+0000","last_fullsized":"2024-04-20T10:27:38.392446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108582+0000","last_change":"2024-04-20T10:27:37.103228+0000","last_active":"2024-04-20T10:27:38.108582+0000","last_peered":"2024-04-20T10:27:38.108582+0000","last_clean":"2024-04-20T10:27:38.108582+0000","last_became_active":"2024-04-20T10:27:37.102768+0000","last_became_peered":"2024-04-20T10:27:37.102768+0000","last_unstale":"2024-04-20T10:27:38.108582+0000","last_undegraded":"2024-04-20T10:27:38.108582+0000","last_fullsized":"2024-04-20T10:27:38.108582+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:13.639017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108024+0000","last_change":"2024-04-20T10:27:33.014266+0000","last_active":"2024-04-20T10:27:38.108024+0000","last_peered":"2024-04-20T10:27:38.108024+0000","last_clean":"2024-04-20T10:27:38.108024+0000","last_became_active":"2024-04-20T10:27:33.013935+0000","last_became_peered":"2024-04-20T10:27:33.013935+0000","last_unstale":"2024-04-20T10:27:38.108024+0000","last_undegraded":"2024-04-20T10:27:38.108024+0000","last_fullsized":"2024-04-20T10:27:38.108024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:03.292296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109594+0000","last_change":"2024-04-20T10:27:32.078144+0000","last_active":"2024-04-20T10:27:38.109594+0000","last_peered":"2024-04-20T10:27:38.109594+0000","last_clean":"2024-04-20T10:27:38.109594+0000","last_became_active":"2024-04-20T10:27:32.075588+0000","last_became_peered":"2024-04-20T10:27:32.075588+0000","last_unstale":"2024-04-20T10:27:38.109594+0000","last_undegraded":"2024-04-20T10:27:38.109594+0000","last_fullsized":"2024-04-20T10:27:38.109594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390435+0000","last_change":"2024-04-20T10:27:37.128340+0000","last_active":"2024-04-20T10:27:38.390435+0000","last_peered":"2024-04-20T10:27:38.390435+0000","last_clean":"2024-04-20T10:27:38.390435+0000","last_became_active":"2024-04-20T10:27:37.127609+0000","last_became_peered":"2024-04-20T10:27:37.127609+0000","last_unstale":"2024-04-20T10:27:38.390435+0000","last_undegraded":"2024-04-20T10:27:38.390435+0000","last_fullsized":"2024-04-20T10:27:38.390435+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:41.767940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388918+0000","last_change":"2024-04-20T10:27:33.007624+0000","last_active":"2024-04-20T10:27:38.388918+0000","last_peered":"2024-04-20T10:27:38.388918+0000","last_clean":"2024-04-20T10:27:38.388918+0000","last_became_active":"2024-04-20T10:27:33.007319+0000","last_became_peered":"2024-04-20T10:27:33.007319+0000","last_unstale":"2024-04-20T10:27:38.388918+0000","last_undegraded":"2024-04-20T10:27:38.388918+0000","last_fullsized":"2024-04-20T10:27:38.388918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112959+0000","last_change":"2024-04-20T10:27:32.087470+0000","last_active":"2024-04-20T10:27:38.112959+0000","last_peered":"2024-04-20T10:27:38.112959+0000","last_clean":"2024-04-20T10:27:38.112959+0000","last_became_active":"2024-04-20T10:27:32.086045+0000","last_became_peered":"2024-04-20T10:27:32.086045+0000","last_unstale":"2024-04-20T10:27:38.112959+0000","last_undegraded":"2024-04-20T10:27:38.112959+0000","last_fullsized":"2024-04-20T10:27:38.112959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111234+0000","last_change":"2024-04-20T10:27:37.118303+0000","last_active":"2024-04-20T10:27:38.111234+0000","last_peered":"2024-04-20T10:27:38.111234+0000","last_clean":"2024-04-20T10:27:38.111234+0000","last_became_active":"2024-04-20T10:27:37.117875+0000","last_became_peered":"2024-04-20T10:27:37.117875+0000","last_unstale":"2024-04-20T10:27:38.111234+0000","last_undegraded":"2024-04-20T10:27:38.111234+0000","last_fullsized":"2024-04-20T10:27:38.111234+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389469+0000","last_change":"2024-04-20T10:27:33.021039+0000","last_active":"2024-04-20T10:27:38.389469+0000","last_peered":"2024-04-20T10:27:38.389469+0000","last_clean":"2024-04-20T10:27:38.389469+0000","last_became_active":"2024-04-20T10:27:33.020767+0000","last_became_peered":"2024-04-20T10:27:33.020767+0000","last_unstale":"2024-04-20T10:27:38.389469+0000","last_undegraded":"2024-04-20T10:27:38.389469+0000","last_fullsized":"2024-04-20T10:27:38.389469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392137+0000","last_change":"2024-04-20T10:27:32.091910+0000","last_active":"2024-04-20T10:27:38.392137+0000","last_peered":"2024-04-20T10:27:38.392137+0000","last_clean":"2024-04-20T10:27:38.392137+0000","last_became_active":"2024-04-20T10:27:32.091500+0000","last_became_peered":"2024-04-20T10:27:32.091500+0000","last_unstale":"2024-04-20T10:27:38.392137+0000","last_undegraded":"2024-04-20T10:27:38.392137+0000","last_fullsized":"2024-04-20T10:27:38.392137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:53:02.468594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105575+0000","last_change":"2024-04-20T10:27:37.119655+0000","last_active":"2024-04-20T10:27:38.105575+0000","last_peered":"2024-04-20T10:27:38.105575+0000","last_clean":"2024-04-20T10:27:38.105575+0000","last_became_active":"2024-04-20T10:27:37.119274+0000","last_became_peered":"2024-04-20T10:27:37.119274+0000","last_unstale":"2024-04-20T10:27:38.105575+0000","last_undegraded":"2024-04-20T10:27:38.105575+0000","last_fullsized":"2024-04-20T10:27:38.105575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:56:33.147822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392306+0000","last_change":"2024-04-20T10:27:33.003557+0000","last_active":"2024-04-20T10:27:38.392306+0000","last_peered":"2024-04-20T10:27:38.392306+0000","last_clean":"2024-04-20T10:27:38.392306+0000","last_became_active":"2024-04-20T10:27:33.001763+0000","last_became_peered":"2024-04-20T10:27:33.001763+0000","last_unstale":"2024-04-20T10:27:38.392306+0000","last_undegraded":"2024-04-20T10:27:38.392306+0000","last_fullsized":"2024-04-20T10:27:38.392306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:45.483619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389540+0000","last_change":"2024-04-20T10:27:32.090042+0000","last_active":"2024-04-20T10:27:38.389540+0000","last_peered":"2024-04-20T10:27:38.389540+0000","last_clean":"2024-04-20T10:27:38.389540+0000","last_became_active":"2024-04-20T10:27:32.089694+0000","last_became_peered":"2024-04-20T10:27:32.089694+0000","last_unstale":"2024-04-20T10:27:38.389540+0000","last_undegraded":"2024-04-20T10:27:38.389540+0000","last_fullsized":"2024-04-20T10:27:38.389540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393350+0000","last_change":"2024-04-20T10:27:37.125234+0000","last_active":"2024-04-20T10:27:38.393350+0000","last_peered":"2024-04-20T10:27:38.393350+0000","last_clean":"2024-04-20T10:27:38.393350+0000","last_became_active":"2024-04-20T10:27:37.124838+0000","last_became_peered":"2024-04-20T10:27:37.124838+0000","last_unstale":"2024-04-20T10:27:38.393350+0000","last_undegraded":"2024-04-20T10:27:38.393350+0000","last_fullsized":"2024-04-20T10:27:38.393350+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:47.071764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105563+0000","last_change":"2024-04-20T10:27:33.013769+0000","last_active":"2024-04-20T10:27:38.105563+0000","last_peered":"2024-04-20T10:27:38.105563+0000","last_clean":"2024-04-20T10:27:38.105563+0000","last_became_active":"2024-04-20T10:27:33.005662+0000","last_became_peered":"2024-04-20T10:27:33.005662+0000","last_unstale":"2024-04-20T10:27:38.105563+0000","last_undegraded":"2024-04-20T10:27:38.105563+0000","last_fullsized":"2024-04-20T10:27:38.105563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389725+0000","last_change":"2024-04-20T10:27:32.080478+0000","last_active":"2024-04-20T10:27:38.389725+0000","last_peered":"2024-04-20T10:27:38.389725+0000","last_clean":"2024-04-20T10:27:38.389725+0000","last_became_active":"2024-04-20T10:27:32.080148+0000","last_became_peered":"2024-04-20T10:27:32.080148+0000","last_unstale":"2024-04-20T10:27:38.389725+0000","last_undegraded":"2024-04-20T10:27:38.389725+0000","last_fullsized":"2024-04-20T10:27:38.389725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:17.004529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112793+0000","last_change":"2024-04-20T10:27:37.126177+0000","last_active":"2024-04-20T10:27:38.112793+0000","last_peered":"2024-04-20T10:27:38.112793+0000","last_clean":"2024-04-20T10:27:38.112793+0000","last_became_active":"2024-04-20T10:27:37.125862+0000","last_became_peered":"2024-04-20T10:27:37.125862+0000","last_unstale":"2024-04-20T10:27:38.112793+0000","last_undegraded":"2024-04-20T10:27:38.112793+0000","last_fullsized":"2024-04-20T10:27:38.112793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:07.395930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391792+0000","last_change":"2024-04-20T10:27:33.016106+0000","last_active":"2024-04-20T10:27:38.391792+0000","last_peered":"2024-04-20T10:27:38.391792+0000","last_clean":"2024-04-20T10:27:38.391792+0000","last_became_active":"2024-04-20T10:27:33.015787+0000","last_became_peered":"2024-04-20T10:27:33.015787+0000","last_unstale":"2024-04-20T10:27:38.391792+0000","last_undegraded":"2024-04-20T10:27:38.391792+0000","last_fullsized":"2024-04-20T10:27:38.391792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388035+0000","last_change":"2024-04-20T10:27:32.075511+0000","last_active":"2024-04-20T10:27:38.388035+0000","last_peered":"2024-04-20T10:27:38.388035+0000","last_clean":"2024-04-20T10:27:38.388035+0000","last_became_active":"2024-04-20T10:27:32.072523+0000","last_became_peered":"2024-04-20T10:27:32.072523+0000","last_unstale":"2024-04-20T10:27:38.388035+0000","last_undegraded":"2024-04-20T10:27:38.388035+0000","last_fullsized":"2024-04-20T10:27:38.388035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113117+0000","last_change":"2024-04-20T10:27:37.121568+0000","last_active":"2024-04-20T10:27:38.113117+0000","last_peered":"2024-04-20T10:27:38.113117+0000","last_clean":"2024-04-20T10:27:38.113117+0000","last_became_active":"2024-04-20T10:27:37.121105+0000","last_became_peered":"2024-04-20T10:27:37.121105+0000","last_unstale":"2024-04-20T10:27:38.113117+0000","last_undegraded":"2024-04-20T10:27:38.113117+0000","last_fullsized":"2024-04-20T10:27:38.113117+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:20.615032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107307+0000","last_change":"2024-04-20T10:27:33.008601+0000","last_active":"2024-04-20T10:27:38.107307+0000","last_peered":"2024-04-20T10:27:38.107307+0000","last_clean":"2024-04-20T10:27:38.107307+0000","last_became_active":"2024-04-20T10:27:33.008317+0000","last_became_peered":"2024-04-20T10:27:33.008317+0000","last_unstale":"2024-04-20T10:27:38.107307+0000","last_undegraded":"2024-04-20T10:27:38.107307+0000","last_fullsized":"2024-04-20T10:27:38.107307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:55.576815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106751+0000","last_change":"2024-04-20T10:27:32.081278+0000","last_active":"2024-04-20T10:27:38.106751+0000","last_peered":"2024-04-20T10:27:38.106751+0000","last_clean":"2024-04-20T10:27:38.106751+0000","last_became_active":"2024-04-20T10:27:32.080466+0000","last_became_peered":"2024-04-20T10:27:32.080466+0000","last_unstale":"2024-04-20T10:27:38.106751+0000","last_undegraded":"2024-04-20T10:27:38.106751+0000","last_fullsized":"2024-04-20T10:27:38.106751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:51.199449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109133+0000","last_change":"2024-04-20T10:27:37.132802+0000","last_active":"2024-04-20T10:27:38.109133+0000","last_peered":"2024-04-20T10:27:38.109133+0000","last_clean":"2024-04-20T10:27:38.109133+0000","last_became_active":"2024-04-20T10:27:37.129193+0000","last_became_peered":"2024-04-20T10:27:37.129193+0000","last_unstale":"2024-04-20T10:27:38.109133+0000","last_undegraded":"2024-04-20T10:27:38.109133+0000","last_fullsized":"2024-04-20T10:27:38.109133+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110211+0000","last_change":"2024-04-20T10:27:33.010100+0000","last_active":"2024-04-20T10:27:38.110211+0000","last_peered":"2024-04-20T10:27:38.110211+0000","last_clean":"2024-04-20T10:27:38.110211+0000","last_became_active":"2024-04-20T10:27:33.009703+0000","last_became_peered":"2024-04-20T10:27:33.009703+0000","last_unstale":"2024-04-20T10:27:38.110211+0000","last_undegraded":"2024-04-20T10:27:38.110211+0000","last_fullsized":"2024-04-20T10:27:38.110211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391899+0000","last_change":"2024-04-20T10:27:32.078724+0000","last_active":"2024-04-20T10:27:38.391899+0000","last_peered":"2024-04-20T10:27:38.391899+0000","last_clean":"2024-04-20T10:27:38.391899+0000","last_became_active":"2024-04-20T10:27:32.078093+0000","last_became_peered":"2024-04-20T10:27:32.078093+0000","last_unstale":"2024-04-20T10:27:38.391899+0000","last_undegraded":"2024-04-20T10:27:38.391899+0000","last_fullsized":"2024-04-20T10:27:38.391899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107857+0000","last_change":"2024-04-20T10:27:37.121822+0000","last_active":"2024-04-20T10:27:38.107857+0000","last_peered":"2024-04-20T10:27:38.107857+0000","last_clean":"2024-04-20T10:27:38.107857+0000","last_became_active":"2024-04-20T10:27:37.121423+0000","last_became_peered":"2024-04-20T10:27:37.121423+0000","last_unstale":"2024-04-20T10:27:38.107857+0000","last_undegraded":"2024-04-20T10:27:38.107857+0000","last_fullsized":"2024-04-20T10:27:38.107857+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:21:35.974697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388424+0000","last_change":"2024-04-20T10:27:33.001752+0000","last_active":"2024-04-20T10:27:38.388424+0000","last_peered":"2024-04-20T10:27:38.388424+0000","last_clean":"2024-04-20T10:27:38.388424+0000","last_became_active":"2024-04-20T10:27:33.001201+0000","last_became_peered":"2024-04-20T10:27:33.001201+0000","last_unstale":"2024-04-20T10:27:38.388424+0000","last_undegraded":"2024-04-20T10:27:38.388424+0000","last_fullsized":"2024-04-20T10:27:38.388424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:44.536794+0000","last_change":"2024-04-20T10:27:32.080328+0000","last_active":"2024-04-20T10:27:44.536794+0000","last_peered":"2024-04-20T10:27:44.536794+0000","last_clean":"2024-04-20T10:27:44.536794+0000","last_became_active":"2024-04-20T10:27:32.080011+0000","last_became_peered":"2024-04-20T10:27:32.080011+0000","last_unstale":"2024-04-20T10:27:44.536794+0000","last_undegraded":"2024-04-20T10:27:44.536794+0000","last_fullsized":"2024-04-20T10:27:44.536794+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:22:14.097676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3701,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389991+0000","last_change":"2024-04-20T10:27:37.126061+0000","last_active":"2024-04-20T10:27:38.389991+0000","last_peered":"2024-04-20T10:27:38.389991+0000","last_clean":"2024-04-20T10:27:38.389991+0000","last_became_active":"2024-04-20T10:27:37.121461+0000","last_became_peered":"2024-04-20T10:27:37.121461+0000","last_unstale":"2024-04-20T10:27:38.389991+0000","last_undegraded":"2024-04-20T10:27:38.389991+0000","last_fullsized":"2024-04-20T10:27:38.389991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:13.289328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107516+0000","last_change":"2024-04-20T10:27:33.007572+0000","last_active":"2024-04-20T10:27:38.107516+0000","last_peered":"2024-04-20T10:27:38.107516+0000","last_clean":"2024-04-20T10:27:38.107516+0000","last_became_active":"2024-04-20T10:27:33.007206+0000","last_became_peered":"2024-04-20T10:27:33.007206+0000","last_unstale":"2024-04-20T10:27:38.107516+0000","last_undegraded":"2024-04-20T10:27:38.107516+0000","last_fullsized":"2024-04-20T10:27:38.107516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:21.846828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393065+0000","last_change":"2024-04-20T10:27:32.080090+0000","last_active":"2024-04-20T10:27:38.393065+0000","last_peered":"2024-04-20T10:27:38.393065+0000","last_clean":"2024-04-20T10:27:38.393065+0000","last_became_active":"2024-04-20T10:27:32.079763+0000","last_became_peered":"2024-04-20T10:27:32.079763+0000","last_unstale":"2024-04-20T10:27:38.393065+0000","last_undegraded":"2024-04-20T10:27:38.393065+0000","last_fullsized":"2024-04-20T10:27:38.393065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390646+0000","last_change":"2024-04-20T10:27:37.113593+0000","last_active":"2024-04-20T10:27:38.390646+0000","last_peered":"2024-04-20T10:27:38.390646+0000","last_clean":"2024-04-20T10:27:38.390646+0000","last_became_active":"2024-04-20T10:27:37.113016+0000","last_became_peered":"2024-04-20T10:27:37.113016+0000","last_unstale":"2024-04-20T10:27:38.390646+0000","last_undegraded":"2024-04-20T10:27:38.390646+0000","last_fullsized":"2024-04-20T10:27:38.390646+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:27.394166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113229+0000","last_change":"2024-04-20T10:27:33.020099+0000","last_active":"2024-04-20T10:27:38.113229+0000","last_peered":"2024-04-20T10:27:38.113229+0000","last_clean":"2024-04-20T10:27:38.113229+0000","last_became_active":"2024-04-20T10:27:33.018743+0000","last_became_peered":"2024-04-20T10:27:33.018743+0000","last_unstale":"2024-04-20T10:27:38.113229+0000","last_undegraded":"2024-04-20T10:27:38.113229+0000","last_fullsized":"2024-04-20T10:27:38.113229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389020+0000","last_change":"2024-04-20T10:27:32.081599+0000","last_active":"2024-04-20T10:27:38.389020+0000","last_peered":"2024-04-20T10:27:38.389020+0000","last_clean":"2024-04-20T10:27:38.389020+0000","last_became_active":"2024-04-20T10:27:32.081288+0000","last_became_peered":"2024-04-20T10:27:32.081288+0000","last_unstale":"2024-04-20T10:27:38.389020+0000","last_undegraded":"2024-04-20T10:27:38.389020+0000","last_fullsized":"2024-04-20T10:27:38.389020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391304+0000","last_change":"2024-04-20T10:27:37.134883+0000","last_active":"2024-04-20T10:27:38.391304+0000","last_peered":"2024-04-20T10:27:38.391304+0000","last_clean":"2024-04-20T10:27:38.391304+0000","last_became_active":"2024-04-20T10:27:37.134221+0000","last_became_peered":"2024-04-20T10:27:37.134221+0000","last_unstale":"2024-04-20T10:27:38.391304+0000","last_undegraded":"2024-04-20T10:27:38.391304+0000","last_fullsized":"2024-04-20T10:27:38.391304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:10.327991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392241+0000","last_change":"2024-04-20T10:27:33.006819+0000","last_active":"2024-04-20T10:27:38.392241+0000","last_peered":"2024-04-20T10:27:38.392241+0000","last_clean":"2024-04-20T10:27:38.392241+0000","last_became_active":"2024-04-20T10:27:33.005657+0000","last_became_peered":"2024-04-20T10:27:33.005657+0000","last_unstale":"2024-04-20T10:27:38.392241+0000","last_undegraded":"2024-04-20T10:27:38.392241+0000","last_fullsized":"2024-04-20T10:27:38.392241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391630+0000","last_change":"2024-04-20T10:27:32.083786+0000","last_active":"2024-04-20T10:27:38.391630+0000","last_peered":"2024-04-20T10:27:38.391630+0000","last_clean":"2024-04-20T10:27:38.391630+0000","last_became_active":"2024-04-20T10:27:32.083256+0000","last_became_peered":"2024-04-20T10:27:32.083256+0000","last_unstale":"2024-04-20T10:27:38.391630+0000","last_undegraded":"2024-04-20T10:27:38.391630+0000","last_fullsized":"2024-04-20T10:27:38.391630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:11.023664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390075+0000","last_change":"2024-04-20T10:27:37.131051+0000","last_active":"2024-04-20T10:27:38.390075+0000","last_peered":"2024-04-20T10:27:38.390075+0000","last_clean":"2024-04-20T10:27:38.390075+0000","last_became_active":"2024-04-20T10:27:37.130723+0000","last_became_peered":"2024-04-20T10:27:37.130723+0000","last_unstale":"2024-04-20T10:27:38.390075+0000","last_undegraded":"2024-04-20T10:27:38.390075+0000","last_fullsized":"2024-04-20T10:27:38.390075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:28.980316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391821+0000","last_change":"2024-04-20T10:27:37.124930+0000","last_active":"2024-04-20T10:27:38.391821+0000","last_peered":"2024-04-20T10:27:38.391821+0000","last_clean":"2024-04-20T10:27:38.391821+0000","last_became_active":"2024-04-20T10:27:37.123089+0000","last_became_peered":"2024-04-20T10:27:37.123089+0000","last_unstale":"2024-04-20T10:27:38.391821+0000","last_undegraded":"2024-04-20T10:27:38.391821+0000","last_fullsized":"2024-04-20T10:27:38.391821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:05.113291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390231+0000","last_change":"2024-04-20T10:27:33.020003+0000","last_active":"2024-04-20T10:27:38.390231+0000","last_peered":"2024-04-20T10:27:38.390231+0000","last_clean":"2024-04-20T10:27:38.390231+0000","last_became_active":"2024-04-20T10:27:33.019709+0000","last_became_peered":"2024-04-20T10:27:33.019709+0000","last_unstale":"2024-04-20T10:27:38.390231+0000","last_undegraded":"2024-04-20T10:27:38.390231+0000","last_fullsized":"2024-04-20T10:27:38.390231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:06.663893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106161+0000","last_change":"2024-04-20T10:27:32.070067+0000","last_active":"2024-04-20T10:27:38.106161+0000","last_peered":"2024-04-20T10:27:38.106161+0000","last_clean":"2024-04-20T10:27:38.106161+0000","last_became_active":"2024-04-20T10:27:32.069761+0000","last_became_peered":"2024-04-20T10:27:32.069761+0000","last_unstale":"2024-04-20T10:27:38.106161+0000","last_undegraded":"2024-04-20T10:27:38.106161+0000","last_fullsized":"2024-04-20T10:27:38.106161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108148+0000","last_change":"2024-04-20T10:27:37.125295+0000","last_active":"2024-04-20T10:27:38.108148+0000","last_peered":"2024-04-20T10:27:38.108148+0000","last_clean":"2024-04-20T10:27:38.108148+0000","last_became_active":"2024-04-20T10:27:37.124922+0000","last_became_peered":"2024-04-20T10:27:37.124922+0000","last_unstale":"2024-04-20T10:27:38.108148+0000","last_undegraded":"2024-04-20T10:27:38.108148+0000","last_fullsized":"2024-04-20T10:27:38.108148+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:45.803688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108085+0000","last_change":"2024-04-20T10:27:33.009010+0000","last_active":"2024-04-20T10:27:38.108085+0000","last_peered":"2024-04-20T10:27:38.108085+0000","last_clean":"2024-04-20T10:27:38.108085+0000","last_became_active":"2024-04-20T10:27:33.008701+0000","last_became_peered":"2024-04-20T10:27:33.008701+0000","last_unstale":"2024-04-20T10:27:38.108085+0000","last_undegraded":"2024-04-20T10:27:38.108085+0000","last_fullsized":"2024-04-20T10:27:38.108085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109186+0000","last_change":"2024-04-20T10:27:32.079095+0000","last_active":"2024-04-20T10:27:38.109186+0000","last_peered":"2024-04-20T10:27:38.109186+0000","last_clean":"2024-04-20T10:27:38.109186+0000","last_became_active":"2024-04-20T10:27:32.074687+0000","last_became_peered":"2024-04-20T10:27:32.074687+0000","last_unstale":"2024-04-20T10:27:38.109186+0000","last_undegraded":"2024-04-20T10:27:38.109186+0000","last_fullsized":"2024-04-20T10:27:38.109186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106749+0000","last_change":"2024-04-20T10:27:32.080751+0000","last_active":"2024-04-20T10:27:38.106749+0000","last_peered":"2024-04-20T10:27:38.106749+0000","last_clean":"2024-04-20T10:27:38.106749+0000","last_became_active":"2024-04-20T10:27:32.078604+0000","last_became_peered":"2024-04-20T10:27:32.078604+0000","last_unstale":"2024-04-20T10:27:38.106749+0000","last_undegraded":"2024-04-20T10:27:38.106749+0000","last_fullsized":"2024-04-20T10:27:38.106749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:10.874813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390089+0000","last_change":"2024-04-20T10:27:33.024156+0000","last_active":"2024-04-20T10:27:38.390089+0000","last_peered":"2024-04-20T10:27:38.390089+0000","last_clean":"2024-04-20T10:27:38.390089+0000","last_became_active":"2024-04-20T10:27:33.023639+0000","last_became_peered":"2024-04-20T10:27:33.023639+0000","last_unstale":"2024-04-20T10:27:38.390089+0000","last_undegraded":"2024-04-20T10:27:38.390089+0000","last_fullsized":"2024-04-20T10:27:38.390089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:45.132088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391353+0000","last_change":"2024-04-20T10:27:37.136584+0000","last_active":"2024-04-20T10:27:38.391353+0000","last_peered":"2024-04-20T10:27:38.391353+0000","last_clean":"2024-04-20T10:27:38.391353+0000","last_became_active":"2024-04-20T10:27:37.135997+0000","last_became_peered":"2024-04-20T10:27:37.135997+0000","last_unstale":"2024-04-20T10:27:38.391353+0000","last_undegraded":"2024-04-20T10:27:38.391353+0000","last_fullsized":"2024-04-20T10:27:38.391353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:27.651374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111655+0000","last_change":"2024-04-20T10:27:32.090018+0000","last_active":"2024-04-20T10:27:38.111655+0000","last_peered":"2024-04-20T10:27:38.111655+0000","last_clean":"2024-04-20T10:27:38.111655+0000","last_became_active":"2024-04-20T10:27:32.089144+0000","last_became_peered":"2024-04-20T10:27:32.089144+0000","last_unstale":"2024-04-20T10:27:38.111655+0000","last_undegraded":"2024-04-20T10:27:38.111655+0000","last_fullsized":"2024-04-20T10:27:38.111655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390744+0000","last_change":"2024-04-20T10:27:33.010778+0000","last_active":"2024-04-20T10:27:38.390744+0000","last_peered":"2024-04-20T10:27:38.390744+0000","last_clean":"2024-04-20T10:27:38.390744+0000","last_became_active":"2024-04-20T10:27:33.010403+0000","last_became_peered":"2024-04-20T10:27:33.010403+0000","last_unstale":"2024-04-20T10:27:38.390744+0000","last_undegraded":"2024-04-20T10:27:38.390744+0000","last_fullsized":"2024-04-20T10:27:38.390744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393172+0000","last_change":"2024-04-20T10:27:37.134702+0000","last_active":"2024-04-20T10:27:38.393172+0000","last_peered":"2024-04-20T10:27:38.393172+0000","last_clean":"2024-04-20T10:27:38.393172+0000","last_became_active":"2024-04-20T10:27:37.134178+0000","last_became_peered":"2024-04-20T10:27:37.134178+0000","last_unstale":"2024-04-20T10:27:38.393172+0000","last_undegraded":"2024-04-20T10:27:38.393172+0000","last_fullsized":"2024-04-20T10:27:38.393172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:48.722800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390536+0000","last_change":"2024-04-20T10:27:32.998168+0000","last_active":"2024-04-20T10:27:38.390536+0000","last_peered":"2024-04-20T10:27:38.390536+0000","last_clean":"2024-04-20T10:27:38.390536+0000","last_became_active":"2024-04-20T10:27:32.996843+0000","last_became_peered":"2024-04-20T10:27:32.996843+0000","last_unstale":"2024-04-20T10:27:38.390536+0000","last_undegraded":"2024-04-20T10:27:38.390536+0000","last_fullsized":"2024-04-20T10:27:38.390536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:13.408788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391052+0000","last_change":"2024-04-20T10:27:33.020874+0000","last_active":"2024-04-20T10:27:38.391052+0000","last_peered":"2024-04-20T10:27:38.391052+0000","last_clean":"2024-04-20T10:27:38.391052+0000","last_became_active":"2024-04-20T10:27:33.016375+0000","last_became_peered":"2024-04-20T10:27:33.016375+0000","last_unstale":"2024-04-20T10:27:38.391052+0000","last_undegraded":"2024-04-20T10:27:38.391052+0000","last_fullsized":"2024-04-20T10:27:38.391052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105932+0000","last_change":"2024-04-20T10:27:33.011331+0000","last_active":"2024-04-20T10:27:38.105932+0000","last_peered":"2024-04-20T10:27:38.105932+0000","last_clean":"2024-04-20T10:27:38.105932+0000","last_became_active":"2024-04-20T10:27:33.010956+0000","last_became_peered":"2024-04-20T10:27:33.010956+0000","last_unstale":"2024-04-20T10:27:38.105932+0000","last_undegraded":"2024-04-20T10:27:38.105932+0000","last_fullsized":"2024-04-20T10:27:38.105932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:16.511398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111379+0000","last_change":"2024-04-20T10:27:33.009386+0000","last_active":"2024-04-20T10:27:38.111379+0000","last_peered":"2024-04-20T10:27:38.111379+0000","last_clean":"2024-04-20T10:27:38.111379+0000","last_became_active":"2024-04-20T10:27:33.008596+0000","last_became_peered":"2024-04-20T10:27:33.008596+0000","last_unstale":"2024-04-20T10:27:38.111379+0000","last_undegraded":"2024-04-20T10:27:38.111379+0000","last_fullsized":"2024-04-20T10:27:38.111379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109220+0000","last_change":"2024-04-20T10:27:33.001780+0000","last_active":"2024-04-20T10:27:38.109220+0000","last_peered":"2024-04-20T10:27:38.109220+0000","last_clean":"2024-04-20T10:27:38.109220+0000","last_became_active":"2024-04-20T10:27:33.000852+0000","last_became_peered":"2024-04-20T10:27:33.000852+0000","last_unstale":"2024-04-20T10:27:38.109220+0000","last_undegraded":"2024-04-20T10:27:38.109220+0000","last_fullsized":"2024-04-20T10:27:38.109220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:52.213855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111488+0000","last_change":"2024-04-20T10:27:33.010234+0000","last_active":"2024-04-20T10:27:38.111488+0000","last_peered":"2024-04-20T10:27:38.111488+0000","last_clean":"2024-04-20T10:27:38.111488+0000","last_became_active":"2024-04-20T10:27:33.009934+0000","last_became_peered":"2024-04-20T10:27:33.009934+0000","last_unstale":"2024-04-20T10:27:38.111488+0000","last_undegraded":"2024-04-20T10:27:38.111488+0000","last_fullsized":"2024-04-20T10:27:38.111488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392171+0000","last_change":"2024-04-20T10:27:33.006041+0000","last_active":"2024-04-20T10:27:38.392171+0000","last_peered":"2024-04-20T10:27:38.392171+0000","last_clean":"2024-04-20T10:27:38.392171+0000","last_became_active":"2024-04-20T10:27:33.005259+0000","last_became_peered":"2024-04-20T10:27:33.005259+0000","last_unstale":"2024-04-20T10:27:38.392171+0000","last_undegraded":"2024-04-20T10:27:38.392171+0000","last_fullsized":"2024-04-20T10:27:38.392171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112860+0000","last_change":"2024-04-20T10:27:33.015530+0000","last_active":"2024-04-20T10:27:38.112860+0000","last_peered":"2024-04-20T10:27:38.112860+0000","last_clean":"2024-04-20T10:27:38.112860+0000","last_became_active":"2024-04-20T10:27:33.015240+0000","last_became_peered":"2024-04-20T10:27:33.015240+0000","last_unstale":"2024-04-20T10:27:38.112860+0000","last_undegraded":"2024-04-20T10:27:38.112860+0000","last_fullsized":"2024-04-20T10:27:38.112860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105792+0000","last_change":"2024-04-20T10:27:33.015916+0000","last_active":"2024-04-20T10:27:38.105792+0000","last_peered":"2024-04-20T10:27:38.105792+0000","last_clean":"2024-04-20T10:27:38.105792+0000","last_became_active":"2024-04-20T10:27:33.015585+0000","last_became_peered":"2024-04-20T10:27:33.015585+0000","last_unstale":"2024-04-20T10:27:38.105792+0000","last_undegraded":"2024-04-20T10:27:38.105792+0000","last_fullsized":"2024-04-20T10:27:38.105792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112167+0000","last_change":"2024-04-20T10:27:33.014307+0000","last_active":"2024-04-20T10:27:38.112167+0000","last_peered":"2024-04-20T10:27:38.112167+0000","last_clean":"2024-04-20T10:27:38.112167+0000","last_became_active":"2024-04-20T10:27:33.013891+0000","last_became_peered":"2024-04-20T10:27:33.013891+0000","last_unstale":"2024-04-20T10:27:38.112167+0000","last_undegraded":"2024-04-20T10:27:38.112167+0000","last_fullsized":"2024-04-20T10:27:38.112167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:32.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390622+0000","last_change":"2024-04-20T10:27:33.010713+0000","last_active":"2024-04-20T10:27:38.390622+0000","last_peered":"2024-04-20T10:27:38.390622+0000","last_clean":"2024-04-20T10:27:38.390622+0000","last_became_active":"2024-04-20T10:27:33.009349+0000","last_became_peered":"2024-04-20T10:27:33.009349+0000","last_unstale":"2024-04-20T10:27:38.390622+0000","last_undegraded":"2024-04-20T10:27:38.390622+0000","last_fullsized":"2024-04-20T10:27:38.390622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389798+0000","last_change":"2024-04-20T10:27:33.018481+0000","last_active":"2024-04-20T10:27:38.389798+0000","last_peered":"2024-04-20T10:27:38.389798+0000","last_clean":"2024-04-20T10:27:38.389798+0000","last_became_active":"2024-04-20T10:27:33.015328+0000","last_became_peered":"2024-04-20T10:27:33.015328+0000","last_unstale":"2024-04-20T10:27:38.389798+0000","last_undegraded":"2024-04-20T10:27:38.389798+0000","last_fullsized":"2024-04-20T10:27:38.389798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:43:48.722150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388831+0000","last_change":"2024-04-20T10:27:33.006877+0000","last_active":"2024-04-20T10:27:38.388831+0000","last_peered":"2024-04-20T10:27:38.388831+0000","last_clean":"2024-04-20T10:27:38.388831+0000","last_became_active":"2024-04-20T10:27:33.006143+0000","last_became_peered":"2024-04-20T10:27:33.006143+0000","last_unstale":"2024-04-20T10:27:38.388831+0000","last_undegraded":"2024-04-20T10:27:38.388831+0000","last_fullsized":"2024-04-20T10:27:38.388831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108600+0000","last_change":"2024-04-20T10:27:33.016666+0000","last_active":"2024-04-20T10:27:38.108600+0000","last_peered":"2024-04-20T10:27:38.108600+0000","last_clean":"2024-04-20T10:27:38.108600+0000","last_became_active":"2024-04-20T10:27:33.016268+0000","last_became_peered":"2024-04-20T10:27:33.016268+0000","last_unstale":"2024-04-20T10:27:38.108600+0000","last_undegraded":"2024-04-20T10:27:38.108600+0000","last_fullsized":"2024-04-20T10:27:38.108600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:07.386901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392668+0000","last_change":"2024-04-20T10:27:33.014742+0000","last_active":"2024-04-20T10:27:38.392668+0000","last_peered":"2024-04-20T10:27:38.392668+0000","last_clean":"2024-04-20T10:27:38.392668+0000","last_became_active":"2024-04-20T10:27:33.014443+0000","last_became_peered":"2024-04-20T10:27:33.014443+0000","last_unstale":"2024-04-20T10:27:38.392668+0000","last_undegraded":"2024-04-20T10:27:38.392668+0000","last_fullsized":"2024-04-20T10:27:38.392668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108659+0000","last_change":"2024-04-20T10:27:33.002993+0000","last_active":"2024-04-20T10:27:38.108659+0000","last_peered":"2024-04-20T10:27:38.108659+0000","last_clean":"2024-04-20T10:27:38.108659+0000","last_became_active":"2024-04-20T10:27:33.002685+0000","last_became_peered":"2024-04-20T10:27:33.002685+0000","last_unstale":"2024-04-20T10:27:38.108659+0000","last_undegraded":"2024-04-20T10:27:38.108659+0000","last_fullsized":"2024-04-20T10:27:38.108659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:38.509857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105782+0000","last_change":"2024-04-20T10:27:33.014659+0000","last_active":"2024-04-20T10:27:38.105782+0000","last_peered":"2024-04-20T10:27:38.105782+0000","last_clean":"2024-04-20T10:27:38.105782+0000","last_became_active":"2024-04-20T10:27:33.014188+0000","last_became_peered":"2024-04-20T10:27:33.014188+0000","last_unstale":"2024-04-20T10:27:38.105782+0000","last_undegraded":"2024-04-20T10:27:38.105782+0000","last_fullsized":"2024-04-20T10:27:38.105782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107794+0000","last_change":"2024-04-20T10:27:33.011041+0000","last_active":"2024-04-20T10:27:38.107794+0000","last_peered":"2024-04-20T10:27:38.107794+0000","last_clean":"2024-04-20T10:27:38.107794+0000","last_became_active":"2024-04-20T10:27:33.010751+0000","last_became_peered":"2024-04-20T10:27:33.010751+0000","last_unstale":"2024-04-20T10:27:38.107794+0000","last_undegraded":"2024-04-20T10:27:38.107794+0000","last_fullsized":"2024-04-20T10:27:38.107794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110132+0000","last_change":"2024-04-20T10:27:33.005787+0000","last_active":"2024-04-20T10:27:38.110132+0000","last_peered":"2024-04-20T10:27:38.110132+0000","last_clean":"2024-04-20T10:27:38.110132+0000","last_became_active":"2024-04-20T10:27:33.004692+0000","last_became_peered":"2024-04-20T10:27:33.004692+0000","last_unstale":"2024-04-20T10:27:38.110132+0000","last_undegraded":"2024-04-20T10:27:38.110132+0000","last_fullsized":"2024-04-20T10:27:38.110132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:08.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112010+0000","last_change":"2024-04-20T10:27:33.010711+0000","last_active":"2024-04-20T10:27:38.112010+0000","last_peered":"2024-04-20T10:27:38.112010+0000","last_clean":"2024-04-20T10:27:38.112010+0000","last_became_active":"2024-04-20T10:27:33.010282+0000","last_became_peered":"2024-04-20T10:27:33.010282+0000","last_unstale":"2024-04-20T10:27:38.112010+0000","last_undegraded":"2024-04-20T10:27:38.112010+0000","last_fullsized":"2024-04-20T10:27:38.112010+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:06.224357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388721+0000","last_change":"2024-04-20T10:27:33.008425+0000","last_active":"2024-04-20T10:27:38.388721+0000","last_peered":"2024-04-20T10:27:38.388721+0000","last_clean":"2024-04-20T10:27:38.388721+0000","last_became_active":"2024-04-20T10:27:33.006532+0000","last_became_peered":"2024-04-20T10:27:33.006532+0000","last_unstale":"2024-04-20T10:27:38.388721+0000","last_undegraded":"2024-04-20T10:27:38.388721+0000","last_fullsized":"2024-04-20T10:27:38.388721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388300+0000","last_change":"2024-04-20T10:27:33.018484+0000","last_active":"2024-04-20T10:27:38.388300+0000","last_peered":"2024-04-20T10:27:38.388300+0000","last_clean":"2024-04-20T10:27:38.388300+0000","last_became_active":"2024-04-20T10:27:33.018257+0000","last_became_peered":"2024-04-20T10:27:33.018257+0000","last_unstale":"2024-04-20T10:27:38.388300+0000","last_undegraded":"2024-04-20T10:27:38.388300+0000","last_fullsized":"2024-04-20T10:27:38.388300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388715+0000","last_change":"2024-04-20T10:27:33.022995+0000","last_active":"2024-04-20T10:27:38.388715+0000","last_peered":"2024-04-20T10:27:38.388715+0000","last_clean":"2024-04-20T10:27:38.388715+0000","last_became_active":"2024-04-20T10:27:33.017740+0000","last_became_peered":"2024-04-20T10:27:33.017740+0000","last_unstale":"2024-04-20T10:27:38.388715+0000","last_undegraded":"2024-04-20T10:27:38.388715+0000","last_fullsized":"2024-04-20T10:27:38.388715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391590+0000","last_change":"2024-04-20T10:27:33.008312+0000","last_active":"2024-04-20T10:27:38.391590+0000","last_peered":"2024-04-20T10:27:38.391590+0000","last_clean":"2024-04-20T10:27:38.391590+0000","last_became_active":"2024-04-20T10:27:33.002597+0000","last_became_peered":"2024-04-20T10:27:33.002597+0000","last_unstale":"2024-04-20T10:27:38.391590+0000","last_undegraded":"2024-04-20T10:27:38.391590+0000","last_fullsized":"2024-04-20T10:27:38.391590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107489+0000","last_change":"2024-04-20T10:27:33.010109+0000","last_active":"2024-04-20T10:27:38.107489+0000","last_peered":"2024-04-20T10:27:38.107489+0000","last_clean":"2024-04-20T10:27:38.107489+0000","last_became_active":"2024-04-20T10:27:33.009838+0000","last_became_peered":"2024-04-20T10:27:33.009838+0000","last_unstale":"2024-04-20T10:27:38.107489+0000","last_undegraded":"2024-04-20T10:27:38.107489+0000","last_fullsized":"2024-04-20T10:27:38.107489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387903+0000","last_change":"2024-04-20T10:27:33.024641+0000","last_active":"2024-04-20T10:27:38.387903+0000","last_peered":"2024-04-20T10:27:38.387903+0000","last_clean":"2024-04-20T10:27:38.387903+0000","last_became_active":"2024-04-20T10:27:33.024410+0000","last_became_peered":"2024-04-20T10:27:33.024410+0000","last_unstale":"2024-04-20T10:27:38.387903+0000","last_undegraded":"2024-04-20T10:27:38.387903+0000","last_fullsized":"2024-04-20T10:27:38.387903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391138+0000","last_change":"2024-04-20T10:27:33.025226+0000","last_active":"2024-04-20T10:27:38.391138+0000","last_peered":"2024-04-20T10:27:38.391138+0000","last_clean":"2024-04-20T10:27:38.391138+0000","last_became_active":"2024-04-20T10:27:33.024311+0000","last_became_peered":"2024-04-20T10:27:33.024311+0000","last_unstale":"2024-04-20T10:27:38.391138+0000","last_undegraded":"2024-04-20T10:27:38.391138+0000","last_fullsized":"2024-04-20T10:27:38.391138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389207+0000","last_change":"2024-04-20T10:27:33.022586+0000","last_active":"2024-04-20T10:27:38.389207+0000","last_peered":"2024-04-20T10:27:38.389207+0000","last_clean":"2024-04-20T10:27:38.389207+0000","last_became_active":"2024-04-20T10:27:33.022293+0000","last_became_peered":"2024-04-20T10:27:33.022293+0000","last_unstale":"2024-04-20T10:27:38.389207+0000","last_undegraded":"2024-04-20T10:27:38.389207+0000","last_fullsized":"2024-04-20T10:27:38.389207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390946+0000","last_change":"2024-04-20T10:27:33.019615+0000","last_active":"2024-04-20T10:27:38.390946+0000","last_peered":"2024-04-20T10:27:38.390946+0000","last_clean":"2024-04-20T10:27:38.390946+0000","last_became_active":"2024-04-20T10:27:33.019288+0000","last_became_peered":"2024-04-20T10:27:33.019288+0000","last_unstale":"2024-04-20T10:27:38.390946+0000","last_undegraded":"2024-04-20T10:27:38.390946+0000","last_fullsized":"2024-04-20T10:27:38.390946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111457+0000","last_change":"2024-04-20T10:27:33.019815+0000","last_active":"2024-04-20T10:27:38.111457+0000","last_peered":"2024-04-20T10:27:38.111457+0000","last_clean":"2024-04-20T10:27:38.111457+0000","last_became_active":"2024-04-20T10:27:33.019369+0000","last_became_peered":"2024-04-20T10:27:33.019369+0000","last_unstale":"2024-04-20T10:27:38.111457+0000","last_undegraded":"2024-04-20T10:27:38.111457+0000","last_fullsized":"2024-04-20T10:27:38.111457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112358+0000","last_change":"2024-04-20T10:27:33.015773+0000","last_active":"2024-04-20T10:27:38.112358+0000","last_peered":"2024-04-20T10:27:38.112358+0000","last_clean":"2024-04-20T10:27:38.112358+0000","last_became_active":"2024-04-20T10:27:33.015430+0000","last_became_peered":"2024-04-20T10:27:33.015430+0000","last_unstale":"2024-04-20T10:27:38.112358+0000","last_undegraded":"2024-04-20T10:27:38.112358+0000","last_fullsized":"2024-04-20T10:27:38.112358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389282+0000","last_change":"2024-04-20T10:27:33.024105+0000","last_active":"2024-04-20T10:27:38.389282+0000","last_peered":"2024-04-20T10:27:38.389282+0000","last_clean":"2024-04-20T10:27:38.389282+0000","last_became_active":"2024-04-20T10:27:33.023418+0000","last_became_peered":"2024-04-20T10:27:33.023418+0000","last_unstale":"2024-04-20T10:27:38.389282+0000","last_undegraded":"2024-04-20T10:27:38.389282+0000","last_fullsized":"2024-04-20T10:27:38.389282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4499,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509448,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":110867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509448,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":113986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509448,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":700723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509448,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542153,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":114799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:27:56.190 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T10:27:56.190 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-20T10:27:56.190 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-20T10:27:56.191 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-20T10:27:56.191 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-20T10:27:56.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.451+0000 7fb989c00640 1 Processor -- start 2024-04-20T10:27:56.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.451+0000 7fb989c00640 1 -- start start 2024-04-20T10:27:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.453+0000 7fb989c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 0x7fb9840ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.453+0000 7fb989c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb984141c60 con 0x7fb984156c20 2024-04-20T10:27:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.453+0000 7fb989c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb98415cf90 con 0x7fb9840ad8e0 2024-04-20T10:27:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.453+0000 7fb989c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb98406c010 con 0x7fb98415ca40 2024-04-20T10:27:56.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.453+0000 7fb982a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb984156c20 0x7fb98406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33612/0 (socket says 172.21.15.47:33612) 2024-04-20T10:27:56.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.453+0000 7fb982a00640 1 -- 172.21.15.47:0/3104260917 learned_addr learned my addr 172.21.15.47:0/3104260917 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb982000640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 0x7fb9840ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 57689963 0 0) 0x7fb984141c60 con 0x7fb984156c20 2024-04-20T10:27:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb968003520 con 0x7fb984156c20 2024-04-20T10:27:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1983527797 0 0) 0x7fb968003520 con 0x7fb984156c20 2024-04-20T10:27:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 msgr2=0x7fb9840ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 0x7fb9840ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 >> v1:172.21.15.175:6789/0 conn(0x7fb9840ad8e0 legacy=0x7fb9840d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98415d270 con 0x7fb984156c20 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fb974001670 con 0x7fb984156c20 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.454+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 >> v1:172.21.15.47:6789/0 conn(0x7fb984156c20 legacy=0x7fb98406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb981600640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 0x7fb9680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb981600640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb968004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb981600640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb968004d40 0x7fb9680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb98406c010 con 0x7fb968003f60 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb96800baa0 con 0x7fb968004d40 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb96800bca0 con 0x7fb98415ca40 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb982a00640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 0x7fb9680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb983400640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb968004d40 0x7fb9680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.455+0000 7fb982000640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb968004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.456+0000 7fb982000640 1 -- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 msgr2=0x7fb9680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.456+0000 7fb982000640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415ca40 0x7fb9680039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.456+0000 7fb982000640 1 -- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb968004d40 msgr2=0x7fb9680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:56.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.456+0000 7fb982000640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb968004d40 0x7fb9680051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.456+0000 7fb982000640 1 -- 172.21.15.47:0/3104260917 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb968003520 con 0x7fb968003f60 2024-04-20T10:27:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.457+0000 7fb982000640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb968004800 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009fc0 tx=0x7fb968005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f9a913a9b02d33 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.457+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c039070 con 0x7fb968003f60 2024-04-20T10:27:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.457+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb96c02cc20 con 0x7fb968003f60 2024-04-20T10:27:56.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.457+0000 7fb981600640 1 -- 172.21.15.47:0/3104260917 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb96c009c10 con 0x7fb968003f60 2024-04-20T10:27:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.458+0000 7fb989c00640 1 -- 172.21.15.47:0/3104260917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 msgr2=0x7fb968004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.458+0000 7fb989c00640 1 --2- 172.21.15.47:0/3104260917 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb968004800 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb96c009fc0 tx=0x7fb968005830 comp rx=0 tx=0).stop 2024-04-20T10:27:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.458+0000 7fb983400640 1 -- 172.21.15.47:0/3104260917 reap_dead start 2024-04-20T10:27:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.458+0000 7fb989c00640 1 -- 172.21.15.47:0/3104260917 shutdown_connections 2024-04-20T10:27:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.458+0000 7fb989c00640 1 -- 172.21.15.47:0/3104260917 >> 172.21.15.47:0/3104260917 conn(0x7fb9840b3350 msgr2=0x7fb9840b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.458+0000 7fb989c00640 1 -- 172.21.15.47:0/3104260917 shutdown_connections 2024-04-20T10:27:56.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.458+0000 7fb989c00640 1 -- 172.21.15.47:0/3104260917 wait complete. 2024-04-20T10:27:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.460+0000 7fb989c00640 1 Processor -- start 2024-04-20T10:27:56.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.460+0000 7fb989c00640 1 -- start start 2024-04-20T10:27:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb989c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb98406d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb989c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb984155480 0x7fb98415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb989c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415b950 0x7fb9840b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb989c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb9840d0a70 con 0x7fb968003f60 2024-04-20T10:27:56.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb989c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb98415d830 con 0x7fb984155480 2024-04-20T10:27:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb989c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9840d0bf0 con 0x7fb98415b950 2024-04-20T10:27:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb982a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb98406d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb982a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb98406d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55794/0 (socket says 172.21.15.47:55794) 2024-04-20T10:27:56.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.461+0000 7fb982a00640 1 -- 172.21.15.47:0/4061154991 learned_addr learned my addr 172.21.15.47:0/4061154991 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb982a00640 1 -- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415b950 msgr2=0x7fb9840b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb982a00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415b950 0x7fb9840b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb982a00640 1 -- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb984155480 msgr2=0x7fb98415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb982000640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb984155480 0x7fb98415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb982a00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb984155480 0x7fb98415b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb982a00640 1 -- 172.21.15.47:0/4061154991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9840ce800 con 0x7fb968003f60 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb982a00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb98406d030 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb9740072c0 tx=0x7fb974007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:56.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb97aa00640 1 -- 172.21.15.47:0/4061154991 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97400f860 con 0x7fb968003f60 2024-04-20T10:27:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb97aa00640 1 -- 172.21.15.47:0/4061154991 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb974018070 con 0x7fb968003f60 2024-04-20T10:27:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb97aa00640 1 -- 172.21.15.47:0/4061154991 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97400fc20 con 0x7fb968003f60 2024-04-20T10:27:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9840ceae0 con 0x7fb968003f60 2024-04-20T10:27:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.462+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9840b21c0 con 0x7fb968003f60 2024-04-20T10:27:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.464+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98415bd70 con 0x7fb968003f60 2024-04-20T10:27:56.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.465+0000 7fb97aa00640 1 -- 172.21.15.47:0/4061154991 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb97401d460 con 0x7fb968003f60 2024-04-20T10:27:56.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.466+0000 7fb97aa00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb96006c940 0x7fb96006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.466+0000 7fb97aa00640 1 -- 172.21.15.47:0/4061154991 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb97408beb0 con 0x7fb968003f60 2024-04-20T10:27:56.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.466+0000 7fb982000640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb96006c940 0x7fb96006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.467+0000 7fb982000640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb96006c940 0x7fb96006ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036000 tx=0x7fb96c02e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:56.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.470+0000 7fb97aa00640 1 -- 172.21.15.47:0/4061154991 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb9740599c0 con 0x7fb968003f60 2024-04-20T10:27:56.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.724+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fb984068980 con 0x7fb968003f60 2024-04-20T10:27:56.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.730+0000 7fb97aa00640 1 -- 172.21.15.47:0/4061154991 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+174143 (secure 0 0 0) 0x7fb97405dcc0 con 0x7fb968003f60 2024-04-20T10:27:56.732 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:56.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb96006c940 msgr2=0x7fb96006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb96006c940 0x7fb96006ee00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb96c036000 tx=0x7fb96c02e000 comp rx=0 tx=0).stop 2024-04-20T10:27:56.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 msgr2=0x7fb98406d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb98406d030 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb9740072c0 tx=0x7fb974007780 comp rx=0 tx=0).stop 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 shutdown_connections 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb98415b950 0x7fb9840b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb984155480 0x7fb98415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb96006c940 0x7fb96006ee00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 --2- 172.21.15.47:0/4061154991 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb968003f60 0x7fb98406d030 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 >> 172.21.15.47:0/4061154991 conn(0x7fb9840b3350 msgr2=0x7fb984154d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 shutdown_connections 2024-04-20T10:27:56.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.735+0000 7fb989c00640 1 -- 172.21.15.47:0/4061154991 wait complete. 2024-04-20T10:27:56.755 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":5,"flags":0,"active_gid":4112,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":1546709378},{"type":"v1","addr":"172.21.15.175:6833","nonce":1546709378}]},"active_addr":"172.21.15.175:6833/1546709378","active_change":"2024-04-20T10:27:13.036651+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2360424156}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3594774984}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2673243924}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":162571004}]}]} 2024-04-20T10:27:56.758 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-20T10:27:56.759 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-20T10:27:56.759 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:56.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.974+0000 7fa64ae00640 1 Processor -- start 2024-04-20T10:27:56.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.974+0000 7fa64ae00640 1 -- start start 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.976+0000 7fa64ae00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa64415b400 0x7fa644072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.976+0000 7fa64ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa64415f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.976+0000 7fa64ae00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa64406ccc0 con 0x7fa64415f870 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.976+0000 7fa64ae00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa64406d280 con 0x7fa64415b400 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.976+0000 7fa64ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa64406cfa0 con 0x7fa64415eec0 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.977+0000 7fa649a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa64415b400 0x7fa644072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.977+0000 7fa649a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa64415b400 0x7fa644072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40658/0 (socket says 172.21.15.47:40658) 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.977+0000 7fa649a00640 1 -- 172.21.15.47:0/2764798455 learned_addr learned my addr 172.21.15.47:0/2764798455 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:56.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.977+0000 7fa649000640 1 --2- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa64415f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.978+0000 7fa649a00640 1 -- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 msgr2=0x7fa64415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:56.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.978+0000 7fa649a00640 1 --2- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa64415f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.978+0000 7fa649a00640 1 -- 172.21.15.47:0/2764798455 >> v1:172.21.15.175:6789/0 conn(0x7fa64415f870 legacy=0x7fa64415b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.978+0000 7fa649a00640 1 -- 172.21.15.47:0/2764798455 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa64406e440 con 0x7fa64415b400 2024-04-20T10:27:56.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.979+0000 7fa649a00640 1 --2- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 0x7fa644072d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa638009870 tx=0x7fa63802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e19667bd2c936d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.980+0000 7fa643e00640 1 -- 172.21.15.47:0/2764798455 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638034070 con 0x7fa64415b400 2024-04-20T10:27:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.980+0000 7fa643e00640 1 -- 172.21.15.47:0/2764798455 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa638030840 con 0x7fa64415b400 2024-04-20T10:27:56.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.980+0000 7fa643e00640 1 -- 172.21.15.47:0/2764798455 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa638030c10 con 0x7fa64415b400 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 -- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 msgr2=0x7fa644072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 --2- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 0x7fa644072d00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa638009870 tx=0x7fa63802b980 comp rx=0 tx=0).stop 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 -- 172.21.15.47:0/2764798455 shutdown_connections 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 --2- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa64415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 --2- 172.21.15.47:0/2764798455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 0x7fa644072d00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 -- 172.21.15.47:0/2764798455 >> 172.21.15.47:0/2764798455 conn(0x7fa6440d0040 msgr2=0x7fa6440d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 -- 172.21.15.47:0/2764798455 shutdown_connections 2024-04-20T10:27:56.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.981+0000 7fa64ae00640 1 -- 172.21.15.47:0/2764798455 wait complete. 2024-04-20T10:27:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.983+0000 7fa64ae00640 1 Processor -- start 2024-04-20T10:27:56.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.983+0000 7fa64ae00640 1 -- start start 2024-04-20T10:27:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 0x7fa6440cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa6440cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64415f870 0x7fa6441f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa644073240 con 0x7fa64415f870 2024-04-20T10:27:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa64415fc90 con 0x7fa64415b400 2024-04-20T10:27:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa64406fc50 con 0x7fa64415eec0 2024-04-20T10:27:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64415f870 0x7fa6441f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64a400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64415f870 0x7fa6441f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55800/0 (socket says 172.21.15.47:55800) 2024-04-20T10:27:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa64a400640 1 -- 172.21.15.47:0/4044558749 learned_addr learned my addr 172.21.15.47:0/4044558749 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa649000640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa6440cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.985+0000 7fa649a00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 0x7fa6440cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.986+0000 7fa64a400640 1 -- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 msgr2=0x7fa6440cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.986+0000 7fa64a400640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa6440cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.986+0000 7fa64a400640 1 -- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 msgr2=0x7fa6440cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.986+0000 7fa64a400640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 0x7fa6440cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.986+0000 7fa64a400640 1 -- 172.21.15.47:0/4044558749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6440cbd60 con 0x7fa64415f870 2024-04-20T10:27:56.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.986+0000 7fa64a400640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64415f870 0x7fa6441f7e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c6b0 tx=0x7fa63c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.987+0000 7fa641600640 1 -- 172.21.15.47:0/4044558749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c003b70 con 0x7fa64415f870 2024-04-20T10:27:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.987+0000 7fa641600640 1 -- 172.21.15.47:0/4044558749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa63c003d30 con 0x7fa64415f870 2024-04-20T10:27:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.987+0000 7fa641600640 1 -- 172.21.15.47:0/4044558749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa63c005310 con 0x7fa64415f870 2024-04-20T10:27:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.987+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa64415e9a0 con 0x7fa64415f870 2024-04-20T10:27:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.987+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6440ca080 con 0x7fa64415f870 2024-04-20T10:27:56.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.988+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa644075b60 con 0x7fa64415f870 2024-04-20T10:27:56.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.989+0000 7fa641600640 1 -- 172.21.15.47:0/4044558749 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa63c0054b0 con 0x7fa64415f870 2024-04-20T10:27:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.991+0000 7fa641600640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa62806c8f0 0x7fa62806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.991+0000 7fa641600640 1 -- 172.21.15.47:0/4044558749 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa63c08bc30 con 0x7fa64415f870 2024-04-20T10:27:56.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.992+0000 7fa649a00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa62806c8f0 0x7fa62806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:56.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.993+0000 7fa649a00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa62806c8f0 0x7fa62806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa638036000 tx=0x7fa638008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:56.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:56.995+0000 7fa641600640 1 -- 172.21.15.47:0/4044558749 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa63c059630 con 0x7fa64415f870 2024-04-20T10:27:57.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.209+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa64414f1b0 con 0x7fa64415f870 2024-04-20T10:27:57.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.211+0000 7fa641600640 1 -- 172.21.15.47:0/4044558749 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7fa63c05d930 con 0x7fa64415f870 2024-04-20T10:27:57.212 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:57.212 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa62806c8f0 msgr2=0x7fa62806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa62806c8f0 0x7fa62806edb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa638036000 tx=0x7fa638008030 comp rx=0 tx=0).stop 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64415f870 msgr2=0x7fa6441f7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64415f870 0x7fa6441f7e50 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c6b0 tx=0x7fa63c00cb70 comp rx=0 tx=0).stop 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 shutdown_connections 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa62806c8f0 0x7fa62806edb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa64415f870 0x7fa6441f7e50 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa64415eec0 0x7fa6440cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 --2- 172.21.15.47:0/4044558749 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa64415b400 0x7fa6440cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 >> 172.21.15.47:0/4044558749 conn(0x7fa6440d0040 msgr2=0x7fa6440b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 shutdown_connections 2024-04-20T10:27:57.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.214+0000 7fa64ae00640 1 -- 172.21.15.47:0/4044558749 wait complete. 2024-04-20T10:27:57.233 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-20T10:27:57.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:27:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.440+0000 7fc271000640 1 Processor -- start 2024-04-20T10:27:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.440+0000 7fc271000640 1 -- start start 2024-04-20T10:27:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.441+0000 7fc271000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc26c15e660 0x7fc26c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.441+0000 7fc271000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc26c0b41a0 0x7fc26c0b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.441+0000 7fc271000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc26c15eeb0 con 0x7fc26c0d0370 2024-04-20T10:27:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.441+0000 7fc271000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc26c15f190 con 0x7fc26c15e660 2024-04-20T10:27:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.441+0000 7fc271000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc26c15f470 con 0x7fc26c0b41a0 2024-04-20T10:27:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.442+0000 7fc26aa00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc26c0d0370 0x7fc26c0b3a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33618/0 (socket says 172.21.15.47:33618) 2024-04-20T10:27:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.442+0000 7fc26aa00640 1 -- 172.21.15.47:0/1377327218 learned_addr learned my addr 172.21.15.47:0/1377327218 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.442+0000 7fc26b400640 1 --2- 172.21.15.47:0/1377327218 >> v2:172.21.15.175:3300/0 conn(0x7fc26c15e660 0x7fc26c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.442+0000 7fc26be00640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc26c0b41a0 0x7fc26c0b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.442+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253372490 0 0) 0x7fc26c15eeb0 con 0x7fc26c0d0370 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.442+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc250003520 con 0x7fc26c0d0370 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1606168854 0 0) 0x7fc250003520 con 0x7fc26c0d0370 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc26c0b41a0 msgr2=0x7fc26c0b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc26c0b41a0 0x7fc26c0b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 >> v2:172.21.15.175:3300/0 conn(0x7fc26c15e660 msgr2=0x7fc26c0cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 --2- 172.21.15.47:0/1377327218 >> v2:172.21.15.175:3300/0 conn(0x7fc26c15e660 0x7fc26c0cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c15f750 con 0x7fc26c0d0370 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fc254001690 con 0x7fc26c0d0370 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 >> v1:172.21.15.47:6789/0 conn(0x7fc26c0d0370 legacy=0x7fc26c0b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2500039b0 0x7fc250003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc250004800 0x7fc250004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2500051c0 0x7fc25000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c15f190 con 0x7fc2500039b0 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c15f470 con 0x7fc2500051c0 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc25000bf30 con 0x7fc250004800 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26be00640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc26c0b41a0 0x7fc26c0b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26b400640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2500039b0 0x7fc250003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26b400640 1 -- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc250004800 msgr2=0x7fc250004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26b400640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc250004800 0x7fc250004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26b400640 1 -- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2500051c0 msgr2=0x7fc25000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.443+0000 7fc26b400640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2500051c0 0x7fc25000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.444+0000 7fc26b400640 1 -- 172.21.15.47:0/1377327218 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc250003520 con 0x7fc2500039b0 2024-04-20T10:27:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.445+0000 7fc26b400640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2500039b0 0x7fc250003dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009ac0 tx=0x7fc250005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9fb74714f6ff6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.445+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c039070 con 0x7fc2500039b0 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.445+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc25c02c710 con 0x7fc2500039b0 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.445+0000 7fc26a000640 1 -- 172.21.15.47:0/1377327218 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25c009690 con 0x7fc2500039b0 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.445+0000 7fc271000640 1 -- 172.21.15.47:0/1377327218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2500039b0 msgr2=0x7fc250003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.445+0000 7fc271000640 1 --2- 172.21.15.47:0/1377327218 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2500039b0 0x7fc250003dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009ac0 tx=0x7fc250005830 comp rx=0 tx=0).stop 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.445+0000 7fc26be00640 1 -- 172.21.15.47:0/1377327218 reap_dead start 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.446+0000 7fc271000640 1 -- 172.21.15.47:0/1377327218 shutdown_connections 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.446+0000 7fc271000640 1 -- 172.21.15.47:0/1377327218 >> 172.21.15.47:0/1377327218 conn(0x7fc26c0cece0 msgr2=0x7fc26c0cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.446+0000 7fc271000640 1 -- 172.21.15.47:0/1377327218 shutdown_connections 2024-04-20T10:27:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.446+0000 7fc271000640 1 -- 172.21.15.47:0/1377327218 wait complete. 2024-04-20T10:27:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.447+0000 7fc271000640 1 Processor -- start 2024-04-20T10:27:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.448+0000 7fc271000640 1 -- start start 2024-04-20T10:27:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.448+0000 7fc271000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2500039b0 0x7fc26c1f8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc271000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26c15d200 0x7fc26c1f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc271000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c15d680 0x7fc26c1f7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc271000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c0b1c40 con 0x7fc26c15d680 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc271000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc26c0b1ac0 con 0x7fc26c15d200 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc271000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc26c0b1dc0 con 0x7fc2500039b0 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c15d680 0x7fc26c1f7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c15d680 0x7fc26c1f7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55822/0 (socket says 172.21.15.47:55822) 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 -- 172.21.15.47:0/1328811726 learned_addr learned my addr 172.21.15.47:0/1328811726 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26aa00640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26c15d200 0x7fc26c1f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 -- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2500039b0 msgr2=0x7fc26c1f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2500039b0 0x7fc26c1f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 -- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26c15d200 msgr2=0x7fc26c1f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26c15d200 0x7fc26c1f3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.449+0000 7fc26be00640 1 -- 172.21.15.47:0/1328811726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc26c0b6960 con 0x7fc26c15d680 2024-04-20T10:27:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.450+0000 7fc26be00640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c15d680 0x7fc26c1f7c90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0d2230 tx=0x7fc26400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.450+0000 7fc263400640 1 -- 172.21.15.47:0/1328811726 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26401a070 con 0x7fc26c15d680 2024-04-20T10:27:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.450+0000 7fc263400640 1 -- 172.21.15.47:0/1328811726 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc264007e20 con 0x7fc26c15d680 2024-04-20T10:27:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.450+0000 7fc263400640 1 -- 172.21.15.47:0/1328811726 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc264003bd0 con 0x7fc26c15d680 2024-04-20T10:27:57.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.450+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc26c0b6c40 con 0x7fc26c15d680 2024-04-20T10:27:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.450+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc26c0b7200 con 0x7fc26c15d680 2024-04-20T10:27:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.451+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc26c0b7a00 con 0x7fc26c15d680 2024-04-20T10:27:57.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.452+0000 7fc263400640 1 -- 172.21.15.47:0/1328811726 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc26401d070 con 0x7fc26c15d680 2024-04-20T10:27:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.453+0000 7fc263400640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc24806c8f0 0x7fc24806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.453+0000 7fc263400640 1 -- 172.21.15.47:0/1328811726 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc26408bd70 con 0x7fc26c15d680 2024-04-20T10:27:57.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.454+0000 7fc26b400640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc24806c8f0 0x7fc24806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.455+0000 7fc26b400640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc24806c8f0 0x7fc24806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009ac0 tx=0x7fc25c02e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:57.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.458+0000 7fc263400640 1 -- 172.21.15.47:0/1328811726 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc26c0b7a00 con 0x7fc26c15d680 2024-04-20T10:27:57.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.666+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc26c0b81e0 con 0x7fc26c15d680 2024-04-20T10:27:57.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.669+0000 7fc263400640 1 -- 172.21.15.47:0/1328811726 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7fc264010070 con 0x7fc26c15d680 2024-04-20T10:27:57.669 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:27:57.670 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:27:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc24806c8f0 msgr2=0x7fc24806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc24806c8f0 0x7fc24806edb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc25c009ac0 tx=0x7fc25c02e000 comp rx=0 tx=0).stop 2024-04-20T10:27:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c15d680 msgr2=0x7fc26c1f7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c15d680 0x7fc26c1f7c90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc26c0d2230 tx=0x7fc26400d4a0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 shutdown_connections 2024-04-20T10:27:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc24806c8f0 0x7fc24806edb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc26c15d680 0x7fc26c1f7c90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc26c15d200 0x7fc26c1f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 --2- 172.21.15.47:0/1328811726 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2500039b0 0x7fc26c1f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 >> 172.21.15.47:0/1328811726 conn(0x7fc26c0cece0 msgr2=0x7fc26c0cc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 shutdown_connections 2024-04-20T10:27:57.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.672+0000 7fc271000640 1 -- 172.21.15.47:0/1328811726 wait complete. 2024-04-20T10:27:57.691 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-20T10:27:57.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-20T10:27:57.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-20T10:27:57.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-20T10:27:57.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-20T10:27:57.693 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-20T10:27:57.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-20T10:27:57.694 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-20T10:27:57.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.989+0000 7fe317200640 1 Processor -- start 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.989+0000 7fe317200640 1 -- start start 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.991+0000 7fe317200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe3100e3160 0x7fe3100e7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.991+0000 7fe317200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe310053cb0 0x7fe310054100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.991+0000 7fe317200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe310058810 con 0x7fe31003c0a0 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.991+0000 7fe317200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe310007b70 con 0x7fe310053cb0 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.991+0000 7fe317200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe310058af0 con 0x7fe3100e3160 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.992+0000 7fe316800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe310053cb0 0x7fe310054100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.992+0000 7fe316800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe310053cb0 0x7fe310054100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55832/0 (socket says 172.21.15.47:55832) 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.992+0000 7fe316800640 1 -- 172.21.15.47:0/1064145367 learned_addr learned my addr 172.21.15.47:0/1064145367 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:57.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.993+0000 7fe315e00640 1 --2- 172.21.15.47:0/1064145367 >> v2:172.21.15.175:3300/0 conn(0x7fe3100e3160 0x7fe3100e7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:57.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.993+0000 7fe314a00640 1 -- 172.21.15.47:0/1064145367 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1606022105 0 0) 0x7fe310058810 con 0x7fe31003c0a0 2024-04-20T10:27:57.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.994+0000 7fe316800640 1 -- 172.21.15.47:0/1064145367 >> v2:172.21.15.175:3300/0 conn(0x7fe3100e3160 msgr2=0x7fe3100e7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:57.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.994+0000 7fe316800640 1 --2- 172.21.15.47:0/1064145367 >> v2:172.21.15.175:3300/0 conn(0x7fe3100e3160 0x7fe3100e7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.994+0000 7fe316800640 1 -- 172.21.15.47:0/1064145367 >> v1:172.21.15.175:6789/0 conn(0x7fe31003c0a0 legacy=0x7fe310058050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.994+0000 7fe316800640 1 -- 172.21.15.47:0/1064145367 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe310006360 con 0x7fe310053cb0 2024-04-20T10:27:57.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.995+0000 7fe316800640 1 --2- 172.21.15.47:0/1064145367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe310054100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe31802db60 tx=0x7fe3180a7260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f8e1c65d7d46f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:57.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.995+0000 7fe314a00640 1 -- 172.21.15.47:0/1064145367 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180b4070 con 0x7fe310053cb0 2024-04-20T10:27:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.995+0000 7fe314a00640 1 -- 172.21.15.47:0/1064145367 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe3180af640 con 0x7fe310053cb0 2024-04-20T10:27:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.995+0000 7fe314a00640 1 -- 172.21.15.47:0/1064145367 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3180afab0 con 0x7fe310053cb0 2024-04-20T10:27:57.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.996+0000 7f4a76400640 1 Processor -- start 2024-04-20T10:27:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.996+0000 7f4a76400640 1 -- start start 2024-04-20T10:27:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.996+0000 7fe317200640 1 -- 172.21.15.47:0/1064145367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 msgr2=0x7fe310054100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.996+0000 7fe317200640 1 --2- 172.21.15.47:0/1064145367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe310054100 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe31802db60 tx=0x7fe3180a7260 comp rx=0 tx=0).stop 2024-04-20T10:27:57.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.997+0000 7fe317200640 1 -- 172.21.15.47:0/1064145367 shutdown_connections 2024-04-20T10:27:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.997+0000 7fe317200640 1 --2- 172.21.15.47:0/1064145367 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe310054100 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.997+0000 7fe317200640 1 --2- 172.21.15.47:0/1064145367 >> v2:172.21.15.175:3300/0 conn(0x7fe3100e3160 0x7fe3100e7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.997+0000 7fe317200640 1 -- 172.21.15.47:0/1064145367 >> 172.21.15.47:0/1064145367 conn(0x7fe310058ee0 msgr2=0x7fe3100592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.997+0000 7fe317200640 1 -- 172.21.15.47:0/1064145367 shutdown_connections 2024-04-20T10:27:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.997+0000 7fe317200640 1 -- 172.21.15.47:0/1064145367 wait complete. 2024-04-20T10:27:57.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.998+0000 7f3456000640 1 Processor -- start 2024-04-20T10:27:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.998+0000 7f4a76400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4a680548a0 0x7f4a68054ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.998+0000 7f4a76400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a6803c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.998+0000 7f4a76400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a680e7e30 con 0x7f4a680e7a50 2024-04-20T10:27:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.998+0000 7f4a76400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4a6803b5a0 con 0x7f4a680548a0 2024-04-20T10:27:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.998+0000 7f4a76400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6803b880 con 0x7f4a6803c360 2024-04-20T10:27:57.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.998+0000 7f3456000640 1 -- start start 2024-04-20T10:27:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.999+0000 7f4a75000640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f4a680e7a50 0x7f4a68049b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47164/0 (socket says 172.21.15.47:47164) 2024-04-20T10:27:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.999+0000 7f4a75000640 1 -- 172.21.15.47:0/3803847348 learned_addr learned my addr 172.21.15.47:0/3803847348 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.999+0000 7f1553200640 1 Processor -- start 2024-04-20T10:27:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.999+0000 7f1553200640 1 -- start start 2024-04-20T10:27:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.999+0000 7f4a6fe00640 1 --2- 172.21.15.47:0/3803847348 >> v2:172.21.15.175:3300/0 conn(0x7f4a680548a0 0x7f4a68054ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:57.999+0000 7f4a75a00640 1 --2- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a6803c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7fe317200640 1 Processor -- start 2024-04-20T10:27:58.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7f3456000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3450145d80 0x7f34500716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7fe317200640 1 -- start start 2024-04-20T10:27:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7f3456000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3450071c20 0x7f34500c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7f3456000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f34500c7760 0x7f345015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7f3456000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3450072080 con 0x7f3450071c20 2024-04-20T10:27:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7f3456000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3450160200 con 0x7f34500c7760 2024-04-20T10:27:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.000+0000 7f3456000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34501604a0 con 0x7f3450145d80 2024-04-20T10:27:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f344f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3450145d80 0x7f34500716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f4a6fe00640 1 -- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 msgr2=0x7f4a6803c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f344f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3450145d80 0x7f34500716e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54096/0 (socket says 172.21.15.47:54096) 2024-04-20T10:27:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f344f400640 1 -- 172.21.15.47:0/3453501746 learned_addr learned my addr 172.21.15.47:0/3453501746 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f4a6fe00640 1 --2- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a6803c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f4a6fe00640 1 -- 172.21.15.47:0/3803847348 >> v1:172.21.15.175:6789/0 conn(0x7f4a680e7a50 legacy=0x7f4a68049b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f4a6fe00640 1 -- 172.21.15.47:0/3803847348 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6803bb60 con 0x7f4a680548a0 2024-04-20T10:27:58.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.001+0000 7f4a75a00640 1 --2- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a6803c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7f344fe00640 1 --2- 172.21.15.47:0/3453501746 >> v2:172.21.15.175:3300/0 conn(0x7f34500c7760 0x7f345015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7fe317200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe31003c0a0 0x7fe31003d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7f1553200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548038630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7f1553200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154804f540 con 0x7f1548050dd0 2024-04-20T10:27:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7fe317200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe31003da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7f1553200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f154804f820 con 0x7f15480e1090 2024-04-20T10:27:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7f1553200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f154804fb00 con 0x7f154803a5d0 2024-04-20T10:27:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7fe317200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3100e3160 0x7fe3100e3730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7fe315e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe31003c0a0 0x7fe31003d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.002+0000 7f344ea00640 1 --2- 172.21.15.47:0/3453501746 >> v2:172.21.15.47:3300/0 conn(0x7f3450071c20 0x7f34500c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7f344fe00640 1 -- 172.21.15.47:0/3453501746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3450145d80 msgr2=0x7f34500716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7f344fe00640 1 --2- 172.21.15.47:0/3453501746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3450145d80 0x7f34500716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7f344fe00640 1 -- 172.21.15.47:0/3453501746 >> v2:172.21.15.47:3300/0 conn(0x7f3450071c20 msgr2=0x7f34500c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7f344fe00640 1 --2- 172.21.15.47:0/3453501746 >> v2:172.21.15.47:3300/0 conn(0x7f3450071c20 0x7f34500c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7f344fe00640 1 -- 172.21.15.47:0/3453501746 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3450160780 con 0x7f34500c7760 2024-04-20T10:27:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7fe317200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe3100e3540 con 0x7fe310053cb0 2024-04-20T10:27:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7fe317200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe310054640 con 0x7fe3100e3160 2024-04-20T10:27:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7fe317200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe310006b60 con 0x7fe31003c0a0 2024-04-20T10:27:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7fe315400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe31003da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7fe315400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe31003da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55850/0 (socket says 172.21.15.47:55850) 2024-04-20T10:27:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7fe315400640 1 -- 172.21.15.47:0/2301066733 learned_addr learned my addr 172.21.15.47:0/2301066733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7f1551400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1548050dd0 0x7f1548005f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33634/0 (socket says 172.21.15.47:33634) 2024-04-20T10:27:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.003+0000 7f1551400640 1 -- 172.21.15.47:0/1472970472 learned_addr learned my addr 172.21.15.47:0/1472970472 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7f1552800640 1 --2- 172.21.15.47:0/1472970472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548038630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7f1550a00640 1 -- 172.21.15.47:0/1472970472 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 891366501 0 0) 0x7f154804f540 con 0x7f1548050dd0 2024-04-20T10:27:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7f1550a00640 1 -- 172.21.15.47:0/1472970472 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1534003520 con 0x7f1548050dd0 2024-04-20T10:27:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7f344fe00640 1 --2- 172.21.15.47:0/3453501746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500c7760 0x7f345015cb60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f34500b3650 tx=0x7f344802be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4abeee594f29c3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7f1550a00640 1 -- 172.21.15.47:0/1472970472 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546349765 0 0) 0x7f154804f820 con 0x7f15480e1090 2024-04-20T10:27:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7f1550a00640 1 -- 172.21.15.47:0/1472970472 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f154804f540 con 0x7f15480e1090 2024-04-20T10:27:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.005+0000 7f1552800640 1 -- 172.21.15.47:0/1472970472 >> v1:172.21.15.175:6789/0 conn(0x7f15480e1090 legacy=0x7f1548039ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.005+0000 7f1552800640 1 -- 172.21.15.47:0/1472970472 >> v1:172.21.15.47:6789/0 conn(0x7f1548050dd0 legacy=0x7f1548005f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.005+0000 7f1552800640 1 -- 172.21.15.47:0/1472970472 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f154804fde0 con 0x7f154803a5d0 2024-04-20T10:27:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.005+0000 7f4a6fe00640 1 --2- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 0x7f4a68054ce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0097a0 tx=0x7f4a5c02b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6efb30ff1eb060cb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7fe315400640 1 -- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe31003c0a0 msgr2=0x7fe31003d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7fe315400640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe31003c0a0 0x7fe31003d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7fe315400640 1 -- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3100e3160 msgr2=0x7fe3100e3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7fe315400640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3100e3160 0x7fe3100e3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.004+0000 7fe315400640 1 -- 172.21.15.47:0/2301066733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31017ce10 con 0x7fe310053cb0 2024-04-20T10:27:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.005+0000 7f344e000640 1 -- 172.21.15.47:0/3453501746 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448039070 con 0x7f34500c7760 2024-04-20T10:27:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.005+0000 7f78a3600640 1 Processor -- start 2024-04-20T10:27:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.005+0000 7f4a6f400640 1 -- 172.21.15.47:0/3803847348 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c0315a0 con 0x7f4a680548a0 2024-04-20T10:27:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f4a6f400640 1 -- 172.21.15.47:0/3803847348 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a5c037070 con 0x7f4a680548a0 2024-04-20T10:27:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f344e000640 1 -- 172.21.15.47:0/3453501746 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f344802cbe0 con 0x7f34500c7760 2024-04-20T10:27:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f344e000640 1 -- 172.21.15.47:0/3453501746 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3448034440 con 0x7f34500c7760 2024-04-20T10:27:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f4a6f400640 1 -- 172.21.15.47:0/3803847348 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c031960 con 0x7f4a680548a0 2024-04-20T10:27:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f78a3600640 1 -- start start 2024-04-20T10:27:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7fe315400640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe31003da10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c460 tx=0x7fe30400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f6c1ce00640 1 Processor -- start 2024-04-20T10:27:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f1552800640 1 --2- 172.21.15.47:0/1472970472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548038630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f15400097d0 tx=0x7f154002b940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b7ee8e723df9057 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f3456000640 1 -- 172.21.15.47:0/3453501746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500c7760 msgr2=0x7f345015cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.006+0000 7f3456000640 1 --2- 172.21.15.47:0/3453501746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500c7760 0x7f345015cb60 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f34500b3650 tx=0x7f344802be30 comp rx=0 tx=0).stop 2024-04-20T10:27:58.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 -- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 msgr2=0x7f4a68054ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 --2- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 0x7f4a68054ce0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c0097a0 tx=0x7f4a5c02b840 comp rx=0 tx=0).stop 2024-04-20T10:27:58.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 -- 172.21.15.47:0/3803847348 shutdown_connections 2024-04-20T10:27:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 --2- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a6803c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 --2- 172.21.15.47:0/3803847348 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 0x7f4a68054ce0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 -- 172.21.15.47:0/3803847348 >> 172.21.15.47:0/3803847348 conn(0x7f4a68056cd0 msgr2=0x7f4a680570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f1550a00640 1 -- 172.21.15.47:0/1472970472 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540034070 con 0x7f154803a5d0 2024-04-20T10:27:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f1550a00640 1 -- 172.21.15.47:0/1472970472 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1540030800 con 0x7f154803a5d0 2024-04-20T10:27:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f1550a00640 1 -- 172.21.15.47:0/1472970472 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1540030bd0 con 0x7f154803a5d0 2024-04-20T10:27:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe304005480 con 0x7fe310053cb0 2024-04-20T10:27:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 -- 172.21.15.47:0/3803847348 shutdown_connections 2024-04-20T10:27:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f4a76400640 1 -- 172.21.15.47:0/3803847348 wait complete. 2024-04-20T10:27:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3100e44a0 con 0x7fe310053cb0 2024-04-20T10:27:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31017c330 con 0x7fe310053cb0 2024-04-20T10:27:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.008+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe304005480 con 0x7fe310053cb0 2024-04-20T10:27:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.008+0000 7f78a3600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f789c0d0360 0x7f789c0d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.008+0000 7f78a3600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789c141c60 con 0x7f789c156c20 2024-04-20T10:27:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.008+0000 7f78a3600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f789c0adab0 con 0x7f789c15ca40 2024-04-20T10:27:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.008+0000 7f78a3600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f789c0d0d10 con 0x7f789c0d0360 2024-04-20T10:27:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f3456000640 1 -- 172.21.15.47:0/3453501746 shutdown_connections 2024-04-20T10:27:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f3456000640 1 --2- 172.21.15.47:0/3453501746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34500c7760 0x7f345015cb60 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f3456000640 1 --2- 172.21.15.47:0/3453501746 >> v2:172.21.15.47:3300/0 conn(0x7f3450071c20 0x7f34500c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f3456000640 1 --2- 172.21.15.47:0/3453501746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3450145d80 0x7f34500716e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f3456000640 1 -- 172.21.15.47:0/3453501746 >> 172.21.15.47:0/3453501746 conn(0x7f34500d2340 msgr2=0x7f34500d2720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f3456000640 1 -- 172.21.15.47:0/3453501746 shutdown_connections 2024-04-20T10:27:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f3456000640 1 -- 172.21.15.47:0/3453501746 wait complete. 2024-04-20T10:27:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.009+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe2f0001030 con 0x7fe310053cb0 2024-04-20T10:27:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.009+0000 7ff415a00640 1 Processor -- start 2024-04-20T10:27:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.009+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3040035f0 con 0x7fe310053cb0 2024-04-20T10:27:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.009+0000 7f78a0c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f789c156c20 0x7f789c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33638/0 (socket says 172.21.15.47:33638) 2024-04-20T10:27:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.009+0000 7f78a0c00640 1 -- 172.21.15.47:0/4013179692 learned_addr learned my addr 172.21.15.47:0/4013179692 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.009+0000 7ff415a00640 1 -- start start 2024-04-20T10:27:58.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.009+0000 7f78a1600640 1 --2- 172.21.15.47:0/4013179692 >> v2:172.21.15.47:3300/0 conn(0x7f789c0d0360 0x7f789c0d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f1553200640 1 -- 172.21.15.47:0/1472970472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 msgr2=0x7f1548038630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f1553200640 1 --2- 172.21.15.47:0/1472970472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548038630 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f15400097d0 tx=0x7f154002b940 comp rx=0 tx=0).stop 2024-04-20T10:27:58.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f1553200640 1 -- 172.21.15.47:0/1472970472 shutdown_connections 2024-04-20T10:27:58.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f4a76400640 1 Processor -- start 2024-04-20T10:27:58.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f1553200640 1 --2- 172.21.15.47:0/1472970472 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548038630 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.027 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:27:58.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f4a76400640 1 -- start start 2024-04-20T10:27:58.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.007+0000 7f6c1ce00640 1 -- start start 2024-04-20T10:27:58.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f6c1ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c180d0a70 0x7f6c180ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f6c1ce00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c1806df70 con 0x7f6c18162c00 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f6c1ce00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6c1806e310 con 0x7f6c1806f090 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f6c1ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c18162fe0 con 0x7f6c180d0a70 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f6c15400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c180d0a70 0x7f6c180ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f6c15400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c180d0a70 0x7f6c180ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54122/0 (socket says 172.21.15.47:54122) 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f6c15400640 1 -- 172.21.15.47:0/2175791769 learned_addr learned my addr 172.21.15.47:0/2175791769 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f1553200640 1 -- 172.21.15.47:0/1472970472 >> 172.21.15.47:0/1472970472 conn(0x7f1548009fb0 msgr2=0x7f154800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f3456000640 1 Processor -- start 2024-04-20T10:27:58.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.010+0000 7f3456000640 1 -- start start 2024-04-20T10:27:58.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7ff415a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4100d36c0 0x7ff410169180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7ff415a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4100cfb40 con 0x7ff41015e750 2024-04-20T10:27:58.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7ff415a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4100cfe10 con 0x7ff41006d070 2024-04-20T10:27:58.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7ff415a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff4100d0a20 con 0x7ff4100d36c0 2024-04-20T10:27:58.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f1553200640 1 -- 172.21.15.47:0/1472970472 shutdown_connections 2024-04-20T10:27:58.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7ff40f400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7ff41006d070 0x7ff4100d2fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47200/0 (socket says 172.21.15.47:47200) 2024-04-20T10:27:58.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7ff40f400640 1 -- 172.21.15.47:0/2194143479 learned_addr learned my addr 172.21.15.47:0/2194143479 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe304003790 con 0x7fe310053cb0 2024-04-20T10:27:58.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f4a76400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a680d4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f4a76400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 0x7f4a680e2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f4a76400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a680e7a50 0x7f4a680e3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f4a76400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6804a270 con 0x7f4a680e7a50 2024-04-20T10:27:58.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f4a76400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6804a5b0 con 0x7f4a680548a0 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f4a76400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a68039cb0 con 0x7f4a6803c360 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f4a75000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a680d4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f4a75000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a680d4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54138/0 (socket says 172.21.15.47:54138) 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f4a75000640 1 -- 172.21.15.47:0/3292660584 learned_addr learned my addr 172.21.15.47:0/3292660584 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f4a6fe00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 0x7f4a680e2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f4a75a00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a680e7a50 0x7f4a680e3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f4a75000640 1 -- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 msgr2=0x7f4a680e2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f4a75000640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a680548a0 0x7f4a680e2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f4a75000640 1 -- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a680e7a50 msgr2=0x7f4a680e3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f4a75000640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a680e7a50 0x7f4a680e3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f4a75000640 1 -- 172.21.15.47:0/3292660584 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a68008720 con 0x7f4a6803c360 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f3456000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3450071c20 0x7f3450159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f3456000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500c7760 0x7f3450201ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f3456000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450145d80 0x7f34502009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7f3456000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f34500c7bf0 con 0x7f3450071c20 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f3456000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3450160d40 con 0x7f3450145d80 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f3456000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34502020e0 con 0x7f34500c7760 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f344ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500c7760 0x7f3450201ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f344ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500c7760 0x7f3450201ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54150/0 (socket says 172.21.15.47:54150) 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f344ea00640 1 -- 172.21.15.47:0/2573120383 learned_addr learned my addr 172.21.15.47:0/2573120383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f344fe00640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450145d80 0x7f34502009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f344ea00640 1 -- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450145d80 msgr2=0x7f34502009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f344ea00640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3450145d80 0x7f34502009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f344ea00640 1 -- 172.21.15.47:0/2573120383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3450071c20 msgr2=0x7f3450159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f344ea00640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3450071c20 0x7f3450159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f344ea00640 1 -- 172.21.15.47:0/2573120383 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f345015a1f0 con 0x7f34500c7760 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2964139316 0 0) 0x7f789c141c60 con 0x7f789c156c20 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7884003520 con 0x7f789c156c20 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 272493768 0 0) 0x7f789c0adab0 con 0x7f789c15ca40 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f78840048e0 con 0x7f789c15ca40 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 <== mon.1 v1:172.21.15.175:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3610206270 0 0) 0x7f78840048e0 con 0x7f789c15ca40 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 >> v2:172.21.15.47:3300/0 conn(0x7f789c0d0360 msgr2=0x7f789c0d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 --2- 172.21.15.47:0/4013179692 >> v2:172.21.15.47:3300/0 conn(0x7f789c0d0360 0x7f789c0d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 >> v1:172.21.15.47:6789/0 conn(0x7f789c156c20 legacy=0x7f789c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 --> v1:172.21.15.175:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c15cf90 con 0x7f789c15ca40 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 <== mon.1 v1:172.21.15.175:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f7888001670 con 0x7f789c15ca40 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 >> v1:172.21.15.175:6789/0 conn(0x7f789c15ca40 legacy=0x7f789c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7884004d90 0x7f78840051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78840079f0 0x7f7884009e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788400a4b0 0x7f788400c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f789c0d0d10 con 0x7f7884004d90 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f788400d250 con 0x7f788400a4b0 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f788400d450 con 0x7f78840079f0 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f78a0c00640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7884004d90 0x7f78840051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f78a1600640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788400a4b0 0x7f788400c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f789be00640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78840079f0 0x7f7884009e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7ff415000640 1 --2- 172.21.15.47:0/2194143479 >> v2:172.21.15.175:3300/0 conn(0x7ff4100d36c0 0x7ff410169180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7ff40ea00640 1 -- 172.21.15.47:0/2194143479 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1836301065 0 0) 0x7ff4100cfe10 con 0x7ff41006d070 2024-04-20T10:27:58.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7ff40ea00640 1 -- 172.21.15.47:0/2194143479 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff3f4003520 con 0x7ff41006d070 2024-04-20T10:27:58.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7ff415000640 1 -- 172.21.15.47:0/2194143479 >> v1:172.21.15.175:6789/0 conn(0x7ff41006d070 legacy=0x7ff4100d2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7ff415000640 1 -- 172.21.15.47:0/2194143479 >> v1:172.21.15.47:6789/0 conn(0x7ff41015e750 legacy=0x7ff4100cf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7ff415000640 1 -- 172.21.15.47:0/2194143479 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4100d00f0 con 0x7ff4100d36c0 2024-04-20T10:27:58.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f6c15400640 1 -- 172.21.15.47:0/2175791769 >> v1:172.21.15.175:6789/0 conn(0x7f6c1806f090 legacy=0x7f6c180ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.011+0000 7f6c15400640 1 -- 172.21.15.47:0/2175791769 >> v1:172.21.15.47:6789/0 conn(0x7f6c18162c00 legacy=0x7f6c180d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f6c15400640 1 -- 172.21.15.47:0/2175791769 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1806e5b0 con 0x7f6c180d0a70 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.014+0000 7f6c15400640 1 --2- 172.21.15.47:0/2175791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c180d0a70 0x7f6c180ad050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6c0801a8a0 tx=0x7f6c0803c8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc1dcc6a4051cae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7fd5cd000640 1 Processor -- start 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.013+0000 7fd5cd000640 1 -- start start 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7fd5cd000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c806dcb0 0x7fd5c8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c8072090 0x7fd5c80d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7fd5cd000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5c80ade40 con 0x7fd5c815b3b0 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7fd5cd000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd5c806f1b0 con 0x7fd5c806dcb0 2024-04-20T10:27:58.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7fd5cd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80ae0e0 con 0x7fd5c8072090 2024-04-20T10:27:58.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f78a0c00640 1 -- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78840079f0 msgr2=0x7f7884009e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f78a0c00640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78840079f0 0x7f7884009e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f78a0c00640 1 -- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788400a4b0 msgr2=0x7f788400c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f6c14a00640 1 -- 172.21.15.47:0/2175791769 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c08046020 con 0x7f6c180d0a70 2024-04-20T10:27:58.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.012+0000 7f1553200640 1 -- 172.21.15.47:0/1472970472 wait complete. 2024-04-20T10:27:58.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f6c14a00640 1 -- 172.21.15.47:0/2175791769 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c08042640 con 0x7f6c180d0a70 2024-04-20T10:27:58.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f6c14a00640 1 -- 172.21.15.47:0/2175791769 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c0803e040 con 0x7f6c180d0a70 2024-04-20T10:27:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f78a0c00640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788400a4b0 0x7f788400c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f78a0c00640 1 -- 172.21.15.47:0/4013179692 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c15cf90 con 0x7f7884004d90 2024-04-20T10:27:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f789be00640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78840079f0 0x7f7884009e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f78a1600640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f788400a4b0 0x7f788400c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f78a1600640 1 -- 172.21.15.47:0/4013179692 reap_dead start 2024-04-20T10:27:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.015+0000 7f1553200640 1 Processor -- start 2024-04-20T10:27:58.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fd5c5400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd5c806dcb0 0x7fd5c8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fd5c5e00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fd5c815b3b0 0x7fd5c806eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47214/0 (socket says 172.21.15.47:47214) 2024-04-20T10:27:58.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fd5c5e00640 1 -- 172.21.15.47:0/2003300828 learned_addr learned my addr 172.21.15.47:0/2003300828 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fd5c6800640 1 --2- 172.21.15.47:0/2003300828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c8072090 0x7fd5c80d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f1553200640 1 -- start start 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fe30e000640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2f806caf0 0x7fe2f806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe30408c0e0 con 0x7fe310053cb0 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fe30e000640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7fe2f8076170 0x7fe2f80785b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 --> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe2f8078c80 con 0x7fe2f8076170 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe30408c490 con 0x7fe310053cb0 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7fe316800640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7fe2f8076170 0x7fe2f80785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1553200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548033030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1553200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548050dd0 0x7f15480496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f4a75000640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a680d4f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a70069030 tx=0x7f4a7006dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7006de20 con 0x7f4a6803c360 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a70090020 con 0x7f4a6803c360 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7006b900 con 0x7f4a6803c360 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a68008a00 con 0x7f4a6803c360 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a68008fc0 con 0x7f4a6803c360 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.016+0000 7f344ea00640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500c7760 0x7f3450201ba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f343c00d800 tx=0x7f343c00dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c00b6c0 con 0x7f34500c7760 2024-04-20T10:27:58.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f343c019070 con 0x7f34500c7760 2024-04-20T10:27:58.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343c00ba80 con 0x7f34500c7760 2024-04-20T10:27:58.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f345015a4d0 con 0x7f34500c7760 2024-04-20T10:27:58.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7ff415000640 1 --2- 172.21.15.47:0/2194143479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 0x7ff410169180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0097a0 tx=0x7ff3fc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=316a6ddcbefc8edf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2175791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c180d0a70 msgr2=0x7f6c180ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f6c1ce00640 1 --2- 172.21.15.47:0/2175791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c180d0a70 0x7f6c180ad050 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6c0801a8a0 tx=0x7f6c0803c8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2175791769 shutdown_connections 2024-04-20T10:27:58.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f6c1ce00640 1 --2- 172.21.15.47:0/2175791769 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c180d0a70 0x7f6c180ad050 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2175791769 >> 172.21.15.47:0/2175791769 conn(0x7f6c180b3350 msgr2=0x7f6c180b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2175791769 shutdown_connections 2024-04-20T10:27:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f345015aa90 con 0x7f34500c7760 2024-04-20T10:27:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7fd5c6800640 1 -- 172.21.15.47:0/2003300828 >> v2:172.21.15.47:3300/0 conn(0x7fd5c806dcb0 msgr2=0x7fd5c8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7fd5c6800640 1 --2- 172.21.15.47:0/2003300828 >> v2:172.21.15.47:3300/0 conn(0x7fd5c806dcb0 0x7fd5c8071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7fd5c6800640 1 -- 172.21.15.47:0/2003300828 >> v1:172.21.15.175:6789/0 conn(0x7fd5c815b3b0 legacy=0x7fd5c806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7fd5c6800640 1 -- 172.21.15.47:0/2003300828 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c815b790 con 0x7fd5c8072090 2024-04-20T10:27:58.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.017+0000 7ff40ea00640 1 -- 172.21.15.47:0/2194143479 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc034070 con 0x7ff4100d36c0 2024-04-20T10:27:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7ff40ea00640 1 -- 172.21.15.47:0/2194143479 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff3fc030610 con 0x7ff4100d36c0 2024-04-20T10:27:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7ff40ea00640 1 -- 172.21.15.47:0/2194143479 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc0309c0 con 0x7ff4100d36c0 2024-04-20T10:27:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7fe315e00640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2f806caf0 0x7fe2f806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2175791769 wait complete. 2024-04-20T10:27:58.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7f78a0c00640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7884004d90 0x7f78840051b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7890001930 tx=0x7f7884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0690f282157e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1553200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15480e1090 0x7f1548049be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1553200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f15480066a0 con 0x7f15480e1090 2024-04-20T10:27:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1553200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15480503a0 con 0x7f1548050dd0 2024-04-20T10:27:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1553200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1548006820 con 0x7f154803a5d0 2024-04-20T10:27:58.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890037070 con 0x7f7884004d90 2024-04-20T10:27:58.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1552800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15480e1090 0x7f1548049be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1552800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15480e1090 0x7f1548049be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55918/0 (socket says 172.21.15.47:55918) 2024-04-20T10:27:58.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1552800640 1 -- 172.21.15.47:0/3747896559 learned_addr learned my addr 172.21.15.47:0/3747896559 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.018+0000 7f1551e00640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548033030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f345015ae90 con 0x7f34500c7760 2024-04-20T10:27:58.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4a680093a0 con 0x7f4a6803c360 2024-04-20T10:27:58.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7fe316800640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7fe2f8076170 0x7fe2f80785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7f1551400640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548050dd0 0x7f15480496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7890005ad0 con 0x7f7884004d90 2024-04-20T10:27:58.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7fe315e00640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2f806caf0 0x7fe2f806efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe31004f410 tx=0x7fe308008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.019+0000 7f789b400640 1 -- 172.21.15.47:0/4013179692 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890001550 con 0x7f7884004d90 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7ff415a00640 1 -- 172.21.15.47:0/2194143479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 msgr2=0x7ff410169180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7ff415a00640 1 --2- 172.21.15.47:0/2194143479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 0x7ff410169180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc0097a0 tx=0x7ff3fc02b960 comp rx=0 tx=0).stop 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1551e00640 1 -- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548050dd0 msgr2=0x7f15480496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1551e00640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548050dd0 0x7f15480496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1551e00640 1 -- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15480e1090 msgr2=0x7f1548049be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1551e00640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15480e1090 0x7f1548049be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1551e00640 1 -- 172.21.15.47:0/3747896559 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1548032e10 con 0x7f154803a5d0 2024-04-20T10:27:58.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1552800640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f15480e1090 0x7f1548049be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:58.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f6c1ce00640 1 Processor -- start 2024-04-20T10:27:58.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7ff415a00640 1 -- 172.21.15.47:0/2194143479 shutdown_connections 2024-04-20T10:27:58.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7ff415a00640 1 --2- 172.21.15.47:0/2194143479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 0x7ff410169180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7ff415a00640 1 -- 172.21.15.47:0/2194143479 >> 172.21.15.47:0/2194143479 conn(0x7ff4100d0e50 msgr2=0x7ff4100d1260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1551400640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1548050dd0 0x7f15480496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:58.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f6c1ce00640 1 -- start start 2024-04-20T10:27:58.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7ff415a00640 1 -- 172.21.15.47:0/2194143479 shutdown_connections 2024-04-20T10:27:58.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7f1551e00640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548033030 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c3c0 tx=0x7f153c00c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.020+0000 7ff415a00640 1 -- 172.21.15.47:0/2194143479 wait complete. 2024-04-20T10:27:58.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f78a3600640 1 -- 172.21.15.47:0/4013179692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7884004d90 msgr2=0x7f78840051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f78a3600640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7884004d90 0x7f78840051b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f7890001930 tx=0x7f7884005830 comp rx=0 tx=0).stop 2024-04-20T10:27:58.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f78a3600640 1 -- 172.21.15.47:0/4013179692 shutdown_connections 2024-04-20T10:27:58.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f78a3600640 1 --2- 172.21.15.47:0/4013179692 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7884004d90 0x7f78840051b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f78a3600640 1 -- 172.21.15.47:0/4013179692 >> 172.21.15.47:0/4013179692 conn(0x7f789c0b3350 msgr2=0x7f789c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== osd.0 v2:172.21.15.47:6800/2647753696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe2f8078c80 con 0x7fe2f8076170 2024-04-20T10:27:58.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c017070 con 0x7f154803a5d0 2024-04-20T10:27:58.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15480e2b90 con 0x7f154803a5d0 2024-04-20T10:27:58.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f154802ca40 con 0x7f154803a5d0 2024-04-20T10:27:58.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153c0052f0 con 0x7f154803a5d0 2024-04-20T10:27:58.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f6c1ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c1806f090 0x7f6c18164c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f6c1ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c180d0a70 0x7f6c180ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f6c1ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c18162c00 0x7f6c180aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f6c1ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1806dbb0 con 0x7f6c1806f090 2024-04-20T10:27:58.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f6c1ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1806ec00 con 0x7f6c180d0a70 2024-04-20T10:27:58.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f6c1ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1806dd30 con 0x7f6c18162c00 2024-04-20T10:27:58.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7fd5c6800640 1 --2- 172.21.15.47:0/2003300828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c8072090 0x7fd5c80d0360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009d90 tx=0x7fd5c002bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3a8cc642f65fc90 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.022+0000 7f6c15e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c1806f090 0x7f6c18164c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.022+0000 7f6c15e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c1806f090 0x7f6c18164c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55924/0 (socket says 172.21.15.47:55924) 2024-04-20T10:27:58.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.022+0000 7f6c15e00640 1 -- 172.21.15.47:0/2433819811 learned_addr learned my addr 172.21.15.47:0/2433819811 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.022+0000 7f6c15400640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c180d0a70 0x7f6c180ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.021+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f153c0108b0 con 0x7f154803a5d0 2024-04-20T10:27:58.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1528001030 con 0x7f154803a5d0 2024-04-20T10:27:58.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a7006bb60 con 0x7f4a6803c360 2024-04-20T10:27:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f78a3600640 1 -- 172.21.15.47:0/4013179692 shutdown_connections 2024-04-20T10:27:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f78a3600640 1 -- 172.21.15.47:0/4013179692 wait complete. 2024-04-20T10:27:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7ff415a00640 1 Processor -- start 2024-04-20T10:27:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f6c15e00640 1 -- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c18162c00 msgr2=0x7f6c180aeae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f6c15e00640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c18162c00 0x7f6c180aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f6c15e00640 1 -- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c180d0a70 msgr2=0x7f6c180ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f6c15e00640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c180d0a70 0x7f6c180ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f6c15e00640 1 -- 172.21.15.47:0/2433819811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c18165530 con 0x7f6c1806f090 2024-04-20T10:27:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7fd5c4a00640 1 -- 172.21.15.47:0/2003300828 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0039070 con 0x7fd5c8072090 2024-04-20T10:27:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7fd5c4a00640 1 -- 172.21.15.47:0/2003300828 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c002cc50 con 0x7fd5c8072090 2024-04-20T10:27:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7fd5c4a00640 1 -- 172.21.15.47:0/2003300828 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0034440 con 0x7fd5c8072090 2024-04-20T10:27:58.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f6c15e00640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c1806f090 0x7f6c18164c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000b3e0 tx=0x7f6c0000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7ff415a00640 1 -- start start 2024-04-20T10:27:58.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.024+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00015020 con 0x7f6c1806f090 2024-04-20T10:27:58.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.024+0000 7f4a6cc00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a5006c9c0 0x7f4a5006ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.024+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4a70107310 con 0x7f4a6803c360 2024-04-20T10:27:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.024+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c00011850 con 0x7f6c1806f090 2024-04-20T10:27:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.024+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c00011c40 con 0x7f6c1806f090 2024-04-20T10:27:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7ff415a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff41006d070 0x7ff4100cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7ff415a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 0x7ff410160780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7ff415a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff41015e750 0x7ff4100aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7f4a6cc00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f4a50076140 0x7f4a50078580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 --> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4a70097e70 con 0x7f4a50076140 2024-04-20T10:27:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f4a70107700 con 0x7f4a6803c360 2024-04-20T10:27:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c18150250 con 0x7f6c1806f090 2024-04-20T10:27:58.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c18150810 con 0x7f6c1806f090 2024-04-20T10:27:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7f4a6fe00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a5006c9c0 0x7f4a5006ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7ff415a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff4100b87c0 con 0x7ff41006d070 2024-04-20T10:27:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7ff415a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4100d06b0 con 0x7ff4100d36c0 2024-04-20T10:27:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7ff415000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff41015e750 0x7ff4100aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7ff415000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff41015e750 0x7ff4100aaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54190/0 (socket says 172.21.15.47:54190) 2024-04-20T10:27:58.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7ff415000640 1 -- 172.21.15.47:0/172373315 learned_addr learned my addr 172.21.15.47:0/172373315 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4100b8940 con 0x7ff41015e750 2024-04-20T10:27:58.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.023+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f343c00bc20 con 0x7f34500c7760 2024-04-20T10:27:58.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.025+0000 7f3447400640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f343006c940 0x7f343006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7f344f400640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f343006c940 0x7f343006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7f78a3600640 1 Processor -- start 2024-04-20T10:27:58.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7fd5cd000640 1 -- 172.21.15.47:0/2003300828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c8072090 msgr2=0x7fd5c80d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7fd5cd000640 1 --2- 172.21.15.47:0/2003300828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c8072090 0x7fd5c80d0360 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd5c0009d90 tx=0x7fd5c002bea0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7ff40f400640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 0x7ff410160780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7fd5cd000640 1 -- 172.21.15.47:0/2003300828 shutdown_connections 2024-04-20T10:27:58.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.026+0000 7f4a75a00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f4a50076140 0x7f4a50078580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7f4a6fe00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a5006c9c0 0x7f4a5006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c036000 tx=0x7f4a5c03d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7fd5cd000640 1 --2- 172.21.15.47:0/2003300828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c8072090 0x7fd5c80d0360 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7fd5cd000640 1 --2- 172.21.15.47:0/2003300828 >> v2:172.21.15.47:3300/0 conn(0x7fd5c806dcb0 0x7fd5c8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7fd5cd000640 1 -- 172.21.15.47:0/2003300828 >> 172.21.15.47:0/2003300828 conn(0x7fd5c80b3350 msgr2=0x7fd5c80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f343c08ce60 con 0x7f34500c7760 2024-04-20T10:27:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7ff40f400640 1 -- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff41015e750 msgr2=0x7ff4100aaf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7ff40f400640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff41015e750 0x7ff4100aaf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7ff40f400640 1 -- 172.21.15.47:0/172373315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff41006d070 msgr2=0x7ff4100cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7ff40f400640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff41006d070 0x7ff4100cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7fd5cd000640 1 -- 172.21.15.47:0/2003300828 shutdown_connections 2024-04-20T10:27:58.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7f3447400640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7f34300760b0 0x7f34300784f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 --> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f343c028e50 con 0x7f34300760b0 2024-04-20T10:27:58.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f343c08d210 con 0x7f34500c7760 2024-04-20T10:27:58.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7ff40f400640 1 -- 172.21.15.47:0/172373315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4100cbcf0 con 0x7ff4100d36c0 2024-04-20T10:27:58.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7f78a3600640 1 -- start start 2024-04-20T10:27:58.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7ff40f400640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 0x7ff410160780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c430 tx=0x7ff40000c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7f4a75a00640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f4a50076140 0x7f4a50078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6bf0001030 con 0x7f6c1806f090 2024-04-20T10:27:58.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7f344f400640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f343006c940 0x7f343006ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3440006a90 tx=0x7f3440006a20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c00018020 con 0x7f6c1806f090 2024-04-20T10:27:58.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff400017070 con 0x7ff4100d36c0 2024-04-20T10:27:58.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff400005ce0 con 0x7ff4100d36c0 2024-04-20T10:27:58.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4000072d0 con 0x7ff4100d36c0 2024-04-20T10:27:58.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4100cbfd0 con 0x7ff4100d36c0 2024-04-20T10:27:58.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.028+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff41014d620 con 0x7ff4100d36c0 2024-04-20T10:27:58.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f153c01a070 con 0x7f154803a5d0 2024-04-20T10:27:58.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.029+0000 7f6c0e000640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bf406c940 0x7f6bf406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.030+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6c0008bd10 con 0x7f6c1806f090 2024-04-20T10:27:58.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.030+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== osd.2 v2:172.21.15.47:6822/2832687130 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4a70097e70 con 0x7f4a50076140 2024-04-20T10:27:58.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.030+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff410075b30 con 0x7ff4100d36c0 2024-04-20T10:27:58.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.030+0000 7f6c0e000640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f6bf40760c0 0x7f6bf4078500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.030+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 --> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6c00020ec0 con 0x7f6bf40760c0 2024-04-20T10:27:58.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.030+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6c0008e460 con 0x7f6c1806f090 2024-04-20T10:27:58.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f6c15400640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bf406c940 0x7f6bf406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f1546000640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f153006c8f0 0x7f153006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f153c08ba10 con 0x7f154803a5d0 2024-04-20T10:27:58.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f78a3600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7884004d90 0x7f789c1f15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f1546000640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7f1530076070 0x7f15300784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 --> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f153c0054a0 con 0x7f1530076070 2024-04-20T10:27:58.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f344fe00640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7f34300760b0 0x7f34300784f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.031+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f153c08be00 con 0x7f154803a5d0 2024-04-20T10:27:58.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.027+0000 7fd5cd000640 1 -- 172.21.15.47:0/2003300828 wait complete. 2024-04-20T10:27:58.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.030+0000 7fd5cd000640 1 Processor -- start 2024-04-20T10:27:58.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f6c15400640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bf406c940 0x7f6bf406ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c08047000 tx=0x7f6c0804d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f78a3600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789c152910 0x7f789c099680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f78a3600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f789c153320 0x7f789c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f78a3600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f789c15d830 con 0x7f789c153320 2024-04-20T10:27:58.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f78a3600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f789c15d550 con 0x7f789c152910 2024-04-20T10:27:58.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f78a3600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f789c0d07d0 con 0x7f7884004d90 2024-04-20T10:27:58.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f1551400640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f153006c8f0 0x7f153006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.032+0000 7f344fe00640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7f34300760b0 0x7f34300784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.033+0000 7f1552800640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7f1530076070 0x7f15300784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.033+0000 7fd5cd000640 1 -- start start 2024-04-20T10:27:58.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.033+0000 7f78a1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f789c153320 0x7f789c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.033+0000 7f78a1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f789c153320 0x7f789c1551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55928/0 (socket says 172.21.15.47:55928) 2024-04-20T10:27:58.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.033+0000 7f78a1600640 1 -- 172.21.15.47:0/1860030897 learned_addr learned my addr 172.21.15.47:0/1860030897 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== osd.3 v2:172.21.15.47:6812/4183939241 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f343c028e50 con 0x7f34300760b0 2024-04-20T10:27:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f78a1600640 1 -- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7884004d90 msgr2=0x7f789c1f15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f78a1600640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7884004d90 0x7f789c1f15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f78a1600640 1 -- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789c152910 msgr2=0x7f789c099680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f78a1600640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f789c152910 0x7f789c099680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f78a1600640 1 -- 172.21.15.47:0/1860030897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c1f1b10 con 0x7f789c153320 2024-04-20T10:27:58.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f1552800640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7f1530076070 0x7f15300784b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff40001a070 con 0x7ff4100d36c0 2024-04-20T10:27:58.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f6c16800640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f6bf40760c0 0x7f6bf4078500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c806dcb0 0x7fd5c8200bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7f78a1600640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f789c153320 0x7f789c1551a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f789c06d880 tx=0x7f789400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894009c10 con 0x7f789c153320 2024-04-20T10:27:58.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f789400fd40 con 0x7f789c153320 2024-04-20T10:27:58.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7ff407e00640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff3f006c8f0 0x7ff3f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff40008b1d0 con 0x7ff4100d36c0 2024-04-20T10:27:58.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.034+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815b3b0 0x7fd5c81f2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5cd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c81f2b50 0x7fd5c815ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5cd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80ae940 con 0x7fd5c815b3b0 2024-04-20T10:27:58.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5cd000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80ae660 con 0x7fd5c81f2b50 2024-04-20T10:27:58.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5cd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80aec20 con 0x7fd5c806dcb0 2024-04-20T10:27:58.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894009c10 con 0x7f789c153320 2024-04-20T10:27:58.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c151870 con 0x7f789c153320 2024-04-20T10:27:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== osd.1 v2:172.21.15.47:6803/2415890922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f153c0054a0 con 0x7f1530076070 2024-04-20T10:27:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7ff407e00640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7ff3f0076070 0x7ff3f00784b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 --> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff400013e50 con 0x7ff3f0076070 2024-04-20T10:27:58.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff400092050 con 0x7ff4100d36c0 2024-04-20T10:27:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5c5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c806dcb0 0x7fd5c8200bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5c5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815b3b0 0x7fd5c81f2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5c5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c806dcb0 0x7fd5c8200bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54200/0 (socket says 172.21.15.47:54200) 2024-04-20T10:27:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.035+0000 7fd5c5e00640 1 -- 172.21.15.47:0/1018526656 learned_addr learned my addr 172.21.15.47:0/1018526656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7fd5c6800640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c81f2b50 0x7fd5c815ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c151e30 con 0x7f789c153320 2024-04-20T10:27:58.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7f6c16800640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f6bf40760c0 0x7f6bf4078500 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7fd5c5400640 1 -- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c806dcb0 msgr2=0x7fd5c8200bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7fd5c5400640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5c806dcb0 0x7fd5c8200bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7fd5c5400640 1 -- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c81f2b50 msgr2=0x7fd5c815ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7fd5c5400640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c81f2b50 0x7fd5c815ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.036+0000 7fd5c5400640 1 -- 172.21.15.47:0/1018526656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80af420 con 0x7fd5c815b3b0 2024-04-20T10:27:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.037+0000 7ff415000640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7ff3f0076070 0x7ff3f00784b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.037+0000 7fd5c6800640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5c81f2b50 0x7fd5c815ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.037+0000 7ff40fe00640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff3f006c8f0 0x7ff3f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.037+0000 7fd5c5400640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815b3b0 0x7fd5c81f2610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400e850 tx=0x7fd5b400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.038+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b4017070 con 0x7fd5c815b3b0 2024-04-20T10:27:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.038+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5b400c940 con 0x7fd5c815b3b0 2024-04-20T10:27:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.038+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5b400cd10 con 0x7fd5c815b3b0 2024-04-20T10:27:58.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.038+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f789c0af7c0 con 0x7f789c153320 2024-04-20T10:27:58.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.038+0000 7ff415000640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7ff3f0076070 0x7ff3f00784b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.037+0000 7f1551400640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f153006c8f0 0x7f153006edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f154c06a770 tx=0x7f154c06f040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.039+0000 7ff40fe00640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff3f006c8f0 0x7ff3f006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4080012e0 tx=0x7ff408007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.039+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== osd.6 v2:172.21.15.175:6822/2500655665 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6c00020ec0 con 0x7f6bf40760c0 2024-04-20T10:27:58.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.040+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== osd.5 v2:172.21.15.175:6800/1477827144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff400013e50 con 0x7ff3f0076070 2024-04-20T10:27:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.038+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c80af700 con 0x7fd5c815b3b0 2024-04-20T10:27:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.038+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c80afcc0 con 0x7fd5c815b3b0 2024-04-20T10:27:58.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.039+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5c8068980 con 0x7fd5c815b3b0 2024-04-20T10:27:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.040+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f789401f020 con 0x7f789c153320 2024-04-20T10:27:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.042+0000 7f7898c00640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f787c06caf0 0x7f787c06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.042+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f789408b5b0 con 0x7f789c153320 2024-04-20T10:27:58.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.042+0000 7f7898c00640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f787c0761e0 0x7f787c078620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.042+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 --> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7894006b90 con 0x7f787c0761e0 2024-04-20T10:27:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.042+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7894090060 con 0x7f789c153320 2024-04-20T10:27:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.043+0000 7f78a0c00640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f787c06caf0 0x7f787c06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.043+0000 7f789be00640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f787c0761e0 0x7f787c078620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.045+0000 7f78a0c00640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f787c06caf0 0x7f787c06efb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7890034000 tx=0x7f7890007000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.043+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5b401a070 con 0x7fd5c815b3b0 2024-04-20T10:27:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.044+0000 7fd5be000640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5a806c9c0 0x7fd5a806ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.044+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fd5b408c140 con 0x7fd5c815b3b0 2024-04-20T10:27:58.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.045+0000 7fd5be000640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fd5a8076130 0x7fd5a8078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.045+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 --> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5b4026e50 con 0x7fd5a8076130 2024-04-20T10:27:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.045+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd5b408c530 con 0x7fd5c815b3b0 2024-04-20T10:27:58.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.045+0000 7f789be00640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f787c0761e0 0x7f787c078620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.046+0000 7fd5c5e00640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5a806c9c0 0x7fd5a806ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.046+0000 7fd5c6800640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fd5a8076130 0x7fd5a8078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.048+0000 7fd5c6800640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fd5a8076130 0x7fd5a8078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.048+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== osd.4 v2:172.21.15.175:6816/1401363349 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7894006b90 con 0x7f787c0761e0 2024-04-20T10:27:58.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.049+0000 7fd5c5e00640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5a806c9c0 0x7fd5a806ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8005880 tx=0x7fd5b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.049+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== osd.7 v2:172.21.15.175:6808/1688864180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd5b4026e50 con 0x7fd5a8076130 2024-04-20T10:27:58.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.054+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 --> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe2f0002dd0 con 0x7fe2f8076170 2024-04-20T10:27:58.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.062+0000 7fe30e000640 1 -- 172.21.15.47:0/2301066733 <== osd.0 v2:172.21.15.47:6800/2647753696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe2f0002dd0 con 0x7fe2f8076170 2024-04-20T10:27:58.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.062+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7fe2f8076170 msgr2=0x7fe2f80785b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.062+0000 7fe317200640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7fe2f8076170 0x7fe2f80785b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2f806caf0 msgr2=0x7fe2f806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2f806caf0 0x7fe2f806efb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe31004f410 tx=0x7fe308008040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 msgr2=0x7fe31003da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 --2- 172.21.15.47:0/2301066733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe310053cb0 0x7fe31003da10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe30400c460 tx=0x7fe30400c920 comp rx=0 tx=0).stop 2024-04-20T10:27:58.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe316800640 1 -- 172.21.15.47:0/2301066733 reap_dead start 2024-04-20T10:27:58.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 shutdown_connections 2024-04-20T10:27:58.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 >> 172.21.15.47:0/2301066733 conn(0x7fe310058ee0 msgr2=0x7fe310039b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 shutdown_connections 2024-04-20T10:27:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.063+0000 7fe317200640 1 -- 172.21.15.47:0/2301066733 wait complete. 2024-04-20T10:27:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.067+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 --> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4a680057e0 con 0x7f4a50076140 2024-04-20T10:27:58.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.068+0000 7f4a6cc00640 1 -- 172.21.15.47:0/3292660584 <== osd.2 v2:172.21.15.47:6822/2832687130 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4a680057e0 con 0x7f4a50076140 2024-04-20T10:27:58.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.070+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 --> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1528002dd0 con 0x7f1530076070 2024-04-20T10:27:58.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.071+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f4a50076140 msgr2=0x7f4a50078580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.071+0000 7f4a76400640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f4a50076140 0x7f4a50078580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.071+0000 7f1546000640 1 -- 172.21.15.47:0/3747896559 <== osd.1 v2:172.21.15.47:6803/2415890922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1528002dd0 con 0x7f1530076070 2024-04-20T10:27:58.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a5006c9c0 msgr2=0x7f4a5006ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a76400640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a5006c9c0 0x7f4a5006ee80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c036000 tx=0x7f4a5c03d040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 msgr2=0x7f4a680d4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7f1530076070 msgr2=0x7f15300784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f1553200640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7f1530076070 0x7f15300784b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a76400640 1 --2- 172.21.15.47:0/3292660584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6803c360 0x7f4a680d4f20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4a70069030 tx=0x7f4a7006dcf0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 --> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff4100d1b00 con 0x7ff3f0076070 2024-04-20T10:27:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a75a00640 1 -- 172.21.15.47:0/3292660584 reap_dead start 2024-04-20T10:27:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 shutdown_connections 2024-04-20T10:27:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 >> 172.21.15.47:0/3292660584 conn(0x7f4a68056cd0 msgr2=0x7f4a68055ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 shutdown_connections 2024-04-20T10:27:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.071+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 --> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3450069090 con 0x7f34300760b0 2024-04-20T10:27:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f153006c8f0 msgr2=0x7f153006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f1553200640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f153006c8f0 0x7f153006edb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f154c06a770 tx=0x7f154c06f040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f4a76400640 1 -- 172.21.15.47:0/3292660584 wait complete. 2024-04-20T10:27:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.071+0000 7f3447400640 1 -- 172.21.15.47:0/2573120383 <== osd.3 v2:172.21.15.47:6812/4183939241 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3450069090 con 0x7f34300760b0 2024-04-20T10:27:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7f34300760b0 msgr2=0x7f34300784f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f3456000640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7f34300760b0 0x7f34300784f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 msgr2=0x7f1548033030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f1553200640 1 --2- 172.21.15.47:0/3747896559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f154803a5d0 0x7f1548033030 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f153c00c3c0 tx=0x7f153c00c7c0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.072+0000 7f1552800640 1 -- 172.21.15.47:0/3747896559 reap_dead start 2024-04-20T10:27:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f343006c940 msgr2=0x7f343006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f3456000640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f343006c940 0x7f343006ee00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3440006a90 tx=0x7f3440006a20 comp rx=0 tx=0).stop 2024-04-20T10:27:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500c7760 msgr2=0x7f3450201ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f3456000640 1 --2- 172.21.15.47:0/2573120383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34500c7760 0x7f3450201ba0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f343c00d800 tx=0x7f343c00dcc0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f344fe00640 1 -- 172.21.15.47:0/2573120383 reap_dead start 2024-04-20T10:27:58.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:27:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 shutdown_connections 2024-04-20T10:27:58.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 >> 172.21.15.47:0/3747896559 conn(0x7f1548009fb0 msgr2=0x7f1548005f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 shutdown_connections 2024-04-20T10:27:58.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7f1553200640 1 -- 172.21.15.47:0/3747896559 wait complete. 2024-04-20T10:27:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 shutdown_connections 2024-04-20T10:27:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.073+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 >> 172.21.15.47:0/2573120383 conn(0x7f34500d2340 msgr2=0x7f3450143370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 shutdown_connections 2024-04-20T10:27:58.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7f3456000640 1 -- 172.21.15.47:0/2573120383 wait complete. 2024-04-20T10:27:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7ff407e00640 1 -- 172.21.15.47:0/172373315 <== osd.5 v2:172.21.15.175:6800/1477827144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff4100d1b00 con 0x7ff3f0076070 2024-04-20T10:27:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7ff3f0076070 msgr2=0x7ff3f00784b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7ff3f0076070 0x7ff3f00784b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff3f006c8f0 msgr2=0x7ff3f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff3f006c8f0 0x7ff3f006edb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4080012e0 tx=0x7ff408007d30 comp rx=0 tx=0).stop 2024-04-20T10:27:58.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 msgr2=0x7ff410160780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 --2- 172.21.15.47:0/172373315 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4100d36c0 0x7ff410160780 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff40000c430 tx=0x7ff40000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415000640 1 -- 172.21.15.47:0/172373315 reap_dead start 2024-04-20T10:27:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 shutdown_connections 2024-04-20T10:27:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 >> 172.21.15.47:0/172373315 conn(0x7ff4100d0e50 msgr2=0x7ff4100b3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 shutdown_connections 2024-04-20T10:27:58.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7ff415a00640 1 -- 172.21.15.47:0/172373315 wait complete. 2024-04-20T10:27:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.074+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 --> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6bf0002dd0 con 0x7f6bf40760c0 2024-04-20T10:27:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.075+0000 7f6c0e000640 1 -- 172.21.15.47:0/2433819811 <== osd.6 v2:172.21.15.175:6822/2500655665 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6bf0002dd0 con 0x7f6bf40760c0 2024-04-20T10:27:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f6bf40760c0 msgr2=0x7f6bf4078500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7f6c1ce00640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f6bf40760c0 0x7f6bf4078500 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bf406c940 msgr2=0x7f6bf406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7f6c1ce00640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bf406c940 0x7f6bf406ee00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6c08047000 tx=0x7f6c0804d040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c1806f090 msgr2=0x7f6c18164c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7f6c1ce00640 1 --2- 172.21.15.47:0/2433819811 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6c1806f090 0x7f6c18164c70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c0000b3e0 tx=0x7f6c0000b8a0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7f6c16800640 1 -- 172.21.15.47:0/2433819811 reap_dead start 2024-04-20T10:27:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 shutdown_connections 2024-04-20T10:27:58.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 >> 172.21.15.47:0/2433819811 conn(0x7f6c180b3350 msgr2=0x7f6c18157790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 shutdown_connections 2024-04-20T10:27:58.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7f6c1ce00640 1 -- 172.21.15.47:0/2433819811 wait complete. 2024-04-20T10:27:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.076+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 --> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5c81f20c0 con 0x7fd5a8076130 2024-04-20T10:27:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.077+0000 7fd5be000640 1 -- 172.21.15.47:0/1018526656 <== osd.7 v2:172.21.15.175:6808/1688864180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5c81f20c0 con 0x7fd5a8076130 2024-04-20T10:27:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fd5a8076130 msgr2=0x7fd5a8078570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7fd5cd000640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fd5a8076130 0x7fd5a8078570 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5a806c9c0 msgr2=0x7fd5a806ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7fd5cd000640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5a806c9c0 0x7fd5a806ee80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8005880 tx=0x7fd5b8009040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815b3b0 msgr2=0x7fd5c81f2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.078+0000 7fd5cd000640 1 --2- 172.21.15.47:0/1018526656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5c815b3b0 0x7fd5c81f2610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fd5b400e850 tx=0x7fd5b400ed10 comp rx=0 tx=0).stop 2024-04-20T10:27:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.079+0000 7fd5c6800640 1 -- 172.21.15.47:0/1018526656 reap_dead start 2024-04-20T10:27:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.079+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 shutdown_connections 2024-04-20T10:27:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.079+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 >> 172.21.15.47:0/1018526656 conn(0x7fd5c80b3350 msgr2=0x7fd5c80b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.079+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 shutdown_connections 2024-04-20T10:27:58.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.079+0000 7fd5cd000640 1 -- 172.21.15.47:0/1018526656 wait complete. 2024-04-20T10:27:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.084+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 --> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f789c068980 con 0x7f787c0761e0 2024-04-20T10:27:58.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.085+0000 7f7898c00640 1 -- 172.21.15.47:0/1860030897 <== osd.4 v2:172.21.15.175:6816/1401363349 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f789c068980 con 0x7f787c0761e0 2024-04-20T10:27:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.091+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f787c0761e0 msgr2=0x7f787c078620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.091+0000 7f78a3600640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f787c0761e0 0x7f787c078620 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.137 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:27:58.138 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T10:27:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.094+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f787c06caf0 msgr2=0x7f787c06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.094+0000 7f78a3600640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f787c06caf0 0x7f787c06efb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7890034000 tx=0x7f7890007000 comp rx=0 tx=0).stop 2024-04-20T10:27:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.094+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f789c153320 msgr2=0x7f789c1551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.094+0000 7f78a3600640 1 --2- 172.21.15.47:0/1860030897 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f789c153320 0x7f789c1551a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f789c06d880 tx=0x7f789400f580 comp rx=0 tx=0).stop 2024-04-20T10:27:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.094+0000 7f78a1600640 1 -- 172.21.15.47:0/1860030897 reap_dead start 2024-04-20T10:27:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.095+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 shutdown_connections 2024-04-20T10:27:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.095+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 >> 172.21.15.47:0/1860030897 conn(0x7f789c0b3350 msgr2=0x7f789c0b1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.095+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 shutdown_connections 2024-04-20T10:27:58.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.098+0000 7f78a3600640 1 -- 172.21.15.47:0/1860030897 wait complete. 2024-04-20T10:27:58.143 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:27:58.143 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T10:27:58.144 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:27:58.145 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T10:27:58.145 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:27:58.146 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T10:27:58.146 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:27:58.147 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T10:27:58.147 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:27:58.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T10:27:58.148 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:27:58.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T10:27:58.149 INFO:teuthology.orchestra.run.smithi047.stdout:60129542154 2024-04-20T10:27:58.149 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T10:27:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.455+0000 7ff73ee00640 1 Processor -- start 2024-04-20T10:27:58.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.455+0000 7ff73ee00640 1 -- start start 2024-04-20T10:27:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.457+0000 7f6bc0a00640 1 Processor -- start 2024-04-20T10:27:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.457+0000 7ff73ee00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7380b3bf0 0x7ff7380ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.457+0000 7ff73ee00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff73806e510 0x7ff73815a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.457+0000 7ff73ee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7380adaf0 con 0x7ff73806bdd0 2024-04-20T10:27:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.457+0000 7ff73ee00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff7380addd0 con 0x7ff7380b3bf0 2024-04-20T10:27:58.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.457+0000 7ff73ee00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff7380ae0b0 con 0x7ff73806e510 2024-04-20T10:27:58.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.458+0000 7ff737e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7380b3bf0 0x7ff7380ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.458+0000 7ff737e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff7380b3bf0 0x7ff7380ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55944/0 (socket says 172.21.15.47:55944) 2024-04-20T10:27:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.459+0000 7ff737e00640 1 -- 172.21.15.47:0/3594646040 learned_addr learned my addr 172.21.15.47:0/3594646040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.459+0000 7ff737400640 1 --1- 172.21.15.47:0/3594646040 >> v1:172.21.15.47:6789/0 conn(0x7ff73806bdd0 0x7ff73806de00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33664/0 (socket says 172.21.15.47:33664) 2024-04-20T10:27:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.459+0000 7ff73ce00640 1 --2- 172.21.15.47:0/3594646040 >> v2:172.21.15.175:3300/0 conn(0x7ff73806e510 0x7ff73815a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.457+0000 7f6bc0a00640 1 -- start start 2024-04-20T10:27:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.459+0000 7f6bc0a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6bb40589c0 0x7f6bb4053530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.459+0000 7f6bc0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb4053a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bb9e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6bb40589c0 0x7f6bb4053530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bc0a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6bb4057d90 con 0x7f6bb40e7710 2024-04-20T10:27:58.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7ff736a00640 1 -- 172.21.15.47:0/3594646040 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1390798105 0 0) 0x7ff7380adaf0 con 0x7ff73806bdd0 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bba800640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f6bb40e7710 0x7f6bb404ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47216/0 (socket says 172.21.15.47:47216) 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bbb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb4053a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bbb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb4053a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54204/0 (socket says 172.21.15.47:54204) 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bbb200640 1 -- 172.21.15.47:0/1578778092 learned_addr learned my addr 172.21.15.47:0/1578778092 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bc0a00640 1 -- 172.21.15.47:0/1578778092 --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6bb4006260 con 0x7f6bb40589c0 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7ff736a00640 1 -- 172.21.15.47:0/3594646040 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff71c003520 con 0x7ff73806bdd0 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7ff737e00640 1 -- 172.21.15.47:0/3594646040 >> v2:172.21.15.175:3300/0 conn(0x7ff73806e510 msgr2=0x7ff73815a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7ff737e00640 1 --2- 172.21.15.47:0/3594646040 >> v2:172.21.15.175:3300/0 conn(0x7ff73806e510 0x7ff73815a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7ff737e00640 1 -- 172.21.15.47:0/3594646040 >> v1:172.21.15.47:6789/0 conn(0x7ff73806bdd0 legacy=0x7ff73806de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.461+0000 7f6bc0a00640 1 -- 172.21.15.47:0/1578778092 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb4006540 con 0x7f6bb4056ab0 2024-04-20T10:27:58.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.461+0000 7ff737e00640 1 -- 172.21.15.47:0/3594646040 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7380ae390 con 0x7ff7380b3bf0 2024-04-20T10:27:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.460+0000 7f6bb9e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6bb40589c0 0x7f6bb4053530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40856/0 (socket says 172.21.15.47:40856) 2024-04-20T10:27:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.462+0000 7f6bb9400640 1 -- 172.21.15.47:0/1578778092 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3030870546 0 0) 0x7f6bb4057d90 con 0x7f6bb40e7710 2024-04-20T10:27:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.462+0000 7f6bb9400640 1 -- 172.21.15.47:0/1578778092 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6ba0003520 con 0x7f6bb40e7710 2024-04-20T10:27:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.462+0000 7f6bb9e00640 1 -- 172.21.15.47:0/1578778092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 msgr2=0x7f6bb4053a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.462+0000 7f6bb9e00640 1 --2- 172.21.15.47:0/1578778092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb4053a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.462+0000 7f6bb9e00640 1 -- 172.21.15.47:0/1578778092 >> v1:172.21.15.175:6789/0 conn(0x7f6bb40e7710 legacy=0x7f6bb404ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.462+0000 7f6bb9e00640 1 -- 172.21.15.47:0/1578778092 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb4006820 con 0x7f6bb40589c0 2024-04-20T10:27:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.462+0000 7f9a0b800640 1 Processor -- start 2024-04-20T10:27:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.463+0000 7f9a0b800640 1 -- start start 2024-04-20T10:27:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.463+0000 7ff737e00640 1 --2- 172.21.15.47:0/3594646040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff7380ad210 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff7380cc830 tx=0x7ff73002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7add6d98d5cb57 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.463+0000 7ff736a00640 1 -- 172.21.15.47:0/3594646040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff73002ca80 con 0x7ff7380b3bf0 2024-04-20T10:27:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.463+0000 7ff736a00640 1 -- 172.21.15.47:0/3594646040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff73002cc20 con 0x7ff7380b3bf0 2024-04-20T10:27:58.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.463+0000 7ff736a00640 1 -- 172.21.15.47:0/3594646040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7300343f0 con 0x7ff7380b3bf0 2024-04-20T10:27:58.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.464+0000 7f6bb9e00640 1 --2- 172.21.15.47:0/1578778092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40589c0 0x7f6bb4053530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc06a770 tx=0x7f6bbc0a7100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e35a1ad0d3ea06c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.464+0000 7f6bb9400640 1 -- 172.21.15.47:0/1578778092 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0b4070 con 0x7f6bb40589c0 2024-04-20T10:27:58.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.464+0000 7ff73ee00640 1 -- 172.21.15.47:0/3594646040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 msgr2=0x7ff7380ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.464+0000 7ff73ee00640 1 --2- 172.21.15.47:0/3594646040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff7380ad210 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff7380cc830 tx=0x7ff73002bee0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f6bb9400640 1 -- 172.21.15.47:0/1578778092 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc0af440 con 0x7f6bb40589c0 2024-04-20T10:27:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f9a0b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a040d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f9a0b800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a040d4650 0x7f9a040b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f9a0b800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a040b17d0 0x7f9a040d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f9a0b800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9a040d40c0 con 0x7f9a040b17d0 2024-04-20T10:27:58.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f9a0b800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9a040d3820 con 0x7f9a040d4650 2024-04-20T10:27:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f9a0b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a040d3b00 con 0x7f9a040d4270 2024-04-20T10:27:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f6bb9400640 1 -- 172.21.15.47:0/1578778092 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0af8b0 con 0x7f6bb40589c0 2024-04-20T10:27:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7ff73ee00640 1 -- 172.21.15.47:0/3594646040 shutdown_connections 2024-04-20T10:27:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7ff73ee00640 1 --2- 172.21.15.47:0/3594646040 >> v2:172.21.15.175:3300/0 conn(0x7ff73806e510 0x7ff73815a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7ff73ee00640 1 --2- 172.21.15.47:0/3594646040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff7380ad210 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7ff73ee00640 1 -- 172.21.15.47:0/3594646040 >> 172.21.15.47:0/3594646040 conn(0x7ff738072090 msgr2=0x7ff7380b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.465+0000 7f9a03e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a040d4650 0x7f9a040b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f9a03e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9a040d4650 0x7f9a040b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40862/0 (socket says 172.21.15.47:40862) 2024-04-20T10:27:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f9a09800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9a040b17d0 0x7f9a040d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f6bc0a00640 1 -- 172.21.15.47:0/1578778092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40589c0 msgr2=0x7f6bb4053530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/1578778092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40589c0 0x7f6bb4053530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc06a770 tx=0x7f6bbc0a7100 comp rx=0 tx=0).stop 2024-04-20T10:27:58.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f9a08e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a040d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f6bc0a00640 1 -- 172.21.15.47:0/1578778092 shutdown_connections 2024-04-20T10:27:58.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/1578778092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb4053a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/1578778092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40589c0 0x7f6bb4053530 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f6bc0a00640 1 -- 172.21.15.47:0/1578778092 >> 172.21.15.47:0/1578778092 conn(0x7f6bb4058ee0 msgr2=0x7f6bb40592c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7ff73ee00640 1 -- 172.21.15.47:0/3594646040 shutdown_connections 2024-04-20T10:27:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f9a08e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a040d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54206/0 (socket says 172.21.15.47:54206) 2024-04-20T10:27:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f9a08e00640 1 -- 172.21.15.47:0/341042164 learned_addr learned my addr 172.21.15.47:0/341042164 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f9a08e00640 1 -- 172.21.15.47:0/341042164 >> v2:172.21.15.175:3300/0 conn(0x7f9a040d4650 msgr2=0x7f9a040b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7f9a08e00640 1 --2- 172.21.15.47:0/341042164 >> v2:172.21.15.175:3300/0 conn(0x7f9a040d4650 0x7f9a040b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.467+0000 7f9a08e00640 1 -- 172.21.15.47:0/341042164 >> v2:172.21.15.47:3300/0 conn(0x7f9a040b17d0 msgr2=0x7f9a040d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.467+0000 7f9a08e00640 1 --2- 172.21.15.47:0/341042164 >> v2:172.21.15.47:3300/0 conn(0x7f9a040b17d0 0x7f9a040d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.467+0000 7f6bc0a00640 1 -- 172.21.15.47:0/1578778092 shutdown_connections 2024-04-20T10:27:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.467+0000 7f9a08e00640 1 -- 172.21.15.47:0/341042164 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a040d3de0 con 0x7f9a040d4270 2024-04-20T10:27:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.467+0000 7f9a09800640 1 --2- 172.21.15.47:0/341042164 >> v2:172.21.15.47:3300/0 conn(0x7f9a040b17d0 0x7f9a040d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:58.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.466+0000 7ff73ee00640 1 -- 172.21.15.47:0/3594646040 wait complete. 2024-04-20T10:27:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.467+0000 7f6bc0a00640 1 -- 172.21.15.47:0/1578778092 wait complete. 2024-04-20T10:27:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.468+0000 7fa11ae00640 1 Processor -- start 2024-04-20T10:27:58.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.468+0000 7ff73ee00640 1 Processor -- start 2024-04-20T10:27:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.468+0000 7fa11ae00640 1 -- start start 2024-04-20T10:27:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.469+0000 7f9a08e00640 1 --2- 172.21.15.47:0/341042164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a040d1a00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9a040683c0 tx=0x7f99f802b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a862a571a72fa138 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.469+0000 7ff73ee00640 1 -- start start 2024-04-20T10:27:58.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.469+0000 7f9a03400640 1 -- 172.21.15.47:0/341042164 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8035020 con 0x7f9a040d4270 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7f9a03400640 1 -- 172.21.15.47:0/341042164 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99f802d040 con 0x7f9a040d4270 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7f9a03400640 1 -- 172.21.15.47:0/341042164 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f8031a80 con 0x7f9a040d4270 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7fa11ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa11406f090 0x7fa1140ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7fa11ae00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa11406df70 con 0x7fa114162c00 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7fa11ae00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa11406e310 con 0x7fa1140d0a70 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7fa11ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa114162fe0 con 0x7fa11406f090 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7fa113e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa114162c00 0x7fa1140d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33668/0 (socket says 172.21.15.47:33668) 2024-04-20T10:27:58.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7fa113e00640 1 -- 172.21.15.47:0/3553532793 learned_addr learned my addr 172.21.15.47:0/3553532793 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.470+0000 7f6bc0a00640 1 Processor -- start 2024-04-20T10:27:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7ff73ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73806bdd0 0x7ff7380731b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7ff73ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73806e510 0x7ff7380cfca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7ff73ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff738168920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7fa118e00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa11406f090 0x7fa1140ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3853263459 0 0) 0x7fa11406df70 con 0x7fa114162c00 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0f8003520 con 0x7fa114162c00 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 801601597 0 0) 0x7fa11406e310 con 0x7fa1140d0a70 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa11406df70 con 0x7fa1140d0a70 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3288089031 0 0) 0x7fa0f8003520 con 0x7fa114162c00 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa11406f090 msgr2=0x7fa1140ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7fa112a00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa11406f090 0x7fa1140ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 >> v1:172.21.15.175:6789/0 conn(0x7fa1140d0a70 legacy=0x7fa1140ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11406e5b0 con 0x7fa114162c00 2024-04-20T10:27:58.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7f9a0b800640 1 -- 172.21.15.47:0/341042164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 msgr2=0x7f9a040d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7f9a0b800640 1 --2- 172.21.15.47:0/341042164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a040d1a00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f9a040683c0 tx=0x7f99f802b900 comp rx=0 tx=0).stop 2024-04-20T10:27:58.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.471+0000 7f6bc0a00640 1 -- start start 2024-04-20T10:27:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7f9a0b800640 1 -- 172.21.15.47:0/341042164 shutdown_connections 2024-04-20T10:27:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7f9a0b800640 1 --2- 172.21.15.47:0/341042164 >> v2:172.21.15.47:3300/0 conn(0x7f9a040b17d0 0x7f9a040d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7f9a0b800640 1 --2- 172.21.15.47:0/341042164 >> v2:172.21.15.175:3300/0 conn(0x7f9a040d4650 0x7f9a040b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7f9a0b800640 1 --2- 172.21.15.47:0/341042164 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a040d1a00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff737e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73806bdd0 0x7ff7380731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff73ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff73815ac20 con 0x7ff7380b3bf0 2024-04-20T10:27:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff737400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73806e510 0x7ff7380cfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7f9a0b800640 1 -- 172.21.15.47:0/341042164 >> 172.21.15.47:0/341042164 conn(0x7f9a040b99c0 msgr2=0x7f9a040b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff73ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff738168920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff737e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73806bdd0 0x7ff7380731b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54236/0 (socket says 172.21.15.47:54236) 2024-04-20T10:27:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff737e00640 1 -- 172.21.15.47:0/2071214335 learned_addr learned my addr 172.21.15.47:0/2071214335 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff73815aaa0 con 0x7ff73806e510 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.472+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff73815ada0 con 0x7ff73806bdd0 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7ff73ce00640 1 -- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73806bdd0 msgr2=0x7ff7380731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7ff73ce00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73806bdd0 0x7ff7380731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7ff73ce00640 1 -- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73806e510 msgr2=0x7ff7380cfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7ff73ce00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73806e510 0x7ff7380cfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fa100001670 con 0x7fa114162c00 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 >> v1:172.21.15.47:6789/0 conn(0x7fa114162c00 legacy=0x7fa1140d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8003b70 0x7fa0f8003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7f6bc0a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb417d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f8004800 0x7fa0f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80051c0 0x7fa0f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f800c090 con 0x7fa0f80051c0 2024-04-20T10:27:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f800c210 con 0x7fa0f8003b70 2024-04-20T10:27:58.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f800c410 con 0x7fa0f8004800 2024-04-20T10:27:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7f6bc0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40589c0 0x7f6bb40da850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f6bc0a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40e7710 0x7f6bb4038a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f9a0b800640 1 -- 172.21.15.47:0/341042164 shutdown_connections 2024-04-20T10:27:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f9a0b800640 1 -- 172.21.15.47:0/341042164 wait complete. 2024-04-20T10:27:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.473+0000 7ff73ce00640 1 -- 172.21.15.47:0/2071214335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7380cec00 con 0x7ff7380b3bf0 2024-04-20T10:27:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7ff737400640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73806e510 0x7ff7380cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7ff737e00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73806bdd0 0x7ff7380731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:27:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f6bba800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb417d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f6bc0a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4057b20 con 0x7f6bb40589c0 2024-04-20T10:27:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f6bc0a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb40579a0 con 0x7f6bb40e7710 2024-04-20T10:27:58.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f6bc0a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bb4007020 con 0x7f6bb4056ab0 2024-04-20T10:27:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7fa113400640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f8004800 0x7fa0f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f6bba800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb417d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54248/0 (socket says 172.21.15.47:54248) 2024-04-20T10:27:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.474+0000 7f6bbb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40e7710 0x7f6bb4038a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.475+0000 7f6bbb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40e7710 0x7f6bb4038a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40872/0 (socket says 172.21.15.47:40872) 2024-04-20T10:27:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.475+0000 7f6bbb200640 1 -- 172.21.15.47:0/3212981423 learned_addr learned my addr 172.21.15.47:0/3212981423 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.475+0000 7ff73ce00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff738168920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d800 tx=0x7ff72800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.475+0000 7f6bb9e00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40589c0 0x7f6bb40da850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.475+0000 7fa118e00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80051c0 0x7fa0f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.475+0000 7f9957000640 1 Processor -- start 2024-04-20T10:27:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7fa113400640 1 -- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8003b70 msgr2=0x7fa0f8003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7fa113400640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8003b70 0x7fa0f8003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7fa113400640 1 -- 172.21.15.47:0/3553532793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80051c0 msgr2=0x7fa0f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7fa113400640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80051c0 0x7fa0f800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7fa113400640 1 -- 172.21.15.47:0/3553532793 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f8003520 con 0x7fa0f8004800 2024-04-20T10:27:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7fa118e00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80051c0 0x7fa0f800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7f6bbb200640 1 -- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 msgr2=0x7f6bb417d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7f6bbb200640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb417d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7f6bbb200640 1 -- 172.21.15.47:0/3212981423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40589c0 msgr2=0x7f6bb40da850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7f6bbb200640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40589c0 0x7f6bb40da850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7f6bbb200640 1 -- 172.21.15.47:0/3212981423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb403c9f0 con 0x7f6bb40e7710 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7ff72fe00640 1 -- 172.21.15.47:0/2071214335 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72800b6c0 con 0x7ff7380b3bf0 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7ff72fe00640 1 -- 172.21.15.47:0/2071214335 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff728019070 con 0x7ff7380b3bf0 2024-04-20T10:27:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7ff72fe00640 1 -- 172.21.15.47:0/2071214335 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72800ba80 con 0x7ff7380b3bf0 2024-04-20T10:27:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7f9a0b800640 1 Processor -- start 2024-04-20T10:27:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7380ceee0 con 0x7ff7380b3bf0 2024-04-20T10:27:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7380cf4a0 con 0x7ff7380b3bf0 2024-04-20T10:27:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f6bbb200640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40e7710 0x7f6bb4038a90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e870 tx=0x7f6ba800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f6bba800640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb417d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f9a0b800640 1 -- start start 2024-04-20T10:27:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.476+0000 7f9957000640 1 -- start start 2024-04-20T10:27:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f9957000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99500d4270 0x7f99500d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f9957000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f9957000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f99500d3820 con 0x7f99500b17d0 2024-04-20T10:27:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f9957000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f99500d40c0 con 0x7f99500d4270 2024-04-20T10:27:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.477+0000 7f9957000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99500d3b00 con 0x7f99500d4650 2024-04-20T10:27:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f994fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f99500d4270 0x7f99500d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7fa113400640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f8004800 0x7fa0f8004c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa104001900 tx=0x7fa0f8005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5a0d0a8eb46270f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9a0b800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a040b17d0 0x7f9a0406c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9a0b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a0414db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9a0b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a040d4650 0x7f9a040b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9a0b800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9a04161f70 con 0x7f9a040b17d0 2024-04-20T10:27:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9a0b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a040d53b0 con 0x7f9a040d4650 2024-04-20T10:27:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9a0b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a041620f0 con 0x7f9a040d4270 2024-04-20T10:27:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f994f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9955000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f99500b17d0 0x7f99500d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33678/0 (socket says 172.21.15.47:33678) 2024-04-20T10:27:58.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.478+0000 7f9955000640 1 -- 172.21.15.47:0/964074694 learned_addr learned my addr 172.21.15.47:0/964074694 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f6bb2a00640 1 -- 172.21.15.47:0/3212981423 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba8017070 con 0x7f6bb40e7710 2024-04-20T10:27:58.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f9a09800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a040d4650 0x7f9a040b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f9a09800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a040d4650 0x7f9a040b5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40898/0 (socket says 172.21.15.47:40898) 2024-04-20T10:27:58.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f9a09800640 1 -- 172.21.15.47:0/1455305498 learned_addr learned my addr 172.21.15.47:0/1455305498 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa104037070 con 0x7fa0f8004800 2024-04-20T10:27:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1040050d0 con 0x7fa0f8004800 2024-04-20T10:27:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7fa112a00640 1 -- 172.21.15.47:0/3553532793 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1040014f0 con 0x7fa0f8004800 2024-04-20T10:27:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb403ce40 con 0x7f6bb40e7710 2024-04-20T10:27:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f6bb2a00640 1 -- 172.21.15.47:0/3212981423 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ba800c940 con 0x7f6bb40e7710 2024-04-20T10:27:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb403d3c0 con 0x7f6bb40e7710 2024-04-20T10:27:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.480+0000 7f405f200640 1 Processor -- start 2024-04-20T10:27:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.480+0000 7f405f200640 1 -- start start 2024-04-20T10:27:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f9a08e00640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a040b17d0 0x7f9a0406c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f9a09800640 1 -- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 msgr2=0x7f9a0414db80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f9a09800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a0414db80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f9a09800640 1 -- 172.21.15.47:0/1455305498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a040b17d0 msgr2=0x7f9a0406c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7f9a09800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a040b17d0 0x7f9a0406c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7f9a09800640 1 -- 172.21.15.47:0/1455305498 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0406b9a0 con 0x7f9a040d4650 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7f9a08e00640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a040b17d0 0x7f9a0406c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7fa11ae00640 1 -- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f8004800 msgr2=0x7fa0f8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7fa11ae00640 1 --2- 172.21.15.47:0/3553532793 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f8004800 0x7fa0f8004c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa104001900 tx=0x7fa0f8005830 comp rx=0 tx=0).stop 2024-04-20T10:27:58.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f994f400640 1 -- 172.21.15.47:0/964074694 >> v2:172.21.15.47:3300/0 conn(0x7f99500d4270 msgr2=0x7f99500d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.480+0000 7f6bb2a00640 1 -- 172.21.15.47:0/3212981423 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba800cd10 con 0x7f6bb40e7710 2024-04-20T10:27:58.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7fe41ee00640 1 Processor -- start 2024-04-20T10:27:58.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f994f400640 1 --2- 172.21.15.47:0/964074694 >> v2:172.21.15.47:3300/0 conn(0x7f99500d4270 0x7f99500d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7fa118e00640 1 -- 172.21.15.47:0/3553532793 reap_dead start 2024-04-20T10:27:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f994ea00640 1 -- 172.21.15.47:0/964074694 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727367459 0 0) 0x7f99500d3820 con 0x7f99500b17d0 2024-04-20T10:27:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7fe41ee00640 1 -- start start 2024-04-20T10:27:58.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7fa11ae00640 1 -- 172.21.15.47:0/3553532793 shutdown_connections 2024-04-20T10:27:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.481+0000 7fa11ae00640 1 -- 172.21.15.47:0/3553532793 >> 172.21.15.47:0/3553532793 conn(0x7fa1140b3350 msgr2=0x7fa1140b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f994f400640 1 -- 172.21.15.47:0/964074694 >> v1:172.21.15.47:6789/0 conn(0x7f99500b17d0 legacy=0x7f99500d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f994f400640 1 -- 172.21.15.47:0/964074694 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99500d3de0 con 0x7f99500d4650 2024-04-20T10:27:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.479+0000 7f994fe00640 1 --2- 172.21.15.47:0/964074694 >> v2:172.21.15.47:3300/0 conn(0x7f99500d4270 0x7f99500d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:58.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f994f400640 1 --2- 172.21.15.47:0/964074694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500b1290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f993c0097a0 tx=0x7f993c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eae2222f9cdff34c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b94005290 con 0x7f6bb40e7710 2024-04-20T10:27:58.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.480+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff710005290 con 0x7ff7380b3bf0 2024-04-20T10:27:58.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7ff72fe00640 1 -- 172.21.15.47:0/2071214335 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff72800bc20 con 0x7ff7380b3bf0 2024-04-20T10:27:58.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7f9a09800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a040d4650 0x7f9a040b5090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00efc0 tx=0x7f99fc00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f405f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406015b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f405f200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406015b890 con 0x7f40600b3680 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f405f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f406015e2f0 con 0x7f40600adb80 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f405f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f406015e5d0 con 0x7f406015edd0 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7f405e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7f405e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406015b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54282/0 (socket says 172.21.15.47:54282) 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7f405e800640 1 -- 172.21.15.47:0/1183256590 learned_addr learned my addr 172.21.15.47:0/1183256590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f994ea00640 1 -- 172.21.15.47:0/964074694 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c035020 con 0x7f99500d4650 2024-04-20T10:27:58.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f994ea00640 1 -- 172.21.15.47:0/964074694 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f993c031830 con 0x7f99500d4650 2024-04-20T10:27:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7fe41ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4180d0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7f994ea00640 1 -- 172.21.15.47:0/964074694 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c031c40 con 0x7f99500d4650 2024-04-20T10:27:58.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7fe41ee00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe418072c70 0x7fe41806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.483+0000 7fe41ee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe418155030 con 0x7fe41815bd40 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7fe41ee00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe41806dbb0 con 0x7fe418072c70 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7fe41ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe41806de90 con 0x7fe418154c50 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7fe417400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe41815bd40 0x7fe418072560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33698/0 (socket says 172.21.15.47:33698) 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7fe417400640 1 -- 172.21.15.47:0/1480648401 learned_addr learned my addr 172.21.15.47:0/1480648401 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7fe417e00640 1 --2- 172.21.15.47:0/1480648401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4180d0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f9a00c00640 1 -- 172.21.15.47:0/1455305498 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc018020 con 0x7f9a040d4650 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f9a00c00640 1 -- 172.21.15.47:0/1455305498 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f99fc014480 con 0x7f9a040d4650 2024-04-20T10:27:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f9a00c00640 1 -- 172.21.15.47:0/1455305498 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99fc014870 con 0x7f9a040d4650 2024-04-20T10:27:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f405e800640 1 -- 172.21.15.47:0/1183256590 >> v1:172.21.15.175:6789/0 conn(0x7f40600adb80 legacy=0x7f40600b2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f405e800640 1 -- 172.21.15.47:0/1183256590 >> v1:172.21.15.47:6789/0 conn(0x7f40600b3680 legacy=0x7f406015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f405e800640 1 -- 172.21.15.47:0/1183256590 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406015e8b0 con 0x7f406015edd0 2024-04-20T10:27:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0406bc80 con 0x7f9a040d4650 2024-04-20T10:27:58.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.484+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0414e0c0 con 0x7f9a040d4650 2024-04-20T10:27:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.485+0000 7fe417e00640 1 -- 172.21.15.47:0/1480648401 >> v2:172.21.15.47:3300/0 conn(0x7fe418072c70 msgr2=0x7fe41806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.485+0000 7fe417e00640 1 --2- 172.21.15.47:0/1480648401 >> v2:172.21.15.47:3300/0 conn(0x7fe418072c70 0x7fe41806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.485+0000 7fe417e00640 1 -- 172.21.15.47:0/1480648401 >> v1:172.21.15.47:6789/0 conn(0x7fe41815bd40 legacy=0x7fe418072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.485+0000 7fe417e00640 1 -- 172.21.15.47:0/1480648401 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe41806e170 con 0x7fe418154c50 2024-04-20T10:27:58.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.482+0000 7fa11ae00640 1 -- 172.21.15.47:0/3553532793 shutdown_connections 2024-04-20T10:27:58.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.485+0000 7f6bb2a00640 1 -- 172.21.15.47:0/3212981423 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6ba801a070 con 0x7f6bb40e7710 2024-04-20T10:27:58.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.486+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a04068980 con 0x7f9a040d4650 2024-04-20T10:27:58.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.486+0000 7f405e800640 1 --2- 172.21.15.47:0/1183256590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406015b270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4060075400 tx=0x7f404802cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e57cb94928f2d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7fe417e00640 1 --2- 172.21.15.47:0/1480648401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4180d0680 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe41000ad90 tx=0x7fe41002dfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b5a0284b7509038 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7f6bb2a00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6b9c06c940 0x7f6b9c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7f6bb2a00640 1 -- 172.21.15.47:0/3212981423 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f6ba808be20 con 0x7f6bb40e7710 2024-04-20T10:27:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7f9957000640 1 -- 172.21.15.47:0/964074694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 msgr2=0x7f99500b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7f9957000640 1 --2- 172.21.15.47:0/964074694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500b1290 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f993c0097a0 tx=0x7f993c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7fe416a00640 1 -- 172.21.15.47:0/1480648401 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41003b070 con 0x7fe418154c50 2024-04-20T10:27:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7fe416a00640 1 -- 172.21.15.47:0/1480648401 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe41002ed50 con 0x7fe418154c50 2024-04-20T10:27:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7fe416a00640 1 -- 172.21.15.47:0/1480648401 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410036660 con 0x7fe418154c50 2024-04-20T10:27:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7f405ca00640 1 -- 172.21.15.47:0/1183256590 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404803a070 con 0x7f406015edd0 2024-04-20T10:27:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7f405ca00640 1 -- 172.21.15.47:0/1183256590 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f404802da50 con 0x7f406015edd0 2024-04-20T10:27:58.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7f405ca00640 1 -- 172.21.15.47:0/1183256590 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f404803a070 con 0x7f406015edd0 2024-04-20T10:27:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.488+0000 7f9957000640 1 -- 172.21.15.47:0/964074694 shutdown_connections 2024-04-20T10:27:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.488+0000 7f9957000640 1 --2- 172.21.15.47:0/964074694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500b1290 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.488+0000 7f9957000640 1 --2- 172.21.15.47:0/964074694 >> v2:172.21.15.47:3300/0 conn(0x7f99500d4270 0x7f99500d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.488+0000 7f9957000640 1 -- 172.21.15.47:0/964074694 >> 172.21.15.47:0/964074694 conn(0x7f99500b99c0 msgr2=0x7f99500b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.486+0000 7ff72fe00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff71806c970 0x7ff71806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7ff72fe00640 1 -- 172.21.15.47:0/2071214335 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff72808c1c0 con 0x7ff7380b3bf0 2024-04-20T10:27:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.487+0000 7ff737e00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff71806c970 0x7ff71806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.486+0000 7fa11ae00640 1 -- 172.21.15.47:0/3553532793 wait complete. 2024-04-20T10:27:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.488+0000 7f6bba800640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6b9c06c940 0x7f6b9c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.488+0000 7fa11ae00640 1 Processor -- start 2024-04-20T10:27:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.488+0000 7f3329e00640 1 Processor -- start 2024-04-20T10:27:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f405f200640 1 -- 172.21.15.47:0/1183256590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 msgr2=0x7f406015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f405f200640 1 --2- 172.21.15.47:0/1183256590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406015b270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f4060075400 tx=0x7f404802cca0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7fe41ee00640 1 -- 172.21.15.47:0/1480648401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 msgr2=0x7fe4180d0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7fe41ee00640 1 --2- 172.21.15.47:0/1480648401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4180d0680 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe41000ad90 tx=0x7fe41002dfa0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f405f200640 1 -- 172.21.15.47:0/1183256590 shutdown_connections 2024-04-20T10:27:58.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f405f200640 1 --2- 172.21.15.47:0/1183256590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406015b270 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f405f200640 1 -- 172.21.15.47:0/1183256590 >> 172.21.15.47:0/1183256590 conn(0x7f40600b1c70 msgr2=0x7f40600b2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f9957000640 1 -- 172.21.15.47:0/964074694 shutdown_connections 2024-04-20T10:27:58.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f9957000640 1 -- 172.21.15.47:0/964074694 wait complete. 2024-04-20T10:27:58.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7fa11ae00640 1 -- start start 2024-04-20T10:27:58.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f3329e00640 1 -- start start 2024-04-20T10:27:58.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f405f200640 1 -- 172.21.15.47:0/1183256590 shutdown_connections 2024-04-20T10:27:58.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7ff737e00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff71806c970 0x7ff71806ee30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f80 tx=0x7ff730030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7f405f200640 1 -- 172.21.15.47:0/1183256590 wait complete. 2024-04-20T10:27:58.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7fe41ee00640 1 -- 172.21.15.47:0/1480648401 shutdown_connections 2024-04-20T10:27:58.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.489+0000 7fe41ee00640 1 --2- 172.21.15.47:0/1480648401 >> v2:172.21.15.47:3300/0 conn(0x7fe418072c70 0x7fe41806ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7f6bba800640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6b9c06c940 0x7f6b9c06ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40e4cb0 tx=0x7f6bac009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fe41ee00640 1 --2- 172.21.15.47:0/1480648401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4180d0680 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fe41ee00640 1 -- 172.21.15.47:0/1480648401 >> 172.21.15.47:0/1480648401 conn(0x7fe4180b3350 msgr2=0x7fe4180b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fe41ee00640 1 -- 172.21.15.47:0/1480648401 shutdown_connections 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fe41ee00640 1 -- 172.21.15.47:0/1480648401 wait complete. 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fa11ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8004800 0x7fa114159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fa11ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa114162fe0 0x7fa1142005f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fa11ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa114200b30 0x7fa114157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fa11ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa11406dbb0 con 0x7fa114162fe0 2024-04-20T10:27:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fa11ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa11406ec00 con 0x7fa0f8004800 2024-04-20T10:27:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fa11ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa11406dd30 con 0x7fa114200b30 2024-04-20T10:27:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7f3329e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f33240d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7f3329e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f33240d0380 0x7f33240d0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7f3329e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33240d0d50 con 0x7f33240d4650 2024-04-20T10:27:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7f3329e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f33240d40c0 con 0x7f33240d0380 2024-04-20T10:27:58.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7f3329e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33240d3820 con 0x7f33240d4270 2024-04-20T10:27:58.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.490+0000 7fa113e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8004800 0x7fa114159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7fa113e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8004800 0x7fa114159120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40906/0 (socket says 172.21.15.47:40906) 2024-04-20T10:27:58.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7fa113e00640 1 -- 172.21.15.47:0/2112171198 learned_addr learned my addr 172.21.15.47:0/2112171198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3322a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f33240d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3322a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f33240d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54288/0 (socket says 172.21.15.47:54288) 2024-04-20T10:27:58.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3322a00640 1 -- 172.21.15.47:0/556502080 learned_addr learned my addr 172.21.15.47:0/556502080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7fa118e00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa114200b30 0x7fa114157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3323400640 1 --2- 172.21.15.47:0/556502080 >> v2:172.21.15.175:3300/0 conn(0x7f33240d0380 0x7f33240d0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7fa113400640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa114162fe0 0x7fa1142005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3322a00640 1 -- 172.21.15.47:0/556502080 >> v2:172.21.15.175:3300/0 conn(0x7f33240d0380 msgr2=0x7f33240d0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3322a00640 1 --2- 172.21.15.47:0/556502080 >> v2:172.21.15.175:3300/0 conn(0x7f33240d0380 0x7f33240d0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3322a00640 1 -- 172.21.15.47:0/556502080 >> v1:172.21.15.47:6789/0 conn(0x7f33240d4650 legacy=0x7f33240b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:27:58.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f3322a00640 1 -- 172.21.15.47:0/556502080 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33240d3b00 con 0x7f33240d4270 2024-04-20T10:27:58.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f405f200640 1 Processor -- start 2024-04-20T10:27:58.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f405f200640 1 -- start start 2024-04-20T10:27:58.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.491+0000 7f9957000640 1 Processor -- start 2024-04-20T10:27:58.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7fa113e00640 1 -- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa114200b30 msgr2=0x7fa114157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7f9957000640 1 -- start start 2024-04-20T10:27:58.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7fa113e00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa114200b30 0x7fa114157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7fa113e00640 1 -- 172.21.15.47:0/2112171198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa114162fe0 msgr2=0x7fa1142005f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7fa113e00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa114162fe0 0x7fa1142005f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7fa113e00640 1 -- 172.21.15.47:0/2112171198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1141641e0 con 0x7fa0f8004800 2024-04-20T10:27:58.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7f3322a00640 1 --2- 172.21.15.47:0/556502080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f33240d1a00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3314009870 tx=0x7f331402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc52f685842deee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.492+0000 7fe41ee00640 1 Processor -- start 2024-04-20T10:27:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40600adb80 0x7f4060151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40600b3680 0x7f4060156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406009c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f40600b2910 con 0x7f40600adb80 2024-04-20T10:27:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40600b2790 con 0x7f40600b3680 2024-04-20T10:27:58.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40601696c0 con 0x7f406015edd0 2024-04-20T10:27:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7fa113e00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8004800 0x7fa114159120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c760 tx=0x7fa10000cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f3321600640 1 -- 172.21.15.47:0/556502080 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314035020 con 0x7f33240d4270 2024-04-20T10:27:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f3321600640 1 -- 172.21.15.47:0/556502080 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3314031640 con 0x7f33240d4270 2024-04-20T10:27:58.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f3321600640 1 -- 172.21.15.47:0/556502080 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331402d040 con 0x7f33240d4270 2024-04-20T10:27:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40600adb80 0x7f4060151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40600adb80 0x7f4060151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56050/0 (socket says 172.21.15.47:56050) 2024-04-20T10:27:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405de00640 1 -- 172.21.15.47:0/1271882630 learned_addr learned my addr 172.21.15.47:0/1271882630 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f405d400640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40600b3680 0x7f4060156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f9957000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500b17d0 0x7f99500b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f9957000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99500d4270 0x7f99500ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f9957000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f9957000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9950161f70 con 0x7f99500d4270 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f9957000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99500d53b0 con 0x7f99500b17d0 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.493+0000 7f9957000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99501620f0 con 0x7f99500d4650 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405e800640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406009c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 -- 172.21.15.47:0/556502080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 msgr2=0x7f33240d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 --2- 172.21.15.47:0/556502080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f33240d1a00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3314009870 tx=0x7f331402b8f0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405de00640 1 -- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 msgr2=0x7f406009c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405de00640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406009c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405de00640 1 -- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40600b3680 msgr2=0x7f4060156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405de00640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40600b3680 0x7f4060156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405de00640 1 -- 172.21.15.47:0/1271882630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406009cdb0 con 0x7f40600adb80 2024-04-20T10:27:58.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405e800640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:58.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 -- 172.21.15.47:0/556502080 shutdown_connections 2024-04-20T10:27:58.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 --2- 172.21.15.47:0/556502080 >> v2:172.21.15.175:3300/0 conn(0x7f33240d0380 0x7f33240d0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 --2- 172.21.15.47:0/556502080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f33240d1a00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 -- 172.21.15.47:0/556502080 >> 172.21.15.47:0/556502080 conn(0x7f33240b99c0 msgr2=0x7f33240b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 -- 172.21.15.47:0/556502080 shutdown_connections 2024-04-20T10:27:58.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f405d400640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40600b3680 0x7f4060156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:27:58.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f3329e00640 1 -- 172.21.15.47:0/556502080 wait complete. 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f9955000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f994fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500b17d0 0x7f99500b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f994fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500b17d0 0x7f99500b4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40928/0 (socket says 172.21.15.47:40928) 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f994fe00640 1 -- 172.21.15.47:0/2724592882 learned_addr learned my addr 172.21.15.47:0/2724592882 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f994f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99500d4270 0x7f99500ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.494+0000 7f9955000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54316/0 (socket says 172.21.15.47:54316) 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f994fe00640 1 -- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 msgr2=0x7f99500ca630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f994fe00640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500ca630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f994fe00640 1 -- 172.21.15.47:0/2724592882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99500d4270 msgr2=0x7f99500ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f994fe00640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99500d4270 0x7f99500ca0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f994fe00640 1 -- 172.21.15.47:0/2724592882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99500cdca0 con 0x7f99500b17d0 2024-04-20T10:27:58.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7ff72fe00640 1 -- 172.21.15.47:0/2071214335 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff72805a290 con 0x7ff7380b3bf0 2024-04-20T10:27:58.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7fe41ee00640 1 -- start start 2024-04-20T10:27:58.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f405de00640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40600adb80 0x7f4060151650 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f405800c400 tx=0x7f405800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f4056000640 1 -- 172.21.15.47:0/1271882630 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058017070 con 0x7f40600adb80 2024-04-20T10:27:58.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f4056000640 1 -- 172.21.15.47:0/1271882630 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40580052d0 con 0x7f40600adb80 2024-04-20T10:27:58.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f994fe00640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500b17d0 0x7f99500b4a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f994000c820 tx=0x7f994000cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f4056000640 1 -- 172.21.15.47:0/1271882630 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058003830 con 0x7f40600adb80 2024-04-20T10:27:58.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.495+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4060158970 con 0x7f40600adb80 2024-04-20T10:27:58.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.496+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4060158f30 con 0x7f40600adb80 2024-04-20T10:27:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.496+0000 7f3329e00640 1 Processor -- start 2024-04-20T10:27:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.496+0000 7f3329e00640 1 -- start start 2024-04-20T10:27:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4060159130 con 0x7f40600adb80 2024-04-20T10:27:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7fe41ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe418072c70 0x7fe418150ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7fe41ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4181557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7f3329e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33240d0380 0x7f332406c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7f3329e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f3324201340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7f3329e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33240d4650 0x7f33240b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7f3329e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f33240b19a0 con 0x7f33240d4650 2024-04-20T10:27:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7f3329e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33240cfe60 con 0x7f33240d0380 2024-04-20T10:27:58.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7f3329e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33240d53b0 con 0x7f33240d4270 2024-04-20T10:27:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.497+0000 7fe41ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41815bd40 0x7fe4181571f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7fe417e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe418072c70 0x7fe418150ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7fe41ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe4180d0bc0 con 0x7fe418072c70 2024-04-20T10:27:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33240d4650 0x7f33240b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7fe417e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe418072c70 0x7fe418150ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56070/0 (socket says 172.21.15.47:56070) 2024-04-20T10:27:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7fe41ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe41806e730 con 0x7fe41815bd40 2024-04-20T10:27:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7fe417e00640 1 -- 172.21.15.47:0/3878201930 learned_addr learned my addr 172.21.15.47:0/3878201930 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7fe417400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4181557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7fe41ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4180d0d40 con 0x7fe418154c50 2024-04-20T10:27:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33240d4650 0x7f33240b5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56084/0 (socket says 172.21.15.47:56084) 2024-04-20T10:27:58.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 -- 172.21.15.47:0/2467346224 learned_addr learned my addr 172.21.15.47:0/2467346224 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3322a00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33240d0380 0x7f332406c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3322000640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f3324201340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 -- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 msgr2=0x7f3324201340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f3324201340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 -- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33240d0380 msgr2=0x7f332406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33240d0380 0x7f332406c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.498+0000 7f3323400640 1 -- 172.21.15.47:0/2467346224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33240cb850 con 0x7f33240d4650 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.499+0000 7fe417e00640 1 -- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 msgr2=0x7fe4181557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.499+0000 7fe417e00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4181557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.499+0000 7fe417e00640 1 -- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41815bd40 msgr2=0x7fe4181571f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.499+0000 7f3323400640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33240d4650 0x7f33240b5480 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f331c007e40 tx=0x7f331c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.499+0000 7f331b400640 1 -- 172.21.15.47:0/2467346224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c014620 con 0x7f33240d4650 2024-04-20T10:27:58.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.500+0000 7f331b400640 1 -- 172.21.15.47:0/2467346224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f331c019070 con 0x7f33240d4650 2024-04-20T10:27:58.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.500+0000 7f331b400640 1 -- 172.21.15.47:0/2467346224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331c014a20 con 0x7f33240d4650 2024-04-20T10:27:58.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.500+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33240cbb30 con 0x7f33240d4650 2024-04-20T10:27:58.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.500+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33240cc0f0 con 0x7f33240d4650 2024-04-20T10:27:58.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.499+0000 7fe41ce00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41815bd40 0x7fe4181571f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.501+0000 7fe417e00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41815bd40 0x7fe4181571f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.501+0000 7fe417e00640 1 -- 172.21.15.47:0/3878201930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe418158020 con 0x7fe418072c70 2024-04-20T10:27:58.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.501+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3324068980 con 0x7f33240d4650 2024-04-20T10:27:58.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.501+0000 7fe41ce00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41815bd40 0x7fe4181571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:27:58.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.501+0000 7fe417e00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe418072c70 0x7fe418150ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe41000ad90 tx=0x7fe410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.499+0000 7f4056000640 1 -- 172.21.15.47:0/1271882630 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f405801a070 con 0x7f40600adb80 2024-04-20T10:27:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.500+0000 7f4056000640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f403c06c970 0x7f403c06ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.500+0000 7f4056000640 1 -- 172.21.15.47:0/1271882630 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f405808b1c0 con 0x7f40600adb80 2024-04-20T10:27:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.501+0000 7f405d400640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f403c06c970 0x7f403c06ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.502+0000 7f405d400640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f403c06c970 0x7f403c06ee30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f40600cc750 tx=0x7f4050009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.506+0000 7f6bb2a00640 1 -- 172.21.15.47:0/3212981423 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6bb403c9f0 con 0x7f6bb40e7710 2024-04-20T10:27:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.506+0000 7fa10be00640 1 -- 172.21.15.47:0/2112171198 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100018070 con 0x7fa0f8004800 2024-04-20T10:27:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.506+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1141644c0 con 0x7fa0f8004800 2024-04-20T10:27:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.506+0000 7fa10be00640 1 -- 172.21.15.47:0/2112171198 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa1000072d0 con 0x7fa0f8004800 2024-04-20T10:27:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.506+0000 7fa10be00640 1 -- 172.21.15.47:0/2112171198 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1000047e0 con 0x7fa0f8004800 2024-04-20T10:27:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.506+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa114164a80 con 0x7fa0f8004800 2024-04-20T10:27:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.507+0000 7f9947e00640 1 -- 172.21.15.47:0/2724592882 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940015070 con 0x7f99500b17d0 2024-04-20T10:27:58.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.508+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa114163f70 con 0x7fa0f8004800 2024-04-20T10:27:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.507+0000 7f9947e00640 1 -- 172.21.15.47:0/2724592882 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9940007a30 con 0x7f99500b17d0 2024-04-20T10:27:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.507+0000 7f9947e00640 1 -- 172.21.15.47:0/2724592882 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9940007e00 con 0x7f99500b17d0 2024-04-20T10:27:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.507+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99500cc660 con 0x7f99500b17d0 2024-04-20T10:27:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.507+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99500ccc20 con 0x7f99500b17d0 2024-04-20T10:27:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.508+0000 7f9a00c00640 1 -- 172.21.15.47:0/1455305498 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f99fc005d10 con 0x7f9a040d4650 2024-04-20T10:27:58.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.509+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9950068980 con 0x7f99500b17d0 2024-04-20T10:27:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.509+0000 7f9a00c00640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f99e406c940 0x7f99e406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.509+0000 7f9a00c00640 1 -- 172.21.15.47:0/1455305498 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f99fc08d400 con 0x7f9a040d4650 2024-04-20T10:27:58.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.510+0000 7fe40fe00640 1 -- 172.21.15.47:0/3878201930 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe410046070 con 0x7fe418072c70 2024-04-20T10:27:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.510+0000 7fe40fe00640 1 -- 172.21.15.47:0/3878201930 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe410041430 con 0x7fe418072c70 2024-04-20T10:27:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.510+0000 7fe40fe00640 1 -- 172.21.15.47:0/3878201930 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe41003b040 con 0x7fe418072c70 2024-04-20T10:27:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.510+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe418157a60 con 0x7fe418072c70 2024-04-20T10:27:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.510+0000 7f4056000640 1 -- 172.21.15.47:0/1271882630 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4060159130 con 0x7f40600adb80 2024-04-20T10:27:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.510+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4180afac0 con 0x7fe418072c70 2024-04-20T10:27:58.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.511+0000 7f9a08e00640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f99e406c940 0x7f99e406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.512+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe418068980 con 0x7fe418072c70 2024-04-20T10:27:58.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.512+0000 7f9a08e00640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f99e406c940 0x7f99e406ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99f8036000 tx=0x7f99f8008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.513+0000 7f331b400640 1 -- 172.21.15.47:0/2467346224 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f331c014bc0 con 0x7f33240d4650 2024-04-20T10:27:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.513+0000 7f331b400640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f330406c940 0x7f330406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.513+0000 7f331b400640 1 -- 172.21.15.47:0/2467346224 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f331c08d530 con 0x7f33240d4650 2024-04-20T10:27:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.516+0000 7f3322a00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f330406c940 0x7f330406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.517+0000 7f3322a00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f330406c940 0x7f330406ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3314036000 tx=0x7f331403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.519+0000 7f331b400640 1 -- 172.21.15.47:0/2467346224 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f331c05afb0 con 0x7f33240d4650 2024-04-20T10:27:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.521+0000 7f9a00c00640 1 -- 172.21.15.47:0/1455305498 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f99fc022020 con 0x7f9a040d4650 2024-04-20T10:27:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.521+0000 7fe40fe00640 1 -- 172.21.15.47:0/3878201930 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe410049070 con 0x7fe418072c70 2024-04-20T10:27:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.522+0000 7fe40fe00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe3f406c940 0x7fe3f406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.523+0000 7fe40fe00640 1 -- 172.21.15.47:0/3878201930 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fe4100adea0 con 0x7fe418072c70 2024-04-20T10:27:58.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.524+0000 7fe417400640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe3f406c940 0x7fe3f406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.525+0000 7fa10be00640 1 -- 172.21.15.47:0/2112171198 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa10001b070 con 0x7fa0f8004800 2024-04-20T10:27:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.526+0000 7fa10be00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.527+0000 7fa10be00640 1 -- 172.21.15.47:0/2112171198 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa10008b750 con 0x7fa0f8004800 2024-04-20T10:27:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.527+0000 7f9947e00640 1 -- 172.21.15.47:0/2724592882 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9940018070 con 0x7f99500b17d0 2024-04-20T10:27:58.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.529+0000 7fe417400640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe3f406c940 0x7fe3f406ee00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe418153980 tx=0x7fe400005fc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.528+0000 7f9947e00640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f993006c9f0 0x7f993006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:27:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.529+0000 7f9947e00640 1 -- 172.21.15.47:0/2724592882 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f994008c1f0 con 0x7f99500b17d0 2024-04-20T10:27:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.529+0000 7fe40fe00640 1 -- 172.21.15.47:0/3878201930 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe410051080 con 0x7fe418072c70 2024-04-20T10:27:58.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.530+0000 7fa113400640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.530+0000 7f994f400640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f993006c9f0 0x7f993006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:27:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.533+0000 7f994f400640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f993006c9f0 0x7f993006eeb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f993c036000 tx=0x7f993c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.533+0000 7fa113400640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa104034000 tx=0x7fa104031000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:27:58.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.537+0000 7fa10be00640 1 -- 172.21.15.47:0/2112171198 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa100022020 con 0x7fa0f8004800 2024-04-20T10:27:58.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.548+0000 7f9947e00640 1 -- 172.21.15.47:0/2724592882 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9940059c70 con 0x7f99500b17d0 2024-04-20T10:27:58.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.952+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7ff710005580 con 0x7ff7380b3bf0 2024-04-20T10:27:58.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.953+0000 7ff72fe00640 1 -- 172.21.15.47:0/2071214335 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff728059b90 con 0x7ff7380b3bf0 2024-04-20T10:27:58.953 INFO:teuthology.orchestra.run.smithi047.stdout:64424509449 2024-04-20T10:27:58.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.956+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f4060075b30 con 0x7f40600adb80 2024-04-20T10:27:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.956+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff71806c970 msgr2=0x7ff71806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff71806c970 0x7ff71806ee30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff730000f80 tx=0x7ff730030000 comp rx=0 tx=0).stop 2024-04-20T10:27:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 msgr2=0x7ff738168920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff738168920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff72800d800 tx=0x7ff72800dcc0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 shutdown_connections 2024-04-20T10:27:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff71806c970 0x7ff71806ee30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7380b3bf0 0x7ff738168920 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff73806e510 0x7ff7380cfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 --2- 172.21.15.47:0/2071214335 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff73806bdd0 0x7ff7380731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 >> 172.21.15.47:0/2071214335 conn(0x7ff738072090 msgr2=0x7ff738142890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.958+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 shutdown_connections 2024-04-20T10:27:58.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.958+0000 7ff73ee00640 1 -- 172.21.15.47:0/2071214335 wait complete. 2024-04-20T10:27:58.961 INFO:teuthology.orchestra.run.smithi047.stdout:64424509449 2024-04-20T10:27:58.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.957+0000 7f4056000640 1 -- 172.21.15.47:0/1271882630 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4058059320 con 0x7f40600adb80 2024-04-20T10:27:58.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.961+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f6b94005850 con 0x7f6bb40e7710 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f403c06c970 msgr2=0x7f403c06ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f403c06c970 0x7f403c06ee30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f40600cc750 tx=0x7f4050009040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40600adb80 msgr2=0x7f4060151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40600adb80 0x7f4060151650 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f405800c400 tx=0x7f405800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 shutdown_connections 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406015edd0 0x7f406009c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40600b3680 0x7f4060156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f403c06c970 0x7f403c06ee30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 --2- 172.21.15.47:0/1271882630 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40600adb80 0x7f4060151650 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.962+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 >> 172.21.15.47:0/1271882630 conn(0x7f40600b1c70 msgr2=0x7f4060072310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.963+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 shutdown_connections 2024-04-20T10:27:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.963+0000 7f6bb2a00640 1 -- 172.21.15.47:0/3212981423 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6ba8059930 con 0x7f6bb40e7710 2024-04-20T10:27:58.964 INFO:teuthology.orchestra.run.smithi047.stdout:64424509449 2024-04-20T10:27:58.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.963+0000 7f405f200640 1 -- 172.21.15.47:0/1271882630 wait complete. 2024-04-20T10:27:58.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.966+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6b9c06c940 msgr2=0x7f6b9c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6b9c06c940 0x7f6b9c06ee00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40e4cb0 tx=0x7f6bac009040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40e7710 msgr2=0x7f6bb4038a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40e7710 0x7f6bb4038a90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6ba800e870 tx=0x7f6ba800ed30 comp rx=0 tx=0).stop 2024-04-20T10:27:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 shutdown_connections 2024-04-20T10:27:58.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bb40e7710 0x7f6bb4038a90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6b9c06c940 0x7f6b9c06ee00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bb40589c0 0x7f6bb40da850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 --2- 172.21.15.47:0/3212981423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bb4056ab0 0x7f6bb417d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.967+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 >> 172.21.15.47:0/3212981423 conn(0x7f6bb4058ee0 msgr2=0x7f6bb40e1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.968+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 shutdown_connections 2024-04-20T10:27:58.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.968+0000 7f6bc0a00640 1 -- 172.21.15.47:0/3212981423 wait complete. 2024-04-20T10:27:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.975+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f9a0406ea50 con 0x7f9a040d4650 2024-04-20T10:27:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.975+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f33240d3560 con 0x7f33240d4650 2024-04-20T10:27:58.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.975+0000 7f331b400640 1 -- 172.21.15.47:0/2467346224 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f331c05f2b0 con 0x7f33240d4650 2024-04-20T10:27:58.976 INFO:teuthology.orchestra.run.smithi047.stdout:64424509449 2024-04-20T10:27:58.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.976+0000 7f9a00c00640 1 -- 172.21.15.47:0/1455305498 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f99fc010040 con 0x7f9a040d4650 2024-04-20T10:27:58.977 INFO:teuthology.orchestra.run.smithi047.stdout:64424509449 2024-04-20T10:27:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f99e406c940 msgr2=0x7f99e406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f99e406c940 0x7f99e406ee00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f99f8036000 tx=0x7f99f8008030 comp rx=0 tx=0).stop 2024-04-20T10:27:58.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a040d4650 msgr2=0x7f9a040b5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a040d4650 0x7f9a040b5090 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f99fc00efc0 tx=0x7f99fc00c490 comp rx=0 tx=0).stop 2024-04-20T10:27:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.979+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fa114068980 con 0x7fa0f8004800 2024-04-20T10:27:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f330406c940 msgr2=0x7f330406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f330406c940 0x7f330406ee00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3314036000 tx=0x7f331403c040 comp rx=0 tx=0).stop 2024-04-20T10:27:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33240d4650 msgr2=0x7f33240b5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33240d4650 0x7f33240b5480 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f331c007e40 tx=0x7f331c00e4b0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 shutdown_connections 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a040d4650 0x7f9a040b5090 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a040d4270 0x7f9a0414db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f99e406c940 0x7f99e406ee00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 --2- 172.21.15.47:0/1455305498 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9a040b17d0 0x7f9a0406c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 >> 172.21.15.47:0/1455305498 conn(0x7f9a040b99c0 msgr2=0x7f9a040731a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 shutdown_connections 2024-04-20T10:27:58.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f9a0b800640 1 -- 172.21.15.47:0/1455305498 wait complete. 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 shutdown_connections 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f330406c940 0x7f330406ee00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33240d4650 0x7f33240b5480 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33240d4270 0x7f3324201340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 --2- 172.21.15.47:0/2467346224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33240d0380 0x7f332406c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 >> 172.21.15.47:0/2467346224 conn(0x7f33240b99c0 msgr2=0x7f33240d0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 shutdown_connections 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.980+0000 7f3329e00640 1 -- 172.21.15.47:0/2467346224 wait complete. 2024-04-20T10:27:58.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.981+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe41806f700 con 0x7fe418072c70 2024-04-20T10:27:58.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.981+0000 7fa10be00640 1 -- 172.21.15.47:0/2112171198 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1000591d0 con 0x7fa0f8004800 2024-04-20T10:27:58.986 INFO:teuthology.orchestra.run.smithi047.stdout:64424509449 2024-04-20T10:27:58.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.982+0000 7fe40fe00640 1 -- 172.21.15.47:0/3878201930 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe410006e20 con 0x7fe418072c70 2024-04-20T10:27:58.987 INFO:teuthology.orchestra.run.smithi047.stdout:60129542153 2024-04-20T10:27:58.988 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.3 2024-04-20T10:27:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 msgr2=0x7fa0f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fa11ae00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa104034000 tx=0x7fa104031000 comp rx=0 tx=0).stop 2024-04-20T10:27:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8004800 msgr2=0x7fa114159120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fa11ae00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8004800 0x7fa114159120 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c760 tx=0x7fa10000cc20 comp rx=0 tx=0).stop 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe3f406c940 msgr2=0x7fe3f406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fe41ee00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe3f406c940 0x7fe3f406ee00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe418153980 tx=0x7fe400005fc0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe418072c70 msgr2=0x7fe418150ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fe41ee00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe418072c70 0x7fe418150ac0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe41000ad90 tx=0x7fe410004a40 comp rx=0 tx=0).stop 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 shutdown_connections 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.985+0000 7fe41ee00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe41815bd40 0x7fe4181571f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fe41ee00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe418154c50 0x7fe4181557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fe41ee00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe3f406c940 0x7fe3f406ee00 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe418153980 tx=0x7fe400005fc0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fe41ee00640 1 --2- 172.21.15.47:0/3878201930 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe418072c70 0x7fe418150ac0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 >> 172.21.15.47:0/3878201930 conn(0x7fe4180b3350 msgr2=0x7fe418071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 shutdown_connections 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fe41ee00640 1 -- 172.21.15.47:0/3878201930 wait complete. 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 shutdown_connections 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fa11ae00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa114200b30 0x7fa114157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fa11ae00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fa11ae00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa114162fe0 0x7fa1142005f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fa11ae00640 1 --2- 172.21.15.47:0/2112171198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8004800 0x7fa114159120 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.986+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 >> 172.21.15.47:0/2112171198 conn(0x7fa1140b3350 msgr2=0x7fa1141604b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:58.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.987+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 shutdown_connections 2024-04-20T10:27:58.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.987+0000 7fa11ae00640 1 -- 172.21.15.47:0/2112171198 wait complete. 2024-04-20T10:27:58.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.6 2024-04-20T10:27:58.995 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.0 2024-04-20T10:27:58.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.997+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f99500cac70 con 0x7f99500b17d0 2024-04-20T10:27:58.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:58.998+0000 7f9947e00640 1 -- 172.21.15.47:0/2724592882 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f994005df70 con 0x7f99500b17d0 2024-04-20T10:27:58.998 INFO:teuthology.orchestra.run.smithi047.stdout:64424509449 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f993006c9f0 msgr2=0x7f993006eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f993006c9f0 0x7f993006eeb0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f993c036000 tx=0x7f993c03c040 comp rx=0 tx=0).stop 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500b17d0 msgr2=0x7f99500b4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500b17d0 0x7f99500b4a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f994000c820 tx=0x7f994000cc20 comp rx=0 tx=0).stop 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 shutdown_connections 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99500d4650 0x7f99500ca630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f993006c9f0 0x7f993006eeb0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f99500d4270 0x7f99500ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 --2- 172.21.15.47:0/2724592882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99500b17d0 0x7f99500b4a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 >> 172.21.15.47:0/2724592882 conn(0x7f99500b99c0 msgr2=0x7f99500d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 shutdown_connections 2024-04-20T10:27:59.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:27:59.000+0000 7f9957000640 1 -- 172.21.15.47:0/2724592882 wait complete. 2024-04-20T10:27:59.005 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.2 2024-04-20T10:27:59.010 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.1 2024-04-20T10:27:59.011 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.4 2024-04-20T10:27:59.013 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.5 2024-04-20T10:27:59.021 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-04-20T10:27:59.989 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-20T10:27:59.992 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-20T10:27:59.996 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-20T10:28:00.006 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-20T10:28:00.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-20T10:28:00.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-20T10:28:00.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-20T10:28:00.022 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-20T10:28:00.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.260+0000 7f5653c00640 1 Processor -- start 2024-04-20T10:28:00.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.260+0000 7f5653c00640 1 -- start start 2024-04-20T10:28:00.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.264+0000 7f5653c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f564c0ce5f0 0x7f564c0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.264+0000 7f5653c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f564c0b31e0 con 0x7f564c15a750 2024-04-20T10:28:00.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.264+0000 7f5653c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f564c15f060 con 0x7f564c0adeb0 2024-04-20T10:28:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.265+0000 7f5653c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f564c0ad5e0 con 0x7f564c0ce5f0 2024-04-20T10:28:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.265+0000 7f564be00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f564c0adeb0 0x7f564c0cdee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47228/0 (socket says 172.21.15.47:47228) 2024-04-20T10:28:00.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.265+0000 7f564be00640 1 -- 172.21.15.47:0/3756017558 learned_addr learned my addr 172.21.15.47:0/3756017558 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:00.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.266+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3528806433 0 0) 0x7f564c0b31e0 con 0x7f564c15a750 2024-04-20T10:28:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.267+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5634003520 con 0x7f564c15a750 2024-04-20T10:28:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.267+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 432297401 0 0) 0x7f564c15f060 con 0x7f564c0adeb0 2024-04-20T10:28:00.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f564c0b31e0 con 0x7f564c0adeb0 2024-04-20T10:28:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2642995401 0 0) 0x7f5634003520 con 0x7f564c15a750 2024-04-20T10:28:00.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f5651c00640 1 --2- 172.21.15.47:0/3756017558 >> v2:172.21.15.175:3300/0 conn(0x7f564c0ce5f0 0x7f564c0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 >> v2:172.21.15.175:3300/0 conn(0x7f564c0ce5f0 msgr2=0x7f564c0acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 --2- 172.21.15.47:0/3756017558 >> v2:172.21.15.175:3300/0 conn(0x7f564c0ce5f0 0x7f564c0acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 >> v1:172.21.15.175:6789/0 conn(0x7f564c0adeb0 legacy=0x7f564c0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c0ad8c0 con 0x7f564c15a750 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f5640001670 con 0x7f564c15a750 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 >> v1:172.21.15.47:6789/0 conn(0x7f564c15a750 legacy=0x7f564c15e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634003b40 0x7f5634003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56340051c0 0x7f563400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f564c0ad5e0 con 0x7f5634004800 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f563400c0f0 con 0x7f5634003b40 2024-04-20T10:28:00.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.268+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f563400c2f0 con 0x7f56340051c0 2024-04-20T10:28:00.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.270+0000 7f564be00640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.270+0000 7f5651200640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634003b40 0x7f5634003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.271+0000 7f5651c00640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56340051c0 0x7f563400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.271+0000 7f564be00640 1 -- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56340051c0 msgr2=0x7f563400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.271+0000 7f564be00640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56340051c0 0x7f563400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.271+0000 7f564be00640 1 -- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634003b40 msgr2=0x7f5634003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.271+0000 7f564be00640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634003b40 0x7f5634003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.271+0000 7f564be00640 1 -- 172.21.15.47:0/3756017558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c0ad8c0 con 0x7f5634004800 2024-04-20T10:28:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.273+0000 7f564be00640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5634004800 0x7f5634004c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5638007fd0 tx=0x7f5634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90927e6241074fd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.273+0000 7f5651c00640 1 -- 172.21.15.47:0/3756017558 reap_dead start 2024-04-20T10:28:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.274+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56380015b0 con 0x7f5634004800 2024-04-20T10:28:00.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.275+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5638034030 con 0x7f5634004800 2024-04-20T10:28:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.277+0000 7f564b400640 1 -- 172.21.15.47:0/3756017558 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638004ce0 con 0x7f5634004800 2024-04-20T10:28:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.277+0000 7f5653c00640 1 -- 172.21.15.47:0/3756017558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5634004800 msgr2=0x7f5634004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.278+0000 7f5653c00640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5634004800 0x7f5634004c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5638007fd0 tx=0x7f5634005830 comp rx=0 tx=0).stop 2024-04-20T10:28:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.279+0000 7f5653c00640 1 -- 172.21.15.47:0/3756017558 shutdown_connections 2024-04-20T10:28:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.279+0000 7f5653c00640 1 --2- 172.21.15.47:0/3756017558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5634004800 0x7f5634004c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.279+0000 7f5653c00640 1 -- 172.21.15.47:0/3756017558 >> 172.21.15.47:0/3756017558 conn(0x7f564c0cffe0 msgr2=0x7f564c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.279+0000 7f5653c00640 1 -- 172.21.15.47:0/3756017558 shutdown_connections 2024-04-20T10:28:00.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.279+0000 7f5653c00640 1 -- 172.21.15.47:0/3756017558 wait complete. 2024-04-20T10:28:00.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.283+0000 7f5653c00640 1 Processor -- start 2024-04-20T10:28:00.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.284+0000 7f5653c00640 1 -- start start 2024-04-20T10:28:00.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.286+0000 7f5653c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634004800 0x7f564c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.286+0000 7f5653c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f564c150e20 0x7f564c15c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.287+0000 7f5653c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f564c0b46f0 0x7f564c0b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.287+0000 7f5651200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634004800 0x7f564c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.287+0000 7f5651200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634004800 0x7f564c15ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40972/0 (socket says 172.21.15.47:40972) 2024-04-20T10:28:00.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.287+0000 7f5651200640 1 -- 172.21.15.47:0/2277128558 learned_addr learned my addr 172.21.15.47:0/2277128558 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.288+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f564c15e430 con 0x7f564c0b46f0 2024-04-20T10:28:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.288+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f564c15e150 con 0x7f5634004800 2024-04-20T10:28:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.288+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f564c0d0b00 con 0x7f564c150e20 2024-04-20T10:28:00.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.288+0000 7f564be00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f564c150e20 0x7f564c15c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.289+0000 7f5651c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f564c0b46f0 0x7f564c0b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.290+0000 7f5651200640 1 -- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f564c150e20 msgr2=0x7f564c15c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.290+0000 7f5651200640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f564c150e20 0x7f564c15c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.290+0000 7f5651200640 1 -- 172.21.15.47:0/2277128558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f564c0b46f0 msgr2=0x7f564c0b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.290+0000 7f5651200640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f564c0b46f0 0x7f564c0b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.290+0000 7f5651200640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f564c15cac0 con 0x7f5634004800 2024-04-20T10:28:00.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.290+0000 7f5651c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f564c0b46f0 0x7f564c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:00.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.291+0000 7f5651200640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634004800 0x7f564c15ba90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f564000f980 tx=0x7f564000fe40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.292+0000 7f5648c00640 1 -- 172.21.15.47:0/2277128558 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56400052e0 con 0x7f5634004800 2024-04-20T10:28:00.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.292+0000 7f5648c00640 1 -- 172.21.15.47:0/2277128558 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5640018070 con 0x7f5634004800 2024-04-20T10:28:00.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.292+0000 7f5648c00640 1 -- 172.21.15.47:0/2277128558 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f564000da30 con 0x7f5634004800 2024-04-20T10:28:00.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.292+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f564c15cd60 con 0x7f5634004800 2024-04-20T10:28:00.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.292+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f564c15d320 con 0x7f5634004800 2024-04-20T10:28:00.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.294+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5628005290 con 0x7f5634004800 2024-04-20T10:28:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.298+0000 7f5648c00640 1 -- 172.21.15.47:0/2277128558 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f564000dbd0 con 0x7f5634004800 2024-04-20T10:28:00.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.299+0000 7f5648c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f562c06c8f0 0x7f562c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.300+0000 7f5648c00640 1 -- 172.21.15.47:0/2277128558 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f564008caf0 con 0x7f5634004800 2024-04-20T10:28:00.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.300+0000 7f564be00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f562c06c8f0 0x7f562c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.302+0000 7f564be00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f562c06c8f0 0x7f562c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5638038000 tx=0x7f563803d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.307+0000 7f92ce400640 1 Processor -- start 2024-04-20T10:28:00.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.307+0000 7f92ce400640 1 -- start start 2024-04-20T10:28:00.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.309+0000 7f5648c00640 1 -- 172.21.15.47:0/2277128558 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f564005abc0 con 0x7f5634004800 2024-04-20T10:28:00.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.309+0000 7f92ce400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92c00e30b0 0x7f92c0057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.309+0000 7f92ce400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c003e830 0x7f92c0053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.310+0000 7f92ce400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92c00582c0 con 0x7f92c0054120 2024-04-20T10:28:00.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.310+0000 7f92ce400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f92c0058560 con 0x7f92c00e30b0 2024-04-20T10:28:00.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.310+0000 7f92ce400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f92c0058840 con 0x7f92c003e830 2024-04-20T10:28:00.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.310+0000 7f92c7400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92c00e30b0 0x7f92c0057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.310+0000 7f92c7400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f92c00e30b0 0x7f92c0057d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40980/0 (socket says 172.21.15.47:40980) 2024-04-20T10:28:00.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.310+0000 7f92c7400640 1 -- 172.21.15.47:0/2556327830 learned_addr learned my addr 172.21.15.47:0/2556327830 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.311+0000 7f92c6a00640 1 --2- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c003e830 0x7f92c0053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.312+0000 7f92c6000640 1 -- 172.21.15.47:0/2556327830 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2125838205 0 0) 0x7f92c00582c0 con 0x7f92c0054120 2024-04-20T10:28:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.312+0000 7f92c7400640 1 -- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c003e830 msgr2=0x7f92c0053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.312+0000 7f92c7400640 1 --2- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c003e830 0x7f92c0053be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.312+0000 7f92c7400640 1 -- 172.21.15.47:0/2556327830 >> v1:172.21.15.47:6789/0 conn(0x7f92c0054120 legacy=0x7f92c00e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.312+0000 7f92c7400640 1 -- 172.21.15.47:0/2556327830 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c0058b20 con 0x7f92c00e30b0 2024-04-20T10:28:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.313+0000 7f92c6a00640 1 --2- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c003e830 0x7f92c0053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:00.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.314+0000 7f51ef200640 1 Processor -- start 2024-04-20T10:28:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.315+0000 7f92c7400640 1 --2- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c00e30b0 0x7f92c0057d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009870 tx=0x7f92bc02ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1db24e5d53a4dfa4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.315+0000 7f51ef200640 1 -- start start 2024-04-20T10:28:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.316+0000 7f92c6000640 1 -- 172.21.15.47:0/2556327830 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc02f830 con 0x7f92c00e30b0 2024-04-20T10:28:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.317+0000 7f92c6000640 1 -- 172.21.15.47:0/2556327830 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92bc02f9d0 con 0x7f92c00e30b0 2024-04-20T10:28:00.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.317+0000 7f92c6000640 1 -- 172.21.15.47:0/2556327830 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92bc02fda0 con 0x7f92c00e30b0 2024-04-20T10:28:00.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.317+0000 7f51ef200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f51f00b3330 0x7f51f00d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.317+0000 7f51ef200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f006c190 con 0x7f51f006cde0 2024-04-20T10:28:00.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.317+0000 7f51ef200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f51f006c470 con 0x7f51f0161d50 2024-04-20T10:28:00.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 -- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c00e30b0 msgr2=0x7f92c0057d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 --2- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c00e30b0 0x7f92c0057d80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009870 tx=0x7f92bc02ba10 comp rx=0 tx=0).stop 2024-04-20T10:28:00.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.317+0000 7f51ef200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f51f0162130 con 0x7f51f00b3330 2024-04-20T10:28:00.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 -- 172.21.15.47:0/2556327830 shutdown_connections 2024-04-20T10:28:00.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 --2- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c003e830 0x7f92c0053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 --2- 172.21.15.47:0/2556327830 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c00e30b0 0x7f92c0057d80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 -- 172.21.15.47:0/2556327830 >> 172.21.15.47:0/2556327830 conn(0x7f92c00559d0 msgr2=0x7f92c0055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 -- 172.21.15.47:0/2556327830 shutdown_connections 2024-04-20T10:28:00.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.318+0000 7f92ce400640 1 -- 172.21.15.47:0/2556327830 wait complete. 2024-04-20T10:28:00.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.319+0000 7f51ede00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7f51f0161d50 0x7f51f006c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47230/0 (socket says 172.21.15.47:47230) 2024-04-20T10:28:00.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.319+0000 7f51ede00640 1 -- 172.21.15.47:0/3952484142 learned_addr learned my addr 172.21.15.47:0/3952484142 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:00.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.321+0000 7f92ce400640 1 Processor -- start 2024-04-20T10:28:00.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.320+0000 7f51ee800640 1 --2- 172.21.15.47:0/3952484142 >> v2:172.21.15.47:3300/0 conn(0x7f51f00b3330 0x7f51f00d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.321+0000 7f92ce400640 1 -- start start 2024-04-20T10:28:00.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.321+0000 7f51eca00640 1 -- 172.21.15.47:0/3952484142 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2468367768 0 0) 0x7f51f006c470 con 0x7f51f0161d50 2024-04-20T10:28:00.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.322+0000 7f51eca00640 1 -- 172.21.15.47:0/3952484142 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51d4003520 con 0x7f51f0161d50 2024-04-20T10:28:00.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.322+0000 7f51eca00640 1 -- 172.21.15.47:0/3952484142 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1894912548 0 0) 0x7f51f006c190 con 0x7f51f006cde0 2024-04-20T10:28:00.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.322+0000 7f51eca00640 1 -- 172.21.15.47:0/3952484142 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f51f006c470 con 0x7f51f006cde0 2024-04-20T10:28:00.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.322+0000 7f51ee800640 1 -- 172.21.15.47:0/3952484142 >> v1:172.21.15.175:6789/0 conn(0x7f51f0161d50 legacy=0x7f51f006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.322+0000 7f51ee800640 1 -- 172.21.15.47:0/3952484142 >> v1:172.21.15.47:6789/0 conn(0x7f51f006cde0 legacy=0x7f51f00b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.322+0000 7f51ee800640 1 -- 172.21.15.47:0/3952484142 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f00d2e00 con 0x7f51f00b3330 2024-04-20T10:28:00.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.324+0000 7f92ce400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c003e830 0x7f92c00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.324+0000 7f92ce400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0054120 0x7f92c00d8400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92c7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c003e830 0x7f92c00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f51ee800640 1 --2- 172.21.15.47:0/3952484142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f00b3330 0x7f51f00d38e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009d90 tx=0x7f51e802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9b0c43b0185a54e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92c7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c003e830 0x7f92c00cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40984/0 (socket says 172.21.15.47:40984) 2024-04-20T10:28:00.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92c7400640 1 -- 172.21.15.47:0/669202265 learned_addr learned my addr 172.21.15.47:0/669202265 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92ce400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c00e30b0 0x7f92c00cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92c00e3490 con 0x7f92c0054120 2024-04-20T10:28:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92c003bfb0 con 0x7f92c003e830 2024-04-20T10:28:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f92c00078d0 con 0x7f92c00e30b0 2024-04-20T10:28:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.325+0000 7f92c6a00640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0054120 0x7f92c00d8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.326+0000 7f92c7e00640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c00e30b0 0x7f92c00cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.326+0000 7f92c6a00640 1 -- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c00e30b0 msgr2=0x7f92c00cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.326+0000 7f92c6a00640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c00e30b0 0x7f92c00cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.326+0000 7f92c6a00640 1 -- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c003e830 msgr2=0x7f92c00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.326+0000 7f92c6a00640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c003e830 0x7f92c00cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.326+0000 7f92c6a00640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92c00cbf80 con 0x7f92c0054120 2024-04-20T10:28:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.326+0000 7f92c7400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c003e830 0x7f92c00cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.327+0000 7f92c6a00640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0054120 0x7f92c00d8400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f92b4006fd0 tx=0x7f92b400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.327+0000 7f92c7e00640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c00e30b0 0x7f92c00cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:00.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.327+0000 7f51eca00640 1 -- 172.21.15.47:0/3952484142 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8039070 con 0x7f51f00b3330 2024-04-20T10:28:00.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.327+0000 7f51eca00640 1 -- 172.21.15.47:0/3952484142 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51e802cc60 con 0x7f51f00b3330 2024-04-20T10:28:00.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.327+0000 7f51eca00640 1 -- 172.21.15.47:0/3952484142 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e8034660 con 0x7f51f00b3330 2024-04-20T10:28:00.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.328+0000 7f92bb400640 1 -- 172.21.15.47:0/669202265 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b4004030 con 0x7f92c0054120 2024-04-20T10:28:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.328+0000 7f51ef200640 1 -- 172.21.15.47:0/3952484142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f00b3330 msgr2=0x7f51f00d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.328+0000 7f51ef200640 1 --2- 172.21.15.47:0/3952484142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f00b3330 0x7f51f00d38e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009d90 tx=0x7f51e802beb0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.329+0000 7f51ef200640 1 -- 172.21.15.47:0/3952484142 shutdown_connections 2024-04-20T10:28:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.329+0000 7f51ef200640 1 --2- 172.21.15.47:0/3952484142 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f00b3330 0x7f51f00d38e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.329+0000 7f51ef200640 1 -- 172.21.15.47:0/3952484142 >> 172.21.15.47:0/3952484142 conn(0x7f51f00d4450 msgr2=0x7f51f00d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.330+0000 7f51ef200640 1 -- 172.21.15.47:0/3952484142 shutdown_connections 2024-04-20T10:28:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.328+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92c00cc260 con 0x7f92c0054120 2024-04-20T10:28:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.328+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92c00cc820 con 0x7f92c0054120 2024-04-20T10:28:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.328+0000 7f92bb400640 1 -- 172.21.15.47:0/669202265 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f92b40041d0 con 0x7f92c0054120 2024-04-20T10:28:00.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.329+0000 7f92bb400640 1 -- 172.21.15.47:0/669202265 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92b4004030 con 0x7f92c0054120 2024-04-20T10:28:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.330+0000 7f51ef200640 1 -- 172.21.15.47:0/3952484142 wait complete. 2024-04-20T10:28:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.333+0000 7f51ef200640 1 Processor -- start 2024-04-20T10:28:00.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.333+0000 7f51ef200640 1 -- start start 2024-04-20T10:28:00.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.335+0000 7f92bb400640 1 -- 172.21.15.47:0/669202265 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f92b4004380 con 0x7f92c0054120 2024-04-20T10:28:00.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.335+0000 7f51ef200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f006cde0 0x7f51f0072c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.336+0000 7f51ede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f006cde0 0x7f51f0072c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.337+0000 7f51ede00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f006cde0 0x7f51f0072c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56126/0 (socket says 172.21.15.47:56126) 2024-04-20T10:28:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.337+0000 7f51ede00640 1 -- 172.21.15.47:0/2689915758 learned_addr learned my addr 172.21.15.47:0/2689915758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.337+0000 7f92bb400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f92a806caf0 0x7f92a806efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.337+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f00b3330 0x7f51f00c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.337+0000 7f92bb400640 1 -- 172.21.15.47:0/669202265 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f92b408bd80 con 0x7f92c0054120 2024-04-20T10:28:00.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.337+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51f0161d50 0x7f51f00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.338+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51f00b3880 con 0x7f51f006cde0 2024-04-20T10:28:00.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.338+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51f00d33c0 con 0x7f51f00b3330 2024-04-20T10:28:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.338+0000 7f92c7400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f92a806caf0 0x7f92a806efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.338+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51f00b3a00 con 0x7f51f0161d50 2024-04-20T10:28:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.338+0000 7f51ed400640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f00b3330 0x7f51f00c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.338+0000 7f51ee800640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51f0161d50 0x7f51f00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.339+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a4005290 con 0x7f92c0054120 2024-04-20T10:28:00.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.340+0000 7f51ede00640 1 -- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51f0161d50 msgr2=0x7f51f00c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.341+0000 7f51ede00640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51f0161d50 0x7f51f00c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.341+0000 7f51ede00640 1 -- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f00b3330 msgr2=0x7f51f00c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.341+0000 7f51ede00640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f00b3330 0x7f51f00c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.341+0000 7f51ede00640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f00c6c30 con 0x7f51f006cde0 2024-04-20T10:28:00.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.342+0000 7f51ee800640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51f0161d50 0x7f51f00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:28:00.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.342+0000 7f51ede00640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f006cde0 0x7f51f0072c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c440 tx=0x7f51dc00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.340+0000 7f92c7400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f92a806caf0 0x7f92a806efb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0090b0 tx=0x7f92bc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.350+0000 7f92bb400640 1 -- 172.21.15.47:0/669202265 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f92b4059dd0 con 0x7f92c0054120 2024-04-20T10:28:00.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.351+0000 7f51e6000640 1 -- 172.21.15.47:0/2689915758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc017070 con 0x7f51f006cde0 2024-04-20T10:28:00.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.351+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f00c6ee0 con 0x7f51f006cde0 2024-04-20T10:28:00.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.351+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f00c74a0 con 0x7f51f006cde0 2024-04-20T10:28:00.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.352+0000 7f51e6000640 1 -- 172.21.15.47:0/2689915758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51dc005ce0 con 0x7f51f006cde0 2024-04-20T10:28:00.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.352+0000 7f51e6000640 1 -- 172.21.15.47:0/2689915758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc0072b0 con 0x7f51f006cde0 2024-04-20T10:28:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.356+0000 7f51e6000640 1 -- 172.21.15.47:0/2689915758 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f51dc01a070 con 0x7f51f006cde0 2024-04-20T10:28:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.358+0000 7f51e6000640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.358+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51c8005290 con 0x7f51f006cde0 2024-04-20T10:28:00.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.359+0000 7f51e6000640 1 -- 172.21.15.47:0/2689915758 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f51dc08b1f0 con 0x7f51f006cde0 2024-04-20T10:28:00.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.359+0000 7f51ed400640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.364+0000 7f51ed400640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f51d006c8f0 0x7f51d006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f51f01f6660 tx=0x7f51e0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.371+0000 7f51e6000640 1 -- 172.21.15.47:0/2689915758 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f51dc00f260 con 0x7f51f006cde0 2024-04-20T10:28:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.377+0000 7f7ba3000640 1 Processor -- start 2024-04-20T10:28:00.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.377+0000 7f7ba3000640 1 -- start start 2024-04-20T10:28:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.379+0000 7f7ba3000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b9c0d0b00 0x7f7b9c0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.379+0000 7f7ba3000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b9c15e950 0x7f7b9c0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.380+0000 7f7ba3000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7b9c0ad5e0 con 0x7f7b9c0adeb0 2024-04-20T10:28:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.380+0000 7f7ba1000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7b9c15e950 0x7f7b9c0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.380+0000 7f7b9be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b9c0d0b00 0x7f7b9c0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.381+0000 7f7b9be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7b9c0d0b00 0x7f7b9c0acf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:41002/0 (socket says 172.21.15.47:41002) 2024-04-20T10:28:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.381+0000 7f7b9be00640 1 -- 172.21.15.47:0/1464480622 learned_addr learned my addr 172.21.15.47:0/1464480622 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.381+0000 7f7ba3000640 1 -- 172.21.15.47:0/1464480622 --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7b9c0ad8c0 con 0x7f7b9c15e950 2024-04-20T10:28:00.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.381+0000 7f7ba3000640 1 -- 172.21.15.47:0/1464480622 --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7b9c0adba0 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.382+0000 7f7ba1000640 1 -- 172.21.15.47:0/1464480622 >> v2:172.21.15.175:3300/0 conn(0x7f7b9c0d0b00 msgr2=0x7f7b9c0acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.382+0000 7f7ba1000640 1 --2- 172.21.15.47:0/1464480622 >> v2:172.21.15.175:3300/0 conn(0x7f7b9c0d0b00 0x7f7b9c0acf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.382+0000 7f7ba1000640 1 -- 172.21.15.47:0/1464480622 >> v1:172.21.15.47:6789/0 conn(0x7f7b9c0adeb0 legacy=0x7f7b9c15a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.382+0000 7f7ba1000640 1 -- 172.21.15.47:0/1464480622 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c15e150 con 0x7f7b9c15e950 2024-04-20T10:28:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.384+0000 7f87fee00640 1 Processor -- start 2024-04-20T10:28:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.384+0000 7f87fee00640 1 -- start start 2024-04-20T10:28:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.385+0000 7f7ba1000640 1 --2- 172.21.15.47:0/1464480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 0x7f7b9c0cdee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c071690 tx=0x7f7b9402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7a8f3246b81344 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.386+0000 7f7b9aa00640 1 -- 172.21.15.47:0/1464480622 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94039070 con 0x7f7b9c15e950 2024-04-20T10:28:00.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.386+0000 7f7b9aa00640 1 -- 172.21.15.47:0/1464480622 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b9402cac0 con 0x7f7b9c15e950 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.386+0000 7f7b9aa00640 1 -- 172.21.15.47:0/1464480622 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94039070 con 0x7f7b9c15e950 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.386+0000 7f87fee00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87f0053c70 0x7f87f0038080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.386+0000 7f87fee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0033ef0 0x7f87f004fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.386+0000 7f87fee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87f00544c0 con 0x7f87f000c0a0 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f87fee00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f87f00541e0 con 0x7f87f0053c70 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f87fee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87f0005f30 con 0x7f87f0033ef0 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f87fd000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f87f0053c70 0x7f87f0038080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f87fda00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f87f000c0a0 0x7f87f00df220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33766/0 (socket says 172.21.15.47:33766) 2024-04-20T10:28:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f87fda00640 1 -- 172.21.15.47:0/3046375391 learned_addr learned my addr 172.21.15.47:0/3046375391 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.388+0000 7f87fe400640 1 --2- 172.21.15.47:0/3046375391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0033ef0 0x7f87f004fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.388+0000 7f87f7e00640 1 -- 172.21.15.47:0/3046375391 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3860925668 0 0) 0x7f87f00544c0 con 0x7f87f000c0a0 2024-04-20T10:28:00.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.388+0000 7f87f7e00640 1 -- 172.21.15.47:0/3046375391 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87e0003520 con 0x7f87f000c0a0 2024-04-20T10:28:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f7ba3000640 1 -- 172.21.15.47:0/1464480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 msgr2=0x7f7b9c0cdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1464480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 0x7f7b9c0cdee0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c071690 tx=0x7f7b9402bd90 comp rx=0 tx=0).stop 2024-04-20T10:28:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f7ba3000640 1 -- 172.21.15.47:0/1464480622 shutdown_connections 2024-04-20T10:28:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1464480622 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 0x7f7b9c0cdee0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1464480622 >> v2:172.21.15.175:3300/0 conn(0x7f7b9c0d0b00 0x7f7b9c0acf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.387+0000 7f7ba3000640 1 -- 172.21.15.47:0/1464480622 >> 172.21.15.47:0/1464480622 conn(0x7f7b9c0cffe0 msgr2=0x7f7b9c0d03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.388+0000 7f7ba3000640 1 -- 172.21.15.47:0/1464480622 shutdown_connections 2024-04-20T10:28:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.389+0000 7fba20c00640 1 Processor -- start 2024-04-20T10:28:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.389+0000 7fba20c00640 1 -- start start 2024-04-20T10:28:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.389+0000 7f87fd000640 1 -- 172.21.15.47:0/3046375391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0033ef0 msgr2=0x7f87f004fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.389+0000 7f87fd000640 1 --2- 172.21.15.47:0/3046375391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0033ef0 0x7f87f004fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.389+0000 7f87fd000640 1 -- 172.21.15.47:0/3046375391 >> v1:172.21.15.47:6789/0 conn(0x7f87f000c0a0 legacy=0x7f87f00df220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.389+0000 7f87fd000640 1 -- 172.21.15.47:0/3046375391 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f0006210 con 0x7f87f0053c70 2024-04-20T10:28:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.389+0000 7f7ba3000640 1 -- 172.21.15.47:0/1464480622 wait complete. 2024-04-20T10:28:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fa83ac00640 1 Processor -- start 2024-04-20T10:28:00.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fa83ac00640 1 -- start start 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fba20c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba140e30b0 0x7fba14057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fba20c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba1403e830 0x7fba14053be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fba20c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fba14054120 0x7fba140e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fba20c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fba140582c0 con 0x7fba140e30b0 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fba20c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fba14058560 con 0x7fba14054120 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.390+0000 7fba20c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba14058840 con 0x7fba1403e830 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.391+0000 7fba19e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba140e30b0 0x7fba14057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.391+0000 7fba19e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fba140e30b0 0x7fba14057d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56144/0 (socket says 172.21.15.47:56144) 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.391+0000 7fba19e00640 1 -- 172.21.15.47:0/3708568524 learned_addr learned my addr 172.21.15.47:0/3708568524 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.391+0000 7f87fd000640 1 --2- 172.21.15.47:0/3046375391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0038080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097a0 tx=0x7f87ec02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6831cfb89bb327f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.391+0000 7fba19400640 1 --2- 172.21.15.47:0/3708568524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba1403e830 0x7fba14053be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.392+0000 7f7ba3000640 1 Processor -- start 2024-04-20T10:28:00.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7f7ba3000640 1 -- start start 2024-04-20T10:28:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.392+0000 7fba1a800640 1 --2- 172.21.15.47:0/3708568524 >> v2:172.21.15.175:3300/0 conn(0x7fba14054120 0x7fba140e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.392+0000 7fba19e00640 1 -- 172.21.15.47:0/3708568524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba1403e830 msgr2=0x7fba14053be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.392+0000 7fba19e00640 1 --2- 172.21.15.47:0/3708568524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba1403e830 0x7fba14053be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.392+0000 7fba19e00640 1 -- 172.21.15.47:0/3708568524 >> v2:172.21.15.175:3300/0 conn(0x7fba14054120 msgr2=0x7fba140e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.392+0000 7fba19e00640 1 --2- 172.21.15.47:0/3708568524 >> v2:172.21.15.175:3300/0 conn(0x7fba14054120 0x7fba140e6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7fa83ac00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa83415a7d0 0x7fa8340ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.392+0000 7fba19e00640 1 -- 172.21.15.47:0/3708568524 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14058b20 con 0x7fba140e30b0 2024-04-20T10:28:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7fa83ac00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8340ad810 con 0x7fa83406de00 2024-04-20T10:28:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7fa83ac00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8340adaf0 con 0x7fa83406bdd0 2024-04-20T10:28:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7fa83ac00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa8340addd0 con 0x7fa83415a7d0 2024-04-20T10:28:00.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7fa833400640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fa83406bdd0 0x7fa83415abb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:47242/0 (socket says 172.21.15.47:47242) 2024-04-20T10:28:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7fa833400640 1 -- 172.21.15.47:0/1795354333 learned_addr learned my addr 172.21.15.47:0/1795354333 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f7ba3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c0adeb0 0x7f7b9c0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f7ba3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0d0b00 0x7f7b9c15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f7ba3000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 0x7f7b9c09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f7ba3000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c15ef60 con 0x7f7b9c15e950 2024-04-20T10:28:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f7ba3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c15ede0 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f7ba3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9c15f0e0 con 0x7f7b9c0adeb0 2024-04-20T10:28:00.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7fa833e00640 1 --2- 172.21.15.47:0/1795354333 >> v2:172.21.15.47:3300/0 conn(0x7fa83415a7d0 0x7fa8340ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.393+0000 7f87f7e00640 1 -- 172.21.15.47:0/3046375391 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec031780 con 0x7f87f0053c70 2024-04-20T10:28:00.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f87f7e00640 1 -- 172.21.15.47:0/3046375391 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87ec037070 con 0x7f87f0053c70 2024-04-20T10:28:00.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.394+0000 7f87f7e00640 1 -- 172.21.15.47:0/3046375391 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87ec031b20 con 0x7f87f0053c70 2024-04-20T10:28:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7f7b9b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0d0b00 0x7f7b9c15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7f7b9b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0d0b00 0x7f7b9c15d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:41022/0 (socket says 172.21.15.47:41022) 2024-04-20T10:28:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7f7b9b400640 1 -- 172.21.15.47:0/1457367110 learned_addr learned my addr 172.21.15.47:0/1457367110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7f7ba1000640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 0x7f7b9c09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7f7b9be00640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c0adeb0 0x7f7b9c0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7f87fee00640 1 -- 172.21.15.47:0/3046375391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 msgr2=0x7f87f0038080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7f87fee00640 1 --2- 172.21.15.47:0/3046375391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0038080 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f87ec0097a0 tx=0x7f87ec02b820 comp rx=0 tx=0).stop 2024-04-20T10:28:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.395+0000 7fba19e00640 1 --2- 172.21.15.47:0/3708568524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba140e30b0 0x7fba14057d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba140568f0 tx=0x7fba0c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=771388c7eaa5a433 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f7b9b400640 1 -- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c0adeb0 msgr2=0x7f7b9c0b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f7b9b400640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c0adeb0 0x7f7b9c0b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f7b9b400640 1 -- 172.21.15.47:0/1457367110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 msgr2=0x7f7b9c09a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f7b9b400640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 0x7f7b9c09a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f7b9b400640 1 -- 172.21.15.47:0/1457367110 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c09c520 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f87fee00640 1 -- 172.21.15.47:0/3046375391 shutdown_connections 2024-04-20T10:28:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f87fee00640 1 --2- 172.21.15.47:0/3046375391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0033ef0 0x7f87f004fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f87fee00640 1 --2- 172.21.15.47:0/3046375391 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0038080 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f87fee00640 1 -- 172.21.15.47:0/3046375391 >> 172.21.15.47:0/3046375391 conn(0x7f87f0009fb0 msgr2=0x7f87f000a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7f7b9b400640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0d0b00 0x7f7b9c15d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c780 tx=0x7f7b8800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7fa833e00640 1 -- 172.21.15.47:0/1795354333 >> v1:172.21.15.175:6789/0 conn(0x7fa83406bdd0 legacy=0x7fa83415abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7fa833e00640 1 -- 172.21.15.47:0/1795354333 >> v1:172.21.15.47:6789/0 conn(0x7fa83406de00 legacy=0x7fa8340ae8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.396+0000 7fa833e00640 1 -- 172.21.15.47:0/1795354333 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8340ae0b0 con 0x7fa83415a7d0 2024-04-20T10:28:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7fba18a00640 1 -- 172.21.15.47:0/3708568524 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c034070 con 0x7fba140e30b0 2024-04-20T10:28:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7f87fee00640 1 -- 172.21.15.47:0/3046375391 shutdown_connections 2024-04-20T10:28:00.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7fba18a00640 1 -- 172.21.15.47:0/3708568524 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba0c030860 con 0x7fba140e30b0 2024-04-20T10:28:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7fba18a00640 1 -- 172.21.15.47:0/3708568524 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0c030c30 con 0x7fba140e30b0 2024-04-20T10:28:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7f87fee00640 1 -- 172.21.15.47:0/3046375391 wait complete. 2024-04-20T10:28:00.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7f3424c00640 1 Processor -- start 2024-04-20T10:28:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7f7b93e00640 1 -- 172.21.15.47:0/1457367110 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b88017070 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7f7b93e00640 1 -- 172.21.15.47:0/1457367110 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b88005290 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7f7b93e00640 1 -- 172.21.15.47:0/1457367110 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b8800a880 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.397+0000 7f3424c00640 1 -- start start 2024-04-20T10:28:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.398+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c09c800 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.398+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c09cdc0 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fba20c00640 1 -- 172.21.15.47:0/3708568524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba140e30b0 msgr2=0x7fba14057d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fba20c00640 1 --2- 172.21.15.47:0/3708568524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba140e30b0 0x7fba14057d80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fba140568f0 tx=0x7fba0c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fa833e00640 1 --2- 172.21.15.47:0/1795354333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83415a7d0 0x7fa8340ad210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8240097a0 tx=0x7fa82402db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4657ce7d440cf5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fa832a00640 1 -- 172.21.15.47:0/1795354333 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82403b070 con 0x7fa83415a7d0 2024-04-20T10:28:00.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fa832a00640 1 -- 172.21.15.47:0/1795354333 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82402e930 con 0x7fa83415a7d0 2024-04-20T10:28:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fa832a00640 1 -- 172.21.15.47:0/1795354333 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82403b070 con 0x7fa83415a7d0 2024-04-20T10:28:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7f87fee00640 1 Processor -- start 2024-04-20T10:28:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fba20c00640 1 -- 172.21.15.47:0/3708568524 shutdown_connections 2024-04-20T10:28:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fba20c00640 1 --2- 172.21.15.47:0/3708568524 >> v2:172.21.15.175:3300/0 conn(0x7fba14054120 0x7fba140e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fba20c00640 1 --2- 172.21.15.47:0/3708568524 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba1403e830 0x7fba14053be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fba20c00640 1 --2- 172.21.15.47:0/3708568524 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba140e30b0 0x7fba14057d80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7fba20c00640 1 -- 172.21.15.47:0/3708568524 >> 172.21.15.47:0/3708568524 conn(0x7fba140559d0 msgr2=0x7fba14055de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.400+0000 7fba20c00640 1 -- 172.21.15.47:0/3708568524 shutdown_connections 2024-04-20T10:28:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.399+0000 7f3424c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3420156b60 0x7f34200cff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.400+0000 7f3424c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f342006b900 0x7f342006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.400+0000 7f3424c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f342013f830 con 0x7f342015ca30 2024-04-20T10:28:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.400+0000 7f3424c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3420160080 con 0x7f342006b900 2024-04-20T10:28:00.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.400+0000 7f3424c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3420160360 con 0x7f3420156b60 2024-04-20T10:28:00.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.400+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c075b30 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341de00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3420156b60 0x7f34200cff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7fba20c00640 1 -- 172.21.15.47:0/3708568524 wait complete. 2024-04-20T10:28:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341de00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3420156b60 0x7f34200cff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:41036/0 (socket says 172.21.15.47:41036) 2024-04-20T10:28:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341de00640 1 -- 172.21.15.47:0/3066888953 learned_addr learned my addr 172.21.15.47:0/3066888953 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341e800640 1 --2- 172.21.15.47:0/3066888953 >> v2:172.21.15.47:3300/0 conn(0x7f342006b900 0x7f342006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341de00640 1 -- 172.21.15.47:0/3066888953 >> v2:172.21.15.47:3300/0 conn(0x7f342006b900 msgr2=0x7f342006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7fa83ac00640 1 -- 172.21.15.47:0/1795354333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83415a7d0 msgr2=0x7fa8340ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341de00640 1 --2- 172.21.15.47:0/3066888953 >> v2:172.21.15.47:3300/0 conn(0x7f342006b900 0x7f342006bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341de00640 1 -- 172.21.15.47:0/3066888953 >> v1:172.21.15.47:6789/0 conn(0x7f342015ca30 legacy=0x7f34201607f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.401+0000 7f341de00640 1 -- 172.21.15.47:0/3066888953 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3420156f40 con 0x7f3420156b60 2024-04-20T10:28:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7fa83ac00640 1 --2- 172.21.15.47:0/1795354333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83415a7d0 0x7fa8340ad210 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa8240097a0 tx=0x7fa82402db80 comp rx=0 tx=0).stop 2024-04-20T10:28:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7fa83ac00640 1 -- 172.21.15.47:0/1795354333 shutdown_connections 2024-04-20T10:28:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7fa83ac00640 1 --2- 172.21.15.47:0/1795354333 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83415a7d0 0x7fa8340ad210 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7fa83ac00640 1 -- 172.21.15.47:0/1795354333 >> 172.21.15.47:0/1795354333 conn(0x7fa834072090 msgr2=0x7fa8340b4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7f7b93e00640 1 -- 172.21.15.47:0/1457367110 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7b8801a070 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.400+0000 7f87fee00640 1 -- start start 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7f87fee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f000c0a0 0x7f87f004aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7f87fee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f0033ef0 0x7f87f004c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7f87fee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0034d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7f87fee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0037a90 con 0x7f87f0053c70 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7f87fee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87f0037910 con 0x7f87f0033ef0 2024-04-20T10:28:00.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.402+0000 7f87fee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87f004ffd0 con 0x7f87f000c0a0 2024-04-20T10:28:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.403+0000 7f87fe400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0034d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.403+0000 7f87fe400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0034d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56176/0 (socket says 172.21.15.47:56176) 2024-04-20T10:28:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.403+0000 7f87fe400640 1 -- 172.21.15.47:0/2861231371 learned_addr learned my addr 172.21.15.47:0/2861231371 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.403+0000 7f341e800640 1 --2- 172.21.15.47:0/3066888953 >> v2:172.21.15.47:3300/0 conn(0x7f342006b900 0x7f342006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:00.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.403+0000 7fba20c00640 1 Processor -- start 2024-04-20T10:28:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7fba20c00640 1 -- start start 2024-04-20T10:28:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.403+0000 7f341de00640 1 --2- 172.21.15.47:0/3066888953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3420156b60 0x7f34200cff70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f34200cd100 tx=0x7f341002b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=851a815b89dd8b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.403+0000 7f87fda00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f000c0a0 0x7f87f004aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f87fe400640 1 -- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f000c0a0 msgr2=0x7f87f004aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f87fe400640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f000c0a0 0x7f87f004aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f87fe400640 1 -- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f0033ef0 msgr2=0x7f87f004c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f87fe400640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f0033ef0 0x7f87f004c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f87fe400640 1 -- 172.21.15.47:0/2861231371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f0046ac0 con 0x7f87f0053c70 2024-04-20T10:28:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f7b93e00640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7b7c06c940 0x7f7b7c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f7b93e00640 1 -- 172.21.15.47:0/1457367110 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f7b8808b350 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.404+0000 7f7b9be00640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7b7c06c940 0x7f7b7c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.405+0000 7fa83ac00640 1 -- 172.21.15.47:0/1795354333 shutdown_connections 2024-04-20T10:28:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.405+0000 7f87fe400640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0034d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f87f8051240 tx=0x7f87f8069460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.405+0000 7fa83ac00640 1 -- 172.21.15.47:0/1795354333 wait complete. 2024-04-20T10:28:00.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.405+0000 7f87f5600640 1 -- 172.21.15.47:0/2861231371 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8098070 con 0x7f87f0053c70 2024-04-20T10:28:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.405+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87f0046da0 con 0x7f87f0053c70 2024-04-20T10:28:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.405+0000 7f87f5600640 1 -- 172.21.15.47:0/2861231371 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f8068bb0 con 0x7f87f0053c70 2024-04-20T10:28:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.405+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87f0047360 con 0x7f87f0053c70 2024-04-20T10:28:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.406+0000 7fba20c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1403e830 0x7fba1404e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.406+0000 7fba20c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba14054120 0x7fba14051700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.406+0000 7fba20c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba140e30b0 0x7fba140e1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7f7b9be00640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7b7c06c940 0x7f7b7c06ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b10f0 tx=0x7f7b8c009240 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7f87f5600640 1 -- 172.21.15.47:0/2861231371 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f8074dd0 con 0x7f87f0053c70 2024-04-20T10:28:00.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fba20c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fba140e3490 con 0x7fba1403e830 2024-04-20T10:28:00.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87f0053840 con 0x7f87f0053c70 2024-04-20T10:28:00.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fba20c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fba1403bfb0 con 0x7fba140e30b0 2024-04-20T10:28:00.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fba20c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fba140e7160 con 0x7fba14054120 2024-04-20T10:28:00.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fba1a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba140e30b0 0x7fba140e1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fba1a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba140e30b0 0x7fba140e1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:41058/0 (socket says 172.21.15.47:41058) 2024-04-20T10:28:00.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fba1a800640 1 -- 172.21.15.47:0/413525583 learned_addr learned my addr 172.21.15.47:0/413525583 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fa83ac00640 1 Processor -- start 2024-04-20T10:28:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fba19400640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba14054120 0x7fba14051700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.407+0000 7fa83ac00640 1 -- start start 2024-04-20T10:28:00.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba1a800640 1 -- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba14054120 msgr2=0x7fba14051700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba1a800640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba14054120 0x7fba14051700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba1a800640 1 -- 172.21.15.47:0/413525583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1403e830 msgr2=0x7fba1404e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba19e00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1403e830 0x7fba1404e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba1a800640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1403e830 0x7fba1404e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba1a800640 1 -- 172.21.15.47:0/413525583 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba14050660 con 0x7fba140e30b0 2024-04-20T10:28:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba19e00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1403e830 0x7fba1404e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:00.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fba1a800640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba140e30b0 0x7fba140e1040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba1c065480 tx=0x7fba1c069460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fa83ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83406bdd0 0x7fa834098fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fa83ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83406de00 0x7fa8340d02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fa83ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83415a7d0 0x7fa83409aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.408+0000 7fa83ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa83406baf0 con 0x7fa83406de00 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7fa83ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa83406b970 con 0x7fa83415a7d0 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7fa83ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8340b3bf0 con 0x7fa83406bdd0 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7fa833400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83406de00 0x7fa8340d02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7fa833400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83406de00 0x7fa8340d02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56192/0 (socket says 172.21.15.47:56192) 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7fa833400640 1 -- 172.21.15.47:0/3617781422 learned_addr learned my addr 172.21.15.47:0/3617781422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7fa833e00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83406bdd0 0x7fa834098fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7f87f5600640 1 -- 172.21.15.47:0/2861231371 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87f8074f70 con 0x7f87f0053c70 2024-04-20T10:28:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.409+0000 7fa838c00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83415a7d0 0x7fa83409aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.410+0000 7fa833400640 1 -- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83406bdd0 msgr2=0x7fa834098fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.410+0000 7fa833400640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83406bdd0 0x7fa834098fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.410+0000 7fa833400640 1 -- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83415a7d0 msgr2=0x7fa83409aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.410+0000 7fa833400640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83415a7d0 0x7fa83409aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.410+0000 7fa833400640 1 -- 172.21.15.47:0/3617781422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83409a2e0 con 0x7fa83406de00 2024-04-20T10:28:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.410+0000 7fa833e00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83406bdd0 0x7fa834098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.410+0000 7fa833400640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83406de00 0x7fa8340d02a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa82000d5e0 tx=0x7fa82000daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.411+0000 7f87f5600640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87d806c8f0 0x7f87d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.411+0000 7f87f5600640 1 -- 172.21.15.47:0/2861231371 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f87f8106b80 con 0x7f87f0053c70 2024-04-20T10:28:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.412+0000 7f87fda00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87d806c8f0 0x7f87d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.413+0000 7f87fda00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87d806c8f0 0x7f87d806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87e4007fd0 tx=0x7f87e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.418+0000 7fba12000640 1 -- 172.21.15.47:0/413525583 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c099070 con 0x7fba140e30b0 2024-04-20T10:28:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.418+0000 7fba12000640 1 -- 172.21.15.47:0/413525583 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba1c074dd0 con 0x7fba140e30b0 2024-04-20T10:28:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.418+0000 7fba12000640 1 -- 172.21.15.47:0/413525583 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c099070 con 0x7fba140e30b0 2024-04-20T10:28:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.418+0000 7f341ca00640 1 -- 172.21.15.47:0/3066888953 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410035020 con 0x7f3420156b60 2024-04-20T10:28:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.418+0000 7f341ca00640 1 -- 172.21.15.47:0/3066888953 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3410031870 con 0x7f3420156b60 2024-04-20T10:28:00.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.418+0000 7f341ca00640 1 -- 172.21.15.47:0/3066888953 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3410031c40 con 0x7f3420156b60 2024-04-20T10:28:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.419+0000 7f7b93e00640 1 -- 172.21.15.47:0/1457367110 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7b880012b0 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.419+0000 7f3424c00640 1 -- 172.21.15.47:0/3066888953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3420156b60 msgr2=0x7f34200cff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.419+0000 7f3424c00640 1 --2- 172.21.15.47:0/3066888953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3420156b60 0x7f34200cff70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f34200cd100 tx=0x7f341002b930 comp rx=0 tx=0).stop 2024-04-20T10:28:00.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.419+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba14050940 con 0x7fba140e30b0 2024-04-20T10:28:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.419+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba14050f00 con 0x7fba140e30b0 2024-04-20T10:28:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.420+0000 7fa82be00640 1 -- 172.21.15.47:0/3617781422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820014070 con 0x7fa83406de00 2024-04-20T10:28:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.420+0000 7fa82be00640 1 -- 172.21.15.47:0/3617781422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa820009040 con 0x7fa83406de00 2024-04-20T10:28:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.420+0000 7fa82be00640 1 -- 172.21.15.47:0/3617781422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8200017a0 con 0x7fa83406de00 2024-04-20T10:28:00.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.420+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83409dcc0 con 0x7fa83406de00 2024-04-20T10:28:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.420+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba140057d0 con 0x7fba140e30b0 2024-04-20T10:28:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.420+0000 7f87f5600640 1 -- 172.21.15.47:0/2861231371 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87f0053840 con 0x7f87f0053c70 2024-04-20T10:28:00.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.421+0000 7f3424c00640 1 -- 172.21.15.47:0/3066888953 shutdown_connections 2024-04-20T10:28:00.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.421+0000 7f3424c00640 1 --2- 172.21.15.47:0/3066888953 >> v2:172.21.15.47:3300/0 conn(0x7f342006b900 0x7f342006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.421+0000 7f3424c00640 1 --2- 172.21.15.47:0/3066888953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3420156b60 0x7f34200cff70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.421+0000 7f3424c00640 1 -- 172.21.15.47:0/3066888953 >> 172.21.15.47:0/3066888953 conn(0x7f34200d0570 msgr2=0x7f34200d0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.420+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83409a5a0 con 0x7fa83406de00 2024-04-20T10:28:00.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.421+0000 7f3424c00640 1 -- 172.21.15.47:0/3066888953 shutdown_connections 2024-04-20T10:28:00.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.421+0000 7f3424c00640 1 -- 172.21.15.47:0/3066888953 wait complete. 2024-04-20T10:28:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.422+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa80c005290 con 0x7fa83406de00 2024-04-20T10:28:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.424+0000 7fba12000640 1 -- 172.21.15.47:0/413525583 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fba1c074f80 con 0x7fba140e30b0 2024-04-20T10:28:00.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.424+0000 7f3424c00640 1 Processor -- start 2024-04-20T10:28:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.426+0000 7fa82be00640 1 -- 172.21.15.47:0/3617781422 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa820017070 con 0x7fa83406de00 2024-04-20T10:28:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.427+0000 7fba12000640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb9fc06caf0 0x7fb9fc06efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.427+0000 7fba12000640 1 -- 172.21.15.47:0/413525583 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fba1c106ed0 con 0x7fba140e30b0 2024-04-20T10:28:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.426+0000 7f3424c00640 1 -- start start 2024-04-20T10:28:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.427+0000 7f3424c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342006b900 0x7f34200b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7f3424c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3420156b60 0x7f342013e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7f3424c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015ca30 0x7f342014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7f3424c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f342015d310 con 0x7f3420156b60 2024-04-20T10:28:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7f3424c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f342006c250 con 0x7f342006b900 2024-04-20T10:28:00.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7f3424c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f342015d5f0 con 0x7f342015ca30 2024-04-20T10:28:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7fa82be00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81406c940 0x7fa81406ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7fa82be00640 1 -- 172.21.15.47:0/3617781422 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fa82008c3c0 con 0x7fa83406de00 2024-04-20T10:28:00.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.428+0000 7fba19e00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb9fc06caf0 0x7fb9fc06efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.429+0000 7f341d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3420156b60 0x7f342013e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.429+0000 7f341d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3420156b60 0x7f342013e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56194/0 (socket says 172.21.15.47:56194) 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.429+0000 7f341d400640 1 -- 172.21.15.47:0/2579422400 learned_addr learned my addr 172.21.15.47:0/2579422400 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.429+0000 7f341e800640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015ca30 0x7f342014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.429+0000 7f341de00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342006b900 0x7f34200b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.430+0000 7f341d400640 1 -- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015ca30 msgr2=0x7f342014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.430+0000 7f341d400640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015ca30 0x7f342014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.430+0000 7f341d400640 1 -- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342006b900 msgr2=0x7f34200b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.430+0000 7f341d400640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342006b900 0x7f34200b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.430+0000 7f341d400640 1 -- 172.21.15.47:0/2579422400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f342015e650 con 0x7f3420156b60 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.430+0000 7f341e800640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015ca30 0x7f342014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.431+0000 7f341d400640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3420156b60 0x7f342013e400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b3f0 tx=0x7f340c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.430+0000 7fa833e00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81406c940 0x7fa81406ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.431+0000 7fba19e00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb9fc06caf0 0x7fb9fc06efb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba0c036000 tx=0x7fba0c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.431+0000 7fa833e00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81406c940 0x7fa81406ee00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa824038000 tx=0x7fa824032000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.436+0000 7f3416000640 1 -- 172.21.15.47:0/2579422400 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c015020 con 0x7f3420156b60 2024-04-20T10:28:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.436+0000 7f3416000640 1 -- 172.21.15.47:0/2579422400 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f340c011830 con 0x7f3420156b60 2024-04-20T10:28:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.436+0000 7f3416000640 1 -- 172.21.15.47:0/2579422400 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340c011c20 con 0x7f3420156b60 2024-04-20T10:28:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.436+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f342015e930 con 0x7f3420156b60 2024-04-20T10:28:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.437+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f342015ec10 con 0x7f3420156b60 2024-04-20T10:28:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.437+0000 7fba12000640 1 -- 172.21.15.47:0/413525583 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fba1c091070 con 0x7fba140e30b0 2024-04-20T10:28:00.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.437+0000 7fa82be00640 1 -- 172.21.15.47:0/3617781422 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa82005a520 con 0x7fa83406de00 2024-04-20T10:28:00.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.439+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3420069090 con 0x7f3420156b60 2024-04-20T10:28:00.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.441+0000 7f3416000640 1 -- 172.21.15.47:0/2579422400 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f340c018020 con 0x7f3420156b60 2024-04-20T10:28:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.443+0000 7f3416000640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f340006c940 0x7f340006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:00.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.443+0000 7f3416000640 1 -- 172.21.15.47:0/2579422400 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f340c0577e0 con 0x7f3420156b60 2024-04-20T10:28:00.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.445+0000 7f341de00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f340006c940 0x7f340006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:00.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.447+0000 7f341de00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f340006c940 0x7f340006ee00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3410036000 tx=0x7f341003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:00.453 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.454+0000 7f3416000640 1 -- 172.21.15.47:0/2579422400 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f340c01f020 con 0x7f3420156b60 2024-04-20T10:28:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.735+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5628005850 con 0x7f5634004800 2024-04-20T10:28:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.736+0000 7f5648c00640 1 -- 172.21.15.47:0/2277128558 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f564005a4c0 con 0x7f5634004800 2024-04-20T10:28:00.736 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:28:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.739+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f562c06c8f0 msgr2=0x7f562c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.739+0000 7f5653c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f562c06c8f0 0x7f562c06edb0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5638038000 tx=0x7f563803d040 comp rx=0 tx=0).stop 2024-04-20T10:28:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.740+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634004800 msgr2=0x7f564c15ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.740+0000 7f5653c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634004800 0x7f564c15ba90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f564000f980 tx=0x7f564000fe40 comp rx=0 tx=0).stop 2024-04-20T10:28:00.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.740+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 shutdown_connections 2024-04-20T10:28:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.740+0000 7f5653c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f562c06c8f0 0x7f562c06edb0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.740+0000 7f5653c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f564c0b46f0 0x7f564c0b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.741+0000 7f5653c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f564c150e20 0x7f564c15c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.741+0000 7f5653c00640 1 --2- 172.21.15.47:0/2277128558 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5634004800 0x7f564c15ba90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.741+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 >> 172.21.15.47:0/2277128558 conn(0x7f564c0cffe0 msgr2=0x7f564c06f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.741+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 shutdown_connections 2024-04-20T10:28:00.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.741+0000 7f5653c00640 1 -- 172.21.15.47:0/2277128558 wait complete. 2024-04-20T10:28:00.768 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.3 2024-04-20T10:28:00.769 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.802+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f92a4005580 con 0x7f92c0054120 2024-04-20T10:28:00.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.803+0000 7f92bb400640 1 -- 172.21.15.47:0/669202265 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f92b40596d0 con 0x7f92c0054120 2024-04-20T10:28:00.803 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:28:00.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f92a806caf0 msgr2=0x7f92a806efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f92a806caf0 0x7f92a806efb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f92bc0090b0 tx=0x7f92bc008030 comp rx=0 tx=0).stop 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0054120 msgr2=0x7f92c00d8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0054120 0x7f92c00d8400 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f92b4006fd0 tx=0x7f92b400dc60 comp rx=0 tx=0).stop 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 shutdown_connections 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92c00e30b0 0x7f92c00cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f92a806caf0 0x7f92a806efb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92c0054120 0x7f92c00d8400 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 --2- 172.21.15.47:0/669202265 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92c003e830 0x7f92c00cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.806+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 >> 172.21.15.47:0/669202265 conn(0x7f92c00559d0 msgr2=0x7f92c0055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.807+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 shutdown_connections 2024-04-20T10:28:00.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.807+0000 7f92ce400640 1 -- 172.21.15.47:0/669202265 wait complete. 2024-04-20T10:28:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.817+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f51c8005580 con 0x7f51f006cde0 2024-04-20T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.818+0000 7f51e6000640 1 -- 172.21.15.47:0/2689915758 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f51dc058c20 con 0x7f51f006cde0 2024-04-20T10:28:00.818 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:28:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.822+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f51d006c8f0 msgr2=0x7f51d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.822+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f51d006c8f0 0x7f51d006edb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f51f01f6660 tx=0x7f51e0009040 comp rx=0 tx=0).stop 2024-04-20T10:28:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.822+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f006cde0 msgr2=0x7f51f0072c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.822+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f006cde0 0x7f51f0072c30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c440 tx=0x7f51dc00c900 comp rx=0 tx=0).stop 2024-04-20T10:28:00.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.823+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 shutdown_connections 2024-04-20T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.823+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51f0161d50 0x7f51f00c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.823+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51f00b3330 0x7f51f00c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.823+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f51d006c8f0 0x7f51d006edb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.823+0000 7f51ef200640 1 --2- 172.21.15.47:0/2689915758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51f006cde0 0x7f51f0072c30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.823+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 >> 172.21.15.47:0/2689915758 conn(0x7f51f00d4450 msgr2=0x7f51f0160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.824+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 shutdown_connections 2024-04-20T10:28:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.824+0000 7f51ef200640 1 -- 172.21.15.47:0/2689915758 wait complete. 2024-04-20T10:28:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.829+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fba14055130 con 0x7fba140e30b0 2024-04-20T10:28:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.830+0000 7fba12000640 1 -- 172.21.15.47:0/413525583 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fba1c106020 con 0x7fba140e30b0 2024-04-20T10:28:00.830 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb9fc06caf0 msgr2=0x7fb9fc06efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb9fc06caf0 0x7fb9fc06efb0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fba0c036000 tx=0x7fba0c03b040 comp rx=0 tx=0).stop 2024-04-20T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba140e30b0 msgr2=0x7fba140e1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.834 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba140e30b0 0x7fba140e1040 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fba1c065480 tx=0x7fba1c069460 comp rx=0 tx=0).stop 2024-04-20T10:28:00.835 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.6 2024-04-20T10:28:00.835 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 shutdown_connections 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fba140e30b0 0x7fba140e1040 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fba14054120 0x7fba14051700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb9fc06caf0 0x7fb9fc06efb0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 --2- 172.21.15.47:0/413525583 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fba1403e830 0x7fba1404e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.834+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 >> 172.21.15.47:0/413525583 conn(0x7fba140559d0 msgr2=0x7fba140e6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.835+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 shutdown_connections 2024-04-20T10:28:00.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.835+0000 7fba20c00640 1 -- 172.21.15.47:0/413525583 wait complete. 2024-04-20T10:28:00.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.841+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f87f0005390 con 0x7f87f0053c70 2024-04-20T10:28:00.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.841+0000 7f87f5600640 1 -- 172.21.15.47:0/2861231371 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f87f8090070 con 0x7f87f0053c70 2024-04-20T10:28:00.842 INFO:teuthology.orchestra.run.smithi047.stdout:60129542154 2024-04-20T10:28:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87d806c8f0 msgr2=0x7f87d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87d806c8f0 0x7f87d806edb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f87e4007fd0 tx=0x7f87e4009040 comp rx=0 tx=0).stop 2024-04-20T10:28:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 msgr2=0x7f87f0034d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0034d10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f87f8051240 tx=0x7f87f8069460 comp rx=0 tx=0).stop 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 shutdown_connections 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87d806c8f0 0x7f87d806edb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0053c70 0x7f87f0034d10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f0033ef0 0x7f87f004c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 --2- 172.21.15.47:0/2861231371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f000c0a0 0x7f87f004aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.844+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 >> 172.21.15.47:0/2861231371 conn(0x7f87f0009fb0 msgr2=0x7f87f00dc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.845+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 shutdown_connections 2024-04-20T10:28:00.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.845+0000 7f87fee00640 1 -- 172.21.15.47:0/2861231371 wait complete. 2024-04-20T10:28:00.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.0 2024-04-20T10:28:00.848 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.5 2024-04-20T10:28:00.862 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542154 for osd.4 2024-04-20T10:28:00.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.867+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f34200cc860 con 0x7f3420156b60 2024-04-20T10:28:00.867 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.867+0000 7f3416000640 1 -- 172.21.15.47:0/2579422400 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f340c059e40 con 0x7f3420156b60 2024-04-20T10:28:00.868 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.869+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f340006c940 msgr2=0x7f340006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.869+0000 7f3424c00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f340006c940 0x7f340006ee00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3410036000 tx=0x7f341003c040 comp rx=0 tx=0).stop 2024-04-20T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.869+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3420156b60 msgr2=0x7f342013e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.869+0000 7f3424c00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3420156b60 0x7f342013e400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f340c00b3f0 tx=0x7f340c00b8b0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 shutdown_connections 2024-04-20T10:28:00.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f342015ca30 0x7f342014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f340006c940 0x7f340006ee00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3420156b60 0x7f342013e400 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 --2- 172.21.15.47:0/2579422400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f342006b900 0x7f34200b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 >> 172.21.15.47:0/2579422400 conn(0x7f34200d0570 msgr2=0x7f342006eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 shutdown_connections 2024-04-20T10:28:00.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.870+0000 7f3424c00640 1 -- 172.21.15.47:0/2579422400 wait complete. 2024-04-20T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.874+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7b9c09cf90 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.875+0000 7f7b93e00640 1 -- 172.21.15.47:0/1457367110 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b88058e60 con 0x7f7b9c0d0b00 2024-04-20T10:28:00.875 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:28:00.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7b7c06c940 msgr2=0x7f7b7c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7b7c06c940 0x7f7b7c06ee00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c0b10f0 tx=0x7f7b8c009240 comp rx=0 tx=0).stop 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0d0b00 msgr2=0x7f7b9c15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0d0b00 0x7f7b9c15d1e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7b8800c780 tx=0x7f7b8800cc40 comp rx=0 tx=0).stop 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 shutdown_connections 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7b7c06c940 0x7f7b7c06ee00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7b9c15e950 0x7f7b9c09a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7b9c0d0b00 0x7f7b9c15d1e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 --2- 172.21.15.47:0/1457367110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7b9c0adeb0 0x7f7b9c0b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 >> 172.21.15.47:0/1457367110 conn(0x7f7b9c0cffe0 msgr2=0x7f7b9c06ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 shutdown_connections 2024-04-20T10:28:00.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.877+0000 7f7ba3000640 1 -- 172.21.15.47:0/1457367110 wait complete. 2024-04-20T10:28:00.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.882+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa80c005850 con 0x7fa83406de00 2024-04-20T10:28:00.883 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.883+0000 7fa82be00640 1 -- 172.21.15.47:0/3617781422 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa820059e20 con 0x7fa83406de00 2024-04-20T10:28:00.883 INFO:teuthology.orchestra.run.smithi047.stdout:64424509450 2024-04-20T10:28:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.885+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81406c940 msgr2=0x7fa81406ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.885+0000 7fa83ac00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81406c940 0x7fa81406ee00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa824038000 tx=0x7fa824032000 comp rx=0 tx=0).stop 2024-04-20T10:28:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83406de00 msgr2=0x7fa8340d02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83406de00 0x7fa8340d02a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa82000d5e0 tx=0x7fa82000daa0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 shutdown_connections 2024-04-20T10:28:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83415a7d0 0x7fa83409aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81406c940 0x7fa81406ee00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83406de00 0x7fa8340d02a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 --2- 172.21.15.47:0/3617781422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83406bdd0 0x7fa834098fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 >> 172.21.15.47:0/3617781422 conn(0x7fa834072090 msgr2=0x7fa834157100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 shutdown_connections 2024-04-20T10:28:00.887 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:00.886+0000 7fa83ac00640 1 -- 172.21.15.47:0/3617781422 wait complete. 2024-04-20T10:28:00.888 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-04-20T10:28:00.888 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.895 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.2 2024-04-20T10:28:00.895 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.1 2024-04-20T10:28:00.902 DEBUG:teuthology.parallel:result is None 2024-04-20T10:28:00.902 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T10:28:00.902 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:28:01.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.152+0000 7f66eb000640 1 Processor -- start 2024-04-20T10:28:01.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.152+0000 7f66eb000640 1 -- start start 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.153+0000 7f66eb000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66e415b400 0x7f66e4072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.153+0000 7f66eb000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f66e415fa40 0x7f66e406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66eb000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f66e415c7f0 con 0x7f66e415eec0 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66eb000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f66e406d280 con 0x7f66e415b400 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66eb000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f66e406ccc0 con 0x7f66e415fa40 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66e9c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66e415b400 0x7f66e4072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66e9c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f66e415b400 0x7f66e4072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56202/0 (socket says 172.21.15.47:56202) 2024-04-20T10:28:01.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66e9c00640 1 -- 172.21.15.47:0/1848701246 learned_addr learned my addr 172.21.15.47:0/1848701246 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:01.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66ea600640 1 --2- 172.21.15.47:0/1848701246 >> v2:172.21.15.175:3300/0 conn(0x7f66e415fa40 0x7f66e406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66e9c00640 1 -- 172.21.15.47:0/1848701246 >> v2:172.21.15.175:3300/0 conn(0x7f66e415fa40 msgr2=0x7f66e406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:01.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66e9c00640 1 --2- 172.21.15.47:0/1848701246 >> v2:172.21.15.175:3300/0 conn(0x7f66e415fa40 0x7f66e406ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66e9c00640 1 -- 172.21.15.47:0/1848701246 >> v1:172.21.15.175:6789/0 conn(0x7f66e415eec0 legacy=0x7f66e415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:01.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.154+0000 7f66e9c00640 1 -- 172.21.15.47:0/1848701246 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e406cfa0 con 0x7f66e415b400 2024-04-20T10:28:01.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.155+0000 7f66e9c00640 1 --2- 172.21.15.47:0/1848701246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415b400 0x7f66e4072d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f66e4072250 tx=0x7f66d802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba2948636ffca368 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:01.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66e3e00640 1 -- 172.21.15.47:0/1848701246 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8035020 con 0x7f66e415b400 2024-04-20T10:28:01.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66e3e00640 1 -- 172.21.15.47:0/1848701246 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d8031870 con 0x7f66e415b400 2024-04-20T10:28:01.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66e3e00640 1 -- 172.21.15.47:0/1848701246 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d8031c40 con 0x7f66e415b400 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66eb000640 1 -- 172.21.15.47:0/1848701246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415b400 msgr2=0x7f66e4072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66eb000640 1 --2- 172.21.15.47:0/1848701246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415b400 0x7f66e4072d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f66e4072250 tx=0x7f66d802b910 comp rx=0 tx=0).stop 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66eb000640 1 -- 172.21.15.47:0/1848701246 shutdown_connections 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66eb000640 1 --2- 172.21.15.47:0/1848701246 >> v2:172.21.15.175:3300/0 conn(0x7f66e415fa40 0x7f66e406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66eb000640 1 --2- 172.21.15.47:0/1848701246 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415b400 0x7f66e4072d00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66eb000640 1 -- 172.21.15.47:0/1848701246 >> 172.21.15.47:0/1848701246 conn(0x7f66e40d0040 msgr2=0x7f66e40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.156+0000 7f66eb000640 1 -- 172.21.15.47:0/1848701246 shutdown_connections 2024-04-20T10:28:01.157 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.157+0000 7f66eb000640 1 -- 172.21.15.47:0/1848701246 wait complete. 2024-04-20T10:28:01.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.158+0000 7f66eb000640 1 Processor -- start 2024-04-20T10:28:01.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.158+0000 7f66eb000640 1 -- start start 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.159+0000 7f66eb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e415b400 0x7f66e41545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.159+0000 7f66eb000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415eec0 0x7f66e4157560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.159+0000 7f66eb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e415fa40 0x7f66e409ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.159+0000 7f66eb000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f66e4073240 con 0x7f66e415eec0 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.159+0000 7f66eb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66e406e720 con 0x7f66e415b400 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.159+0000 7f66eb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f66e406f180 con 0x7f66e415fa40 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.159+0000 7f66e9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415eec0 0x7f66e4157560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415eec0 0x7f66e4157560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56214/0 (socket says 172.21.15.47:56214) 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 -- 172.21.15.47:0/2219538913 learned_addr learned my addr 172.21.15.47:0/2219538913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:01.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9c00640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e415b400 0x7f66e41545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66ea600640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e415fa40 0x7f66e409ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 -- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e415fa40 msgr2=0x7f66e409ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e415fa40 0x7f66e409ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 -- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e415b400 msgr2=0x7f66e41545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e415b400 0x7f66e41545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 -- 172.21.15.47:0/2219538913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e40cf6f0 con 0x7f66e415eec0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e9200640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415eec0 0x7f66e4157560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c440 tx=0x7f66d000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.160+0000 7f66e1600640 1 -- 172.21.15.47:0/2219538913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d0017070 con 0x7f66e415eec0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.161+0000 7f66e1600640 1 -- 172.21.15.47:0/2219538913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f66d00035f0 con 0x7f66e415eec0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.161+0000 7f66e1600640 1 -- 172.21.15.47:0/2219538913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66d00039e0 con 0x7f66e415eec0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.161+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e409b6a0 con 0x7f66e415eec0 2024-04-20T10:28:01.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.161+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e409b1a0 con 0x7f66e415eec0 2024-04-20T10:28:01.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.162+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e4075b60 con 0x7f66e415eec0 2024-04-20T10:28:01.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.163+0000 7f66e1600640 1 -- 172.21.15.47:0/2219538913 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f66d0003b80 con 0x7f66e415eec0 2024-04-20T10:28:01.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.164+0000 7f66e1600640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f66c406c920 0x7f66c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.164+0000 7f66e1600640 1 -- 172.21.15.47:0/2219538913 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f66d008b110 con 0x7f66e415eec0 2024-04-20T10:28:01.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.164+0000 7f66e9c00640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f66c406c920 0x7f66c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.165+0000 7f66e9c00640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f66c406c920 0x7f66c406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66d8036000 tx=0x7f66d803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:01.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.168+0000 7f66e1600640 1 -- 172.21.15.47:0/2219538913 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f66d0059160 con 0x7f66e415eec0 2024-04-20T10:28:01.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.363+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f66e40cf4a0 con 0x7f66c406c920 2024-04-20T10:28:01.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.388+0000 7f66e1600640 1 -- 172.21.15.47:0/2219538913 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f66e40cf4a0 con 0x7f66c406c920 2024-04-20T10:28:01.393 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:01.395 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f66c406c920 msgr2=0x7f66c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f66c406c920 0x7f66c406ede0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f66d8036000 tx=0x7f66d803c040 comp rx=0 tx=0).stop 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415eec0 msgr2=0x7f66e4157560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415eec0 0x7f66e4157560 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f66d000c440 tx=0x7f66d000c900 comp rx=0 tx=0).stop 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 shutdown_connections 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66e415fa40 0x7f66e409ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f66c406c920 0x7f66c406ede0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f66e415eec0 0x7f66e4157560 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 --2- 172.21.15.47:0/2219538913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66e415b400 0x7f66e41545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 >> 172.21.15.47:0/2219538913 conn(0x7f66e40d0040 msgr2=0x7f66e41556d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 shutdown_connections 2024-04-20T10:28:01.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.396+0000 7f66eb000640 1 -- 172.21.15.47:0/2219538913 wait complete. 2024-04-20T10:28:01.416 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-04-20T10:28:01.074553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227932,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746788,"statfs":{"total":773094113280,"available":772860710912,"internally_reserved":0,"allocated":3514368,"data_stored":2072296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005662"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392064+0000","last_change":"2024-04-20T10:27:32.090082+0000","last_active":"2024-04-20T10:27:38.392064+0000","last_peered":"2024-04-20T10:27:38.392064+0000","last_clean":"2024-04-20T10:27:38.392064+0000","last_became_active":"2024-04-20T10:27:32.089819+0000","last_became_peered":"2024-04-20T10:27:32.089819+0000","last_unstale":"2024-04-20T10:27:38.392064+0000","last_undegraded":"2024-04-20T10:27:38.392064+0000","last_fullsized":"2024-04-20T10:27:38.392064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:16.568087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112567+0000","last_change":"2024-04-20T10:27:33.015363+0000","last_active":"2024-04-20T10:27:38.112567+0000","last_peered":"2024-04-20T10:27:38.112567+0000","last_clean":"2024-04-20T10:27:38.112567+0000","last_became_active":"2024-04-20T10:27:33.013421+0000","last_became_peered":"2024-04-20T10:27:33.013421+0000","last_unstale":"2024-04-20T10:27:38.112567+0000","last_undegraded":"2024-04-20T10:27:38.112567+0000","last_fullsized":"2024-04-20T10:27:38.112567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391215+0000","last_change":"2024-04-20T10:27:37.132637+0000","last_active":"2024-04-20T10:27:38.391215+0000","last_peered":"2024-04-20T10:27:38.391215+0000","last_clean":"2024-04-20T10:27:38.391215+0000","last_became_active":"2024-04-20T10:27:37.132147+0000","last_became_peered":"2024-04-20T10:27:37.132147+0000","last_unstale":"2024-04-20T10:27:38.391215+0000","last_undegraded":"2024-04-20T10:27:38.391215+0000","last_fullsized":"2024-04-20T10:27:38.391215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:13.081027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112843+0000","last_change":"2024-04-20T10:27:32.093624+0000","last_active":"2024-04-20T10:27:38.112843+0000","last_peered":"2024-04-20T10:27:38.112843+0000","last_clean":"2024-04-20T10:27:38.112843+0000","last_became_active":"2024-04-20T10:27:32.093349+0000","last_became_peered":"2024-04-20T10:27:32.093349+0000","last_unstale":"2024-04-20T10:27:38.112843+0000","last_undegraded":"2024-04-20T10:27:38.112843+0000","last_fullsized":"2024-04-20T10:27:38.112843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113178+0000","last_change":"2024-04-20T10:27:33.019078+0000","last_active":"2024-04-20T10:27:38.113178+0000","last_peered":"2024-04-20T10:27:38.113178+0000","last_clean":"2024-04-20T10:27:38.113178+0000","last_became_active":"2024-04-20T10:27:33.018362+0000","last_became_peered":"2024-04-20T10:27:33.018362+0000","last_unstale":"2024-04-20T10:27:38.113178+0000","last_undegraded":"2024-04-20T10:27:38.113178+0000","last_fullsized":"2024-04-20T10:27:38.113178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390250+0000","last_change":"2024-04-20T10:27:37.118407+0000","last_active":"2024-04-20T10:27:38.390250+0000","last_peered":"2024-04-20T10:27:38.390250+0000","last_clean":"2024-04-20T10:27:38.390250+0000","last_became_active":"2024-04-20T10:27:37.117850+0000","last_became_peered":"2024-04-20T10:27:37.117850+0000","last_unstale":"2024-04-20T10:27:38.390250+0000","last_undegraded":"2024-04-20T10:27:38.390250+0000","last_fullsized":"2024-04-20T10:27:38.390250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:36.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388629+0000","last_change":"2024-04-20T10:27:32.080920+0000","last_active":"2024-04-20T10:27:38.388629+0000","last_peered":"2024-04-20T10:27:38.388629+0000","last_clean":"2024-04-20T10:27:38.388629+0000","last_became_active":"2024-04-20T10:27:32.080614+0000","last_became_peered":"2024-04-20T10:27:32.080614+0000","last_unstale":"2024-04-20T10:27:38.388629+0000","last_undegraded":"2024-04-20T10:27:38.388629+0000","last_fullsized":"2024-04-20T10:27:38.388629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:51.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392440+0000","last_change":"2024-04-20T10:27:33.006318+0000","last_active":"2024-04-20T10:27:38.392440+0000","last_peered":"2024-04-20T10:27:38.392440+0000","last_clean":"2024-04-20T10:27:38.392440+0000","last_became_active":"2024-04-20T10:27:33.006040+0000","last_became_peered":"2024-04-20T10:27:33.006040+0000","last_unstale":"2024-04-20T10:27:38.392440+0000","last_undegraded":"2024-04-20T10:27:38.392440+0000","last_fullsized":"2024-04-20T10:27:38.392440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391743+0000","last_change":"2024-04-20T10:27:37.123688+0000","last_active":"2024-04-20T10:27:38.391743+0000","last_peered":"2024-04-20T10:27:38.391743+0000","last_clean":"2024-04-20T10:27:38.391743+0000","last_became_active":"2024-04-20T10:27:37.123316+0000","last_became_peered":"2024-04-20T10:27:37.123316+0000","last_unstale":"2024-04-20T10:27:38.391743+0000","last_undegraded":"2024-04-20T10:27:38.391743+0000","last_fullsized":"2024-04-20T10:27:38.391743+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388939+0000","last_change":"2024-04-20T10:27:32.082282+0000","last_active":"2024-04-20T10:27:38.388939+0000","last_peered":"2024-04-20T10:27:38.388939+0000","last_clean":"2024-04-20T10:27:38.388939+0000","last_became_active":"2024-04-20T10:27:32.081943+0000","last_became_peered":"2024-04-20T10:27:32.081943+0000","last_unstale":"2024-04-20T10:27:38.388939+0000","last_undegraded":"2024-04-20T10:27:38.388939+0000","last_fullsized":"2024-04-20T10:27:38.388939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:12.394383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108312+0000","last_change":"2024-04-20T10:27:33.003700+0000","last_active":"2024-04-20T10:27:38.108312+0000","last_peered":"2024-04-20T10:27:38.108312+0000","last_clean":"2024-04-20T10:27:38.108312+0000","last_became_active":"2024-04-20T10:27:33.003443+0000","last_became_peered":"2024-04-20T10:27:33.003443+0000","last_unstale":"2024-04-20T10:27:38.108312+0000","last_undegraded":"2024-04-20T10:27:38.108312+0000","last_fullsized":"2024-04-20T10:27:38.108312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:33.760650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390259+0000","last_change":"2024-04-20T10:27:37.127775+0000","last_active":"2024-04-20T10:27:38.390259+0000","last_peered":"2024-04-20T10:27:38.390259+0000","last_clean":"2024-04-20T10:27:38.390259+0000","last_became_active":"2024-04-20T10:27:37.126837+0000","last_became_peered":"2024-04-20T10:27:37.126837+0000","last_unstale":"2024-04-20T10:27:38.390259+0000","last_undegraded":"2024-04-20T10:27:38.390259+0000","last_fullsized":"2024-04-20T10:27:38.390259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:31.369897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106458+0000","last_change":"2024-04-20T10:27:33.004497+0000","last_active":"2024-04-20T10:27:38.106458+0000","last_peered":"2024-04-20T10:27:38.106458+0000","last_clean":"2024-04-20T10:27:38.106458+0000","last_became_active":"2024-04-20T10:27:33.004228+0000","last_became_peered":"2024-04-20T10:27:33.004228+0000","last_unstale":"2024-04-20T10:27:38.106458+0000","last_undegraded":"2024-04-20T10:27:38.106458+0000","last_fullsized":"2024-04-20T10:27:38.106458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:04.936311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106698+0000","last_change":"2024-04-20T10:27:32.069803+0000","last_active":"2024-04-20T10:27:38.106698+0000","last_peered":"2024-04-20T10:27:38.106698+0000","last_clean":"2024-04-20T10:27:38.106698+0000","last_became_active":"2024-04-20T10:27:32.069542+0000","last_became_peered":"2024-04-20T10:27:32.069542+0000","last_unstale":"2024-04-20T10:27:38.106698+0000","last_undegraded":"2024-04-20T10:27:38.106698+0000","last_fullsized":"2024-04-20T10:27:38.106698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107502+0000","last_change":"2024-04-20T10:27:37.122386+0000","last_active":"2024-04-20T10:27:38.107502+0000","last_peered":"2024-04-20T10:27:38.107502+0000","last_clean":"2024-04-20T10:27:38.107502+0000","last_became_active":"2024-04-20T10:27:37.121943+0000","last_became_peered":"2024-04-20T10:27:37.121943+0000","last_unstale":"2024-04-20T10:27:38.107502+0000","last_undegraded":"2024-04-20T10:27:38.107502+0000","last_fullsized":"2024-04-20T10:27:38.107502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:53:31.071316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391786+0000","last_change":"2024-04-20T10:27:33.015242+0000","last_active":"2024-04-20T10:27:38.391786+0000","last_peered":"2024-04-20T10:27:38.391786+0000","last_clean":"2024-04-20T10:27:38.391786+0000","last_became_active":"2024-04-20T10:27:33.014003+0000","last_became_peered":"2024-04-20T10:27:33.014003+0000","last_unstale":"2024-04-20T10:27:38.391786+0000","last_undegraded":"2024-04-20T10:27:38.391786+0000","last_fullsized":"2024-04-20T10:27:38.391786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110379+0000","last_change":"2024-04-20T10:27:32.087085+0000","last_active":"2024-04-20T10:27:38.110379+0000","last_peered":"2024-04-20T10:27:38.110379+0000","last_clean":"2024-04-20T10:27:38.110379+0000","last_became_active":"2024-04-20T10:27:32.086777+0000","last_became_peered":"2024-04-20T10:27:32.086777+0000","last_unstale":"2024-04-20T10:27:38.110379+0000","last_undegraded":"2024-04-20T10:27:38.110379+0000","last_fullsized":"2024-04-20T10:27:38.110379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388751+0000","last_change":"2024-04-20T10:27:37.123374+0000","last_active":"2024-04-20T10:27:38.388751+0000","last_peered":"2024-04-20T10:27:38.388751+0000","last_clean":"2024-04-20T10:27:38.388751+0000","last_became_active":"2024-04-20T10:27:37.118855+0000","last_became_peered":"2024-04-20T10:27:37.118855+0000","last_unstale":"2024-04-20T10:27:38.388751+0000","last_undegraded":"2024-04-20T10:27:38.388751+0000","last_fullsized":"2024-04-20T10:27:38.388751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389358+0000","last_change":"2024-04-20T10:27:33.013100+0000","last_active":"2024-04-20T10:27:38.389358+0000","last_peered":"2024-04-20T10:27:38.389358+0000","last_clean":"2024-04-20T10:27:38.389358+0000","last_became_active":"2024-04-20T10:27:33.012720+0000","last_became_peered":"2024-04-20T10:27:33.012720+0000","last_unstale":"2024-04-20T10:27:38.389358+0000","last_undegraded":"2024-04-20T10:27:38.389358+0000","last_fullsized":"2024-04-20T10:27:38.389358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:52.950066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390484+0000","last_change":"2024-04-20T10:27:32.082330+0000","last_active":"2024-04-20T10:27:38.390484+0000","last_peered":"2024-04-20T10:27:38.390484+0000","last_clean":"2024-04-20T10:27:38.390484+0000","last_became_active":"2024-04-20T10:27:32.080602+0000","last_became_peered":"2024-04-20T10:27:32.080602+0000","last_unstale":"2024-04-20T10:27:38.390484+0000","last_undegraded":"2024-04-20T10:27:38.390484+0000","last_fullsized":"2024-04-20T10:27:38.390484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:59.067061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392568+0000","last_change":"2024-04-20T10:27:37.125949+0000","last_active":"2024-04-20T10:27:38.392568+0000","last_peered":"2024-04-20T10:27:38.392568+0000","last_clean":"2024-04-20T10:27:38.392568+0000","last_became_active":"2024-04-20T10:27:37.125481+0000","last_became_peered":"2024-04-20T10:27:37.125481+0000","last_unstale":"2024-04-20T10:27:38.392568+0000","last_undegraded":"2024-04-20T10:27:38.392568+0000","last_fullsized":"2024-04-20T10:27:38.392568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:15.992852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105786+0000","last_change":"2024-04-20T10:27:33.015612+0000","last_active":"2024-04-20T10:27:38.105786+0000","last_peered":"2024-04-20T10:27:38.105786+0000","last_clean":"2024-04-20T10:27:38.105786+0000","last_became_active":"2024-04-20T10:27:33.014370+0000","last_became_peered":"2024-04-20T10:27:33.014370+0000","last_unstale":"2024-04-20T10:27:38.105786+0000","last_undegraded":"2024-04-20T10:27:38.105786+0000","last_fullsized":"2024-04-20T10:27:38.105786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387559+0000","last_change":"2024-04-20T10:27:32.068081+0000","last_active":"2024-04-20T10:27:38.387559+0000","last_peered":"2024-04-20T10:27:38.387559+0000","last_clean":"2024-04-20T10:27:38.387559+0000","last_became_active":"2024-04-20T10:27:32.067727+0000","last_became_peered":"2024-04-20T10:27:32.067727+0000","last_unstale":"2024-04-20T10:27:38.387559+0000","last_undegraded":"2024-04-20T10:27:38.387559+0000","last_fullsized":"2024-04-20T10:27:38.387559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:59.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392244+0000","last_change":"2024-04-20T10:27:37.119884+0000","last_active":"2024-04-20T10:27:38.392244+0000","last_peered":"2024-04-20T10:27:38.392244+0000","last_clean":"2024-04-20T10:27:38.392244+0000","last_became_active":"2024-04-20T10:27:37.119461+0000","last_became_peered":"2024-04-20T10:27:37.119461+0000","last_unstale":"2024-04-20T10:27:38.392244+0000","last_undegraded":"2024-04-20T10:27:38.392244+0000","last_fullsized":"2024-04-20T10:27:38.392244+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391977+0000","last_change":"2024-04-20T10:27:33.009139+0000","last_active":"2024-04-20T10:27:38.391977+0000","last_peered":"2024-04-20T10:27:38.391977+0000","last_clean":"2024-04-20T10:27:38.391977+0000","last_became_active":"2024-04-20T10:27:33.008830+0000","last_became_peered":"2024-04-20T10:27:33.008830+0000","last_unstale":"2024-04-20T10:27:38.391977+0000","last_undegraded":"2024-04-20T10:27:38.391977+0000","last_fullsized":"2024-04-20T10:27:38.391977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:06.157535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108669+0000","last_change":"2024-04-20T10:27:32.086356+0000","last_active":"2024-04-20T10:27:38.108669+0000","last_peered":"2024-04-20T10:27:38.108669+0000","last_clean":"2024-04-20T10:27:38.108669+0000","last_became_active":"2024-04-20T10:27:32.077640+0000","last_became_peered":"2024-04-20T10:27:32.077640+0000","last_unstale":"2024-04-20T10:27:38.108669+0000","last_undegraded":"2024-04-20T10:27:38.108669+0000","last_fullsized":"2024-04-20T10:27:38.108669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:44.469214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390797+0000","last_change":"2024-04-20T10:27:37.124270+0000","last_active":"2024-04-20T10:27:38.390797+0000","last_peered":"2024-04-20T10:27:38.390797+0000","last_clean":"2024-04-20T10:27:38.390797+0000","last_became_active":"2024-04-20T10:27:37.123953+0000","last_became_peered":"2024-04-20T10:27:37.123953+0000","last_unstale":"2024-04-20T10:27:38.390797+0000","last_undegraded":"2024-04-20T10:27:38.390797+0000","last_fullsized":"2024-04-20T10:27:38.390797+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:27.841628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390487+0000","last_change":"2024-04-20T10:27:33.017825+0000","last_active":"2024-04-20T10:27:38.390487+0000","last_peered":"2024-04-20T10:27:38.390487+0000","last_clean":"2024-04-20T10:27:38.390487+0000","last_became_active":"2024-04-20T10:27:33.015635+0000","last_became_peered":"2024-04-20T10:27:33.015635+0000","last_unstale":"2024-04-20T10:27:38.390487+0000","last_undegraded":"2024-04-20T10:27:38.390487+0000","last_fullsized":"2024-04-20T10:27:38.390487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:58.914910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392232+0000","last_change":"2024-04-20T10:27:32.082908+0000","last_active":"2024-04-20T10:27:38.392232+0000","last_peered":"2024-04-20T10:27:38.392232+0000","last_clean":"2024-04-20T10:27:38.392232+0000","last_became_active":"2024-04-20T10:27:32.082507+0000","last_became_peered":"2024-04-20T10:27:32.082507+0000","last_unstale":"2024-04-20T10:27:38.392232+0000","last_undegraded":"2024-04-20T10:27:38.392232+0000","last_fullsized":"2024-04-20T10:27:38.392232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:52.804365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390908+0000","last_change":"2024-04-20T10:27:37.126093+0000","last_active":"2024-04-20T10:27:38.390908+0000","last_peered":"2024-04-20T10:27:38.390908+0000","last_clean":"2024-04-20T10:27:38.390908+0000","last_became_active":"2024-04-20T10:27:37.125681+0000","last_became_peered":"2024-04-20T10:27:37.125681+0000","last_unstale":"2024-04-20T10:27:38.390908+0000","last_undegraded":"2024-04-20T10:27:38.390908+0000","last_fullsized":"2024-04-20T10:27:38.390908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:13:36.622877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109415+0000","last_change":"2024-04-20T10:27:33.011662+0000","last_active":"2024-04-20T10:27:38.109415+0000","last_peered":"2024-04-20T10:27:38.109415+0000","last_clean":"2024-04-20T10:27:38.109415+0000","last_became_active":"2024-04-20T10:27:33.010842+0000","last_became_peered":"2024-04-20T10:27:33.010842+0000","last_unstale":"2024-04-20T10:27:38.109415+0000","last_undegraded":"2024-04-20T10:27:38.109415+0000","last_fullsized":"2024-04-20T10:27:38.109415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108664+0000","last_change":"2024-04-20T10:27:32.083502+0000","last_active":"2024-04-20T10:27:38.108664+0000","last_peered":"2024-04-20T10:27:38.108664+0000","last_clean":"2024-04-20T10:27:38.108664+0000","last_became_active":"2024-04-20T10:27:32.083217+0000","last_became_peered":"2024-04-20T10:27:32.083217+0000","last_unstale":"2024-04-20T10:27:38.108664+0000","last_undegraded":"2024-04-20T10:27:38.108664+0000","last_fullsized":"2024-04-20T10:27:38.108664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:41.953724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109193+0000","last_change":"2024-04-20T10:27:37.110995+0000","last_active":"2024-04-20T10:27:38.109193+0000","last_peered":"2024-04-20T10:27:38.109193+0000","last_clean":"2024-04-20T10:27:38.109193+0000","last_became_active":"2024-04-20T10:27:37.110619+0000","last_became_peered":"2024-04-20T10:27:37.110619+0000","last_unstale":"2024-04-20T10:27:38.109193+0000","last_undegraded":"2024-04-20T10:27:38.109193+0000","last_fullsized":"2024-04-20T10:27:38.109193+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:33.173033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391490+0000","last_change":"2024-04-20T10:27:32.997361+0000","last_active":"2024-04-20T10:27:38.391490+0000","last_peered":"2024-04-20T10:27:38.391490+0000","last_clean":"2024-04-20T10:27:38.391490+0000","last_became_active":"2024-04-20T10:27:32.996366+0000","last_became_peered":"2024-04-20T10:27:32.996366+0000","last_unstale":"2024-04-20T10:27:38.391490+0000","last_undegraded":"2024-04-20T10:27:38.391490+0000","last_fullsized":"2024-04-20T10:27:38.391490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110460+0000","last_change":"2024-04-20T10:27:32.087311+0000","last_active":"2024-04-20T10:27:38.110460+0000","last_peered":"2024-04-20T10:27:38.110460+0000","last_clean":"2024-04-20T10:27:38.110460+0000","last_became_active":"2024-04-20T10:27:32.086959+0000","last_became_peered":"2024-04-20T10:27:32.086959+0000","last_unstale":"2024-04-20T10:27:38.110460+0000","last_undegraded":"2024-04-20T10:27:38.110460+0000","last_fullsized":"2024-04-20T10:27:38.110460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110080+0000","last_change":"2024-04-20T10:27:37.121913+0000","last_active":"2024-04-20T10:27:38.110080+0000","last_peered":"2024-04-20T10:27:38.110080+0000","last_clean":"2024-04-20T10:27:38.110080+0000","last_became_active":"2024-04-20T10:27:37.121491+0000","last_became_peered":"2024-04-20T10:27:37.121491+0000","last_unstale":"2024-04-20T10:27:38.110080+0000","last_undegraded":"2024-04-20T10:27:38.110080+0000","last_fullsized":"2024-04-20T10:27:38.110080+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:28.925537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388610+0000","last_change":"2024-04-20T10:27:33.004607+0000","last_active":"2024-04-20T10:27:38.388610+0000","last_peered":"2024-04-20T10:27:38.388610+0000","last_clean":"2024-04-20T10:27:38.388610+0000","last_became_active":"2024-04-20T10:27:33.004299+0000","last_became_peered":"2024-04-20T10:27:33.004299+0000","last_unstale":"2024-04-20T10:27:38.388610+0000","last_undegraded":"2024-04-20T10:27:38.388610+0000","last_fullsized":"2024-04-20T10:27:38.388610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107052+0000","last_change":"2024-04-20T10:27:32.082735+0000","last_active":"2024-04-20T10:27:38.107052+0000","last_peered":"2024-04-20T10:27:38.107052+0000","last_clean":"2024-04-20T10:27:38.107052+0000","last_became_active":"2024-04-20T10:27:32.079695+0000","last_became_peered":"2024-04-20T10:27:32.079695+0000","last_unstale":"2024-04-20T10:27:38.107052+0000","last_undegraded":"2024-04-20T10:27:38.107052+0000","last_fullsized":"2024-04-20T10:27:38.107052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:07.962987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109079+0000","last_change":"2024-04-20T10:27:37.131952+0000","last_active":"2024-04-20T10:27:38.109079+0000","last_peered":"2024-04-20T10:27:38.109079+0000","last_clean":"2024-04-20T10:27:38.109079+0000","last_became_active":"2024-04-20T10:27:37.130721+0000","last_became_peered":"2024-04-20T10:27:37.130721+0000","last_unstale":"2024-04-20T10:27:38.109079+0000","last_undegraded":"2024-04-20T10:27:38.109079+0000","last_fullsized":"2024-04-20T10:27:38.109079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107777+0000","last_change":"2024-04-20T10:27:33.013537+0000","last_active":"2024-04-20T10:27:38.107777+0000","last_peered":"2024-04-20T10:27:38.107777+0000","last_clean":"2024-04-20T10:27:38.107777+0000","last_became_active":"2024-04-20T10:27:33.013223+0000","last_became_peered":"2024-04-20T10:27:33.013223+0000","last_unstale":"2024-04-20T10:27:38.107777+0000","last_undegraded":"2024-04-20T10:27:38.107777+0000","last_fullsized":"2024-04-20T10:27:38.107777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391320+0000","last_change":"2024-04-20T10:27:32.086422+0000","last_active":"2024-04-20T10:27:38.391320+0000","last_peered":"2024-04-20T10:27:38.391320+0000","last_clean":"2024-04-20T10:27:38.391320+0000","last_became_active":"2024-04-20T10:27:32.085353+0000","last_became_peered":"2024-04-20T10:27:32.085353+0000","last_unstale":"2024-04-20T10:27:38.391320+0000","last_undegraded":"2024-04-20T10:27:38.391320+0000","last_fullsized":"2024-04-20T10:27:38.391320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112264+0000","last_change":"2024-04-20T10:27:37.113304+0000","last_active":"2024-04-20T10:27:38.112264+0000","last_peered":"2024-04-20T10:27:38.112264+0000","last_clean":"2024-04-20T10:27:38.112264+0000","last_became_active":"2024-04-20T10:27:37.112828+0000","last_became_peered":"2024-04-20T10:27:37.112828+0000","last_unstale":"2024-04-20T10:27:38.112264+0000","last_undegraded":"2024-04-20T10:27:38.112264+0000","last_fullsized":"2024-04-20T10:27:38.112264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:19.178189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108899+0000","last_change":"2024-04-20T10:27:32.989233+0000","last_active":"2024-04-20T10:27:38.108899+0000","last_peered":"2024-04-20T10:27:38.108899+0000","last_clean":"2024-04-20T10:27:38.108899+0000","last_became_active":"2024-04-20T10:27:32.988092+0000","last_became_peered":"2024-04-20T10:27:32.988092+0000","last_unstale":"2024-04-20T10:27:38.108899+0000","last_undegraded":"2024-04-20T10:27:38.108899+0000","last_fullsized":"2024-04-20T10:27:38.108899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:11.889218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106243+0000","last_change":"2024-04-20T10:27:32.092257+0000","last_active":"2024-04-20T10:27:38.106243+0000","last_peered":"2024-04-20T10:27:38.106243+0000","last_clean":"2024-04-20T10:27:38.106243+0000","last_became_active":"2024-04-20T10:27:32.091944+0000","last_became_peered":"2024-04-20T10:27:32.091944+0000","last_unstale":"2024-04-20T10:27:38.106243+0000","last_undegraded":"2024-04-20T10:27:38.106243+0000","last_fullsized":"2024-04-20T10:27:38.106243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:56.139394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112022+0000","last_change":"2024-04-20T10:27:37.124420+0000","last_active":"2024-04-20T10:27:38.112022+0000","last_peered":"2024-04-20T10:27:38.112022+0000","last_clean":"2024-04-20T10:27:38.112022+0000","last_became_active":"2024-04-20T10:27:37.124048+0000","last_became_peered":"2024-04-20T10:27:37.124048+0000","last_unstale":"2024-04-20T10:27:38.112022+0000","last_undegraded":"2024-04-20T10:27:38.112022+0000","last_fullsized":"2024-04-20T10:27:38.112022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:37.864284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390325+0000","last_change":"2024-04-20T10:27:33.016860+0000","last_active":"2024-04-20T10:27:38.390325+0000","last_peered":"2024-04-20T10:27:38.390325+0000","last_clean":"2024-04-20T10:27:38.390325+0000","last_became_active":"2024-04-20T10:27:33.016323+0000","last_became_peered":"2024-04-20T10:27:33.016323+0000","last_unstale":"2024-04-20T10:27:38.390325+0000","last_undegraded":"2024-04-20T10:27:38.390325+0000","last_fullsized":"2024-04-20T10:27:38.390325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:11:39.053198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107037+0000","last_change":"2024-04-20T10:27:32.069209+0000","last_active":"2024-04-20T10:27:38.107037+0000","last_peered":"2024-04-20T10:27:38.107037+0000","last_clean":"2024-04-20T10:27:38.107037+0000","last_became_active":"2024-04-20T10:27:32.068711+0000","last_became_peered":"2024-04-20T10:27:32.068711+0000","last_unstale":"2024-04-20T10:27:38.107037+0000","last_undegraded":"2024-04-20T10:27:38.107037+0000","last_fullsized":"2024-04-20T10:27:38.107037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:55.256665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392446+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:27:38.392446+0000","last_peered":"2024-04-20T10:27:38.392446+0000","last_clean":"2024-04-20T10:27:38.392446+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:27:38.392446+0000","last_undegraded":"2024-04-20T10:27:38.392446+0000","last_fullsized":"2024-04-20T10:27:38.392446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108582+0000","last_change":"2024-04-20T10:27:37.103228+0000","last_active":"2024-04-20T10:27:38.108582+0000","last_peered":"2024-04-20T10:27:38.108582+0000","last_clean":"2024-04-20T10:27:38.108582+0000","last_became_active":"2024-04-20T10:27:37.102768+0000","last_became_peered":"2024-04-20T10:27:37.102768+0000","last_unstale":"2024-04-20T10:27:38.108582+0000","last_undegraded":"2024-04-20T10:27:38.108582+0000","last_fullsized":"2024-04-20T10:27:38.108582+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:13.639017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108024+0000","last_change":"2024-04-20T10:27:33.014266+0000","last_active":"2024-04-20T10:27:38.108024+0000","last_peered":"2024-04-20T10:27:38.108024+0000","last_clean":"2024-04-20T10:27:38.108024+0000","last_became_active":"2024-04-20T10:27:33.013935+0000","last_became_peered":"2024-04-20T10:27:33.013935+0000","last_unstale":"2024-04-20T10:27:38.108024+0000","last_undegraded":"2024-04-20T10:27:38.108024+0000","last_fullsized":"2024-04-20T10:27:38.108024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:03.292296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109594+0000","last_change":"2024-04-20T10:27:32.078144+0000","last_active":"2024-04-20T10:27:38.109594+0000","last_peered":"2024-04-20T10:27:38.109594+0000","last_clean":"2024-04-20T10:27:38.109594+0000","last_became_active":"2024-04-20T10:27:32.075588+0000","last_became_peered":"2024-04-20T10:27:32.075588+0000","last_unstale":"2024-04-20T10:27:38.109594+0000","last_undegraded":"2024-04-20T10:27:38.109594+0000","last_fullsized":"2024-04-20T10:27:38.109594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390435+0000","last_change":"2024-04-20T10:27:37.128340+0000","last_active":"2024-04-20T10:27:38.390435+0000","last_peered":"2024-04-20T10:27:38.390435+0000","last_clean":"2024-04-20T10:27:38.390435+0000","last_became_active":"2024-04-20T10:27:37.127609+0000","last_became_peered":"2024-04-20T10:27:37.127609+0000","last_unstale":"2024-04-20T10:27:38.390435+0000","last_undegraded":"2024-04-20T10:27:38.390435+0000","last_fullsized":"2024-04-20T10:27:38.390435+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:41.767940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388918+0000","last_change":"2024-04-20T10:27:33.007624+0000","last_active":"2024-04-20T10:27:38.388918+0000","last_peered":"2024-04-20T10:27:38.388918+0000","last_clean":"2024-04-20T10:27:38.388918+0000","last_became_active":"2024-04-20T10:27:33.007319+0000","last_became_peered":"2024-04-20T10:27:33.007319+0000","last_unstale":"2024-04-20T10:27:38.388918+0000","last_undegraded":"2024-04-20T10:27:38.388918+0000","last_fullsized":"2024-04-20T10:27:38.388918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112959+0000","last_change":"2024-04-20T10:27:32.087470+0000","last_active":"2024-04-20T10:27:38.112959+0000","last_peered":"2024-04-20T10:27:38.112959+0000","last_clean":"2024-04-20T10:27:38.112959+0000","last_became_active":"2024-04-20T10:27:32.086045+0000","last_became_peered":"2024-04-20T10:27:32.086045+0000","last_unstale":"2024-04-20T10:27:38.112959+0000","last_undegraded":"2024-04-20T10:27:38.112959+0000","last_fullsized":"2024-04-20T10:27:38.112959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111234+0000","last_change":"2024-04-20T10:27:37.118303+0000","last_active":"2024-04-20T10:27:38.111234+0000","last_peered":"2024-04-20T10:27:38.111234+0000","last_clean":"2024-04-20T10:27:38.111234+0000","last_became_active":"2024-04-20T10:27:37.117875+0000","last_became_peered":"2024-04-20T10:27:37.117875+0000","last_unstale":"2024-04-20T10:27:38.111234+0000","last_undegraded":"2024-04-20T10:27:38.111234+0000","last_fullsized":"2024-04-20T10:27:38.111234+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389469+0000","last_change":"2024-04-20T10:27:33.021039+0000","last_active":"2024-04-20T10:27:38.389469+0000","last_peered":"2024-04-20T10:27:38.389469+0000","last_clean":"2024-04-20T10:27:38.389469+0000","last_became_active":"2024-04-20T10:27:33.020767+0000","last_became_peered":"2024-04-20T10:27:33.020767+0000","last_unstale":"2024-04-20T10:27:38.389469+0000","last_undegraded":"2024-04-20T10:27:38.389469+0000","last_fullsized":"2024-04-20T10:27:38.389469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392137+0000","last_change":"2024-04-20T10:27:32.091910+0000","last_active":"2024-04-20T10:27:38.392137+0000","last_peered":"2024-04-20T10:27:38.392137+0000","last_clean":"2024-04-20T10:27:38.392137+0000","last_became_active":"2024-04-20T10:27:32.091500+0000","last_became_peered":"2024-04-20T10:27:32.091500+0000","last_unstale":"2024-04-20T10:27:38.392137+0000","last_undegraded":"2024-04-20T10:27:38.392137+0000","last_fullsized":"2024-04-20T10:27:38.392137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:53:02.468594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105575+0000","last_change":"2024-04-20T10:27:37.119655+0000","last_active":"2024-04-20T10:27:38.105575+0000","last_peered":"2024-04-20T10:27:38.105575+0000","last_clean":"2024-04-20T10:27:38.105575+0000","last_became_active":"2024-04-20T10:27:37.119274+0000","last_became_peered":"2024-04-20T10:27:37.119274+0000","last_unstale":"2024-04-20T10:27:38.105575+0000","last_undegraded":"2024-04-20T10:27:38.105575+0000","last_fullsized":"2024-04-20T10:27:38.105575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:56:33.147822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392306+0000","last_change":"2024-04-20T10:27:33.003557+0000","last_active":"2024-04-20T10:27:38.392306+0000","last_peered":"2024-04-20T10:27:38.392306+0000","last_clean":"2024-04-20T10:27:38.392306+0000","last_became_active":"2024-04-20T10:27:33.001763+0000","last_became_peered":"2024-04-20T10:27:33.001763+0000","last_unstale":"2024-04-20T10:27:38.392306+0000","last_undegraded":"2024-04-20T10:27:38.392306+0000","last_fullsized":"2024-04-20T10:27:38.392306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:45.483619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389540+0000","last_change":"2024-04-20T10:27:32.090042+0000","last_active":"2024-04-20T10:27:38.389540+0000","last_peered":"2024-04-20T10:27:38.389540+0000","last_clean":"2024-04-20T10:27:38.389540+0000","last_became_active":"2024-04-20T10:27:32.089694+0000","last_became_peered":"2024-04-20T10:27:32.089694+0000","last_unstale":"2024-04-20T10:27:38.389540+0000","last_undegraded":"2024-04-20T10:27:38.389540+0000","last_fullsized":"2024-04-20T10:27:38.389540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393350+0000","last_change":"2024-04-20T10:27:37.125234+0000","last_active":"2024-04-20T10:27:38.393350+0000","last_peered":"2024-04-20T10:27:38.393350+0000","last_clean":"2024-04-20T10:27:38.393350+0000","last_became_active":"2024-04-20T10:27:37.124838+0000","last_became_peered":"2024-04-20T10:27:37.124838+0000","last_unstale":"2024-04-20T10:27:38.393350+0000","last_undegraded":"2024-04-20T10:27:38.393350+0000","last_fullsized":"2024-04-20T10:27:38.393350+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:47.071764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105563+0000","last_change":"2024-04-20T10:27:33.013769+0000","last_active":"2024-04-20T10:27:38.105563+0000","last_peered":"2024-04-20T10:27:38.105563+0000","last_clean":"2024-04-20T10:27:38.105563+0000","last_became_active":"2024-04-20T10:27:33.005662+0000","last_became_peered":"2024-04-20T10:27:33.005662+0000","last_unstale":"2024-04-20T10:27:38.105563+0000","last_undegraded":"2024-04-20T10:27:38.105563+0000","last_fullsized":"2024-04-20T10:27:38.105563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389725+0000","last_change":"2024-04-20T10:27:32.080478+0000","last_active":"2024-04-20T10:27:38.389725+0000","last_peered":"2024-04-20T10:27:38.389725+0000","last_clean":"2024-04-20T10:27:38.389725+0000","last_became_active":"2024-04-20T10:27:32.080148+0000","last_became_peered":"2024-04-20T10:27:32.080148+0000","last_unstale":"2024-04-20T10:27:38.389725+0000","last_undegraded":"2024-04-20T10:27:38.389725+0000","last_fullsized":"2024-04-20T10:27:38.389725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:17.004529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112793+0000","last_change":"2024-04-20T10:27:37.126177+0000","last_active":"2024-04-20T10:27:38.112793+0000","last_peered":"2024-04-20T10:27:38.112793+0000","last_clean":"2024-04-20T10:27:38.112793+0000","last_became_active":"2024-04-20T10:27:37.125862+0000","last_became_peered":"2024-04-20T10:27:37.125862+0000","last_unstale":"2024-04-20T10:27:38.112793+0000","last_undegraded":"2024-04-20T10:27:38.112793+0000","last_fullsized":"2024-04-20T10:27:38.112793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:07.395930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391792+0000","last_change":"2024-04-20T10:27:33.016106+0000","last_active":"2024-04-20T10:27:38.391792+0000","last_peered":"2024-04-20T10:27:38.391792+0000","last_clean":"2024-04-20T10:27:38.391792+0000","last_became_active":"2024-04-20T10:27:33.015787+0000","last_became_peered":"2024-04-20T10:27:33.015787+0000","last_unstale":"2024-04-20T10:27:38.391792+0000","last_undegraded":"2024-04-20T10:27:38.391792+0000","last_fullsized":"2024-04-20T10:27:38.391792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388035+0000","last_change":"2024-04-20T10:27:32.075511+0000","last_active":"2024-04-20T10:27:38.388035+0000","last_peered":"2024-04-20T10:27:38.388035+0000","last_clean":"2024-04-20T10:27:38.388035+0000","last_became_active":"2024-04-20T10:27:32.072523+0000","last_became_peered":"2024-04-20T10:27:32.072523+0000","last_unstale":"2024-04-20T10:27:38.388035+0000","last_undegraded":"2024-04-20T10:27:38.388035+0000","last_fullsized":"2024-04-20T10:27:38.388035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113117+0000","last_change":"2024-04-20T10:27:37.121568+0000","last_active":"2024-04-20T10:27:38.113117+0000","last_peered":"2024-04-20T10:27:38.113117+0000","last_clean":"2024-04-20T10:27:38.113117+0000","last_became_active":"2024-04-20T10:27:37.121105+0000","last_became_peered":"2024-04-20T10:27:37.121105+0000","last_unstale":"2024-04-20T10:27:38.113117+0000","last_undegraded":"2024-04-20T10:27:38.113117+0000","last_fullsized":"2024-04-20T10:27:38.113117+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:20.615032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107307+0000","last_change":"2024-04-20T10:27:33.008601+0000","last_active":"2024-04-20T10:27:38.107307+0000","last_peered":"2024-04-20T10:27:38.107307+0000","last_clean":"2024-04-20T10:27:38.107307+0000","last_became_active":"2024-04-20T10:27:33.008317+0000","last_became_peered":"2024-04-20T10:27:33.008317+0000","last_unstale":"2024-04-20T10:27:38.107307+0000","last_undegraded":"2024-04-20T10:27:38.107307+0000","last_fullsized":"2024-04-20T10:27:38.107307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:55.576815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106751+0000","last_change":"2024-04-20T10:27:32.081278+0000","last_active":"2024-04-20T10:27:38.106751+0000","last_peered":"2024-04-20T10:27:38.106751+0000","last_clean":"2024-04-20T10:27:38.106751+0000","last_became_active":"2024-04-20T10:27:32.080466+0000","last_became_peered":"2024-04-20T10:27:32.080466+0000","last_unstale":"2024-04-20T10:27:38.106751+0000","last_undegraded":"2024-04-20T10:27:38.106751+0000","last_fullsized":"2024-04-20T10:27:38.106751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:51.199449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109133+0000","last_change":"2024-04-20T10:27:37.132802+0000","last_active":"2024-04-20T10:27:38.109133+0000","last_peered":"2024-04-20T10:27:38.109133+0000","last_clean":"2024-04-20T10:27:38.109133+0000","last_became_active":"2024-04-20T10:27:37.129193+0000","last_became_peered":"2024-04-20T10:27:37.129193+0000","last_unstale":"2024-04-20T10:27:38.109133+0000","last_undegraded":"2024-04-20T10:27:38.109133+0000","last_fullsized":"2024-04-20T10:27:38.109133+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110211+0000","last_change":"2024-04-20T10:27:33.010100+0000","last_active":"2024-04-20T10:27:38.110211+0000","last_peered":"2024-04-20T10:27:38.110211+0000","last_clean":"2024-04-20T10:27:38.110211+0000","last_became_active":"2024-04-20T10:27:33.009703+0000","last_became_peered":"2024-04-20T10:27:33.009703+0000","last_unstale":"2024-04-20T10:27:38.110211+0000","last_undegraded":"2024-04-20T10:27:38.110211+0000","last_fullsized":"2024-04-20T10:27:38.110211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391899+0000","last_change":"2024-04-20T10:27:32.078724+0000","last_active":"2024-04-20T10:27:38.391899+0000","last_peered":"2024-04-20T10:27:38.391899+0000","last_clean":"2024-04-20T10:27:38.391899+0000","last_became_active":"2024-04-20T10:27:32.078093+0000","last_became_peered":"2024-04-20T10:27:32.078093+0000","last_unstale":"2024-04-20T10:27:38.391899+0000","last_undegraded":"2024-04-20T10:27:38.391899+0000","last_fullsized":"2024-04-20T10:27:38.391899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107857+0000","last_change":"2024-04-20T10:27:37.121822+0000","last_active":"2024-04-20T10:27:38.107857+0000","last_peered":"2024-04-20T10:27:38.107857+0000","last_clean":"2024-04-20T10:27:38.107857+0000","last_became_active":"2024-04-20T10:27:37.121423+0000","last_became_peered":"2024-04-20T10:27:37.121423+0000","last_unstale":"2024-04-20T10:27:38.107857+0000","last_undegraded":"2024-04-20T10:27:38.107857+0000","last_fullsized":"2024-04-20T10:27:38.107857+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:21:35.974697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388424+0000","last_change":"2024-04-20T10:27:33.001752+0000","last_active":"2024-04-20T10:27:38.388424+0000","last_peered":"2024-04-20T10:27:38.388424+0000","last_clean":"2024-04-20T10:27:38.388424+0000","last_became_active":"2024-04-20T10:27:33.001201+0000","last_became_peered":"2024-04-20T10:27:33.001201+0000","last_unstale":"2024-04-20T10:27:38.388424+0000","last_undegraded":"2024-04-20T10:27:38.388424+0000","last_fullsized":"2024-04-20T10:27:38.388424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:51.040746+0000","last_change":"2024-04-20T10:27:32.080328+0000","last_active":"2024-04-20T10:27:51.040746+0000","last_peered":"2024-04-20T10:27:51.040746+0000","last_clean":"2024-04-20T10:27:51.040746+0000","last_became_active":"2024-04-20T10:27:32.080011+0000","last_became_peered":"2024-04-20T10:27:32.080011+0000","last_unstale":"2024-04-20T10:27:51.040746+0000","last_undegraded":"2024-04-20T10:27:51.040746+0000","last_fullsized":"2024-04-20T10:27:51.040746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:22:14.097676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389991+0000","last_change":"2024-04-20T10:27:37.126061+0000","last_active":"2024-04-20T10:27:38.389991+0000","last_peered":"2024-04-20T10:27:38.389991+0000","last_clean":"2024-04-20T10:27:38.389991+0000","last_became_active":"2024-04-20T10:27:37.121461+0000","last_became_peered":"2024-04-20T10:27:37.121461+0000","last_unstale":"2024-04-20T10:27:38.389991+0000","last_undegraded":"2024-04-20T10:27:38.389991+0000","last_fullsized":"2024-04-20T10:27:38.389991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:13.289328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107516+0000","last_change":"2024-04-20T10:27:33.007572+0000","last_active":"2024-04-20T10:27:38.107516+0000","last_peered":"2024-04-20T10:27:38.107516+0000","last_clean":"2024-04-20T10:27:38.107516+0000","last_became_active":"2024-04-20T10:27:33.007206+0000","last_became_peered":"2024-04-20T10:27:33.007206+0000","last_unstale":"2024-04-20T10:27:38.107516+0000","last_undegraded":"2024-04-20T10:27:38.107516+0000","last_fullsized":"2024-04-20T10:27:38.107516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:21.846828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393065+0000","last_change":"2024-04-20T10:27:32.080090+0000","last_active":"2024-04-20T10:27:38.393065+0000","last_peered":"2024-04-20T10:27:38.393065+0000","last_clean":"2024-04-20T10:27:38.393065+0000","last_became_active":"2024-04-20T10:27:32.079763+0000","last_became_peered":"2024-04-20T10:27:32.079763+0000","last_unstale":"2024-04-20T10:27:38.393065+0000","last_undegraded":"2024-04-20T10:27:38.393065+0000","last_fullsized":"2024-04-20T10:27:38.393065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390646+0000","last_change":"2024-04-20T10:27:37.113593+0000","last_active":"2024-04-20T10:27:38.390646+0000","last_peered":"2024-04-20T10:27:38.390646+0000","last_clean":"2024-04-20T10:27:38.390646+0000","last_became_active":"2024-04-20T10:27:37.113016+0000","last_became_peered":"2024-04-20T10:27:37.113016+0000","last_unstale":"2024-04-20T10:27:38.390646+0000","last_undegraded":"2024-04-20T10:27:38.390646+0000","last_fullsized":"2024-04-20T10:27:38.390646+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:27.394166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113229+0000","last_change":"2024-04-20T10:27:33.020099+0000","last_active":"2024-04-20T10:27:38.113229+0000","last_peered":"2024-04-20T10:27:38.113229+0000","last_clean":"2024-04-20T10:27:38.113229+0000","last_became_active":"2024-04-20T10:27:33.018743+0000","last_became_peered":"2024-04-20T10:27:33.018743+0000","last_unstale":"2024-04-20T10:27:38.113229+0000","last_undegraded":"2024-04-20T10:27:38.113229+0000","last_fullsized":"2024-04-20T10:27:38.113229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389020+0000","last_change":"2024-04-20T10:27:32.081599+0000","last_active":"2024-04-20T10:27:38.389020+0000","last_peered":"2024-04-20T10:27:38.389020+0000","last_clean":"2024-04-20T10:27:38.389020+0000","last_became_active":"2024-04-20T10:27:32.081288+0000","last_became_peered":"2024-04-20T10:27:32.081288+0000","last_unstale":"2024-04-20T10:27:38.389020+0000","last_undegraded":"2024-04-20T10:27:38.389020+0000","last_fullsized":"2024-04-20T10:27:38.389020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391304+0000","last_change":"2024-04-20T10:27:37.134883+0000","last_active":"2024-04-20T10:27:38.391304+0000","last_peered":"2024-04-20T10:27:38.391304+0000","last_clean":"2024-04-20T10:27:38.391304+0000","last_became_active":"2024-04-20T10:27:37.134221+0000","last_became_peered":"2024-04-20T10:27:37.134221+0000","last_unstale":"2024-04-20T10:27:38.391304+0000","last_undegraded":"2024-04-20T10:27:38.391304+0000","last_fullsized":"2024-04-20T10:27:38.391304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:10.327991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392241+0000","last_change":"2024-04-20T10:27:33.006819+0000","last_active":"2024-04-20T10:27:38.392241+0000","last_peered":"2024-04-20T10:27:38.392241+0000","last_clean":"2024-04-20T10:27:38.392241+0000","last_became_active":"2024-04-20T10:27:33.005657+0000","last_became_peered":"2024-04-20T10:27:33.005657+0000","last_unstale":"2024-04-20T10:27:38.392241+0000","last_undegraded":"2024-04-20T10:27:38.392241+0000","last_fullsized":"2024-04-20T10:27:38.392241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391630+0000","last_change":"2024-04-20T10:27:32.083786+0000","last_active":"2024-04-20T10:27:38.391630+0000","last_peered":"2024-04-20T10:27:38.391630+0000","last_clean":"2024-04-20T10:27:38.391630+0000","last_became_active":"2024-04-20T10:27:32.083256+0000","last_became_peered":"2024-04-20T10:27:32.083256+0000","last_unstale":"2024-04-20T10:27:38.391630+0000","last_undegraded":"2024-04-20T10:27:38.391630+0000","last_fullsized":"2024-04-20T10:27:38.391630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:11.023664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390075+0000","last_change":"2024-04-20T10:27:37.131051+0000","last_active":"2024-04-20T10:27:38.390075+0000","last_peered":"2024-04-20T10:27:38.390075+0000","last_clean":"2024-04-20T10:27:38.390075+0000","last_became_active":"2024-04-20T10:27:37.130723+0000","last_became_peered":"2024-04-20T10:27:37.130723+0000","last_unstale":"2024-04-20T10:27:38.390075+0000","last_undegraded":"2024-04-20T10:27:38.390075+0000","last_fullsized":"2024-04-20T10:27:38.390075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:28.980316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391821+0000","last_change":"2024-04-20T10:27:37.124930+0000","last_active":"2024-04-20T10:27:38.391821+0000","last_peered":"2024-04-20T10:27:38.391821+0000","last_clean":"2024-04-20T10:27:38.391821+0000","last_became_active":"2024-04-20T10:27:37.123089+0000","last_became_peered":"2024-04-20T10:27:37.123089+0000","last_unstale":"2024-04-20T10:27:38.391821+0000","last_undegraded":"2024-04-20T10:27:38.391821+0000","last_fullsized":"2024-04-20T10:27:38.391821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:05.113291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390231+0000","last_change":"2024-04-20T10:27:33.020003+0000","last_active":"2024-04-20T10:27:38.390231+0000","last_peered":"2024-04-20T10:27:38.390231+0000","last_clean":"2024-04-20T10:27:38.390231+0000","last_became_active":"2024-04-20T10:27:33.019709+0000","last_became_peered":"2024-04-20T10:27:33.019709+0000","last_unstale":"2024-04-20T10:27:38.390231+0000","last_undegraded":"2024-04-20T10:27:38.390231+0000","last_fullsized":"2024-04-20T10:27:38.390231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:06.663893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106161+0000","last_change":"2024-04-20T10:27:32.070067+0000","last_active":"2024-04-20T10:27:38.106161+0000","last_peered":"2024-04-20T10:27:38.106161+0000","last_clean":"2024-04-20T10:27:38.106161+0000","last_became_active":"2024-04-20T10:27:32.069761+0000","last_became_peered":"2024-04-20T10:27:32.069761+0000","last_unstale":"2024-04-20T10:27:38.106161+0000","last_undegraded":"2024-04-20T10:27:38.106161+0000","last_fullsized":"2024-04-20T10:27:38.106161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108148+0000","last_change":"2024-04-20T10:27:37.125295+0000","last_active":"2024-04-20T10:27:38.108148+0000","last_peered":"2024-04-20T10:27:38.108148+0000","last_clean":"2024-04-20T10:27:38.108148+0000","last_became_active":"2024-04-20T10:27:37.124922+0000","last_became_peered":"2024-04-20T10:27:37.124922+0000","last_unstale":"2024-04-20T10:27:38.108148+0000","last_undegraded":"2024-04-20T10:27:38.108148+0000","last_fullsized":"2024-04-20T10:27:38.108148+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:45.803688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108085+0000","last_change":"2024-04-20T10:27:33.009010+0000","last_active":"2024-04-20T10:27:38.108085+0000","last_peered":"2024-04-20T10:27:38.108085+0000","last_clean":"2024-04-20T10:27:38.108085+0000","last_became_active":"2024-04-20T10:27:33.008701+0000","last_became_peered":"2024-04-20T10:27:33.008701+0000","last_unstale":"2024-04-20T10:27:38.108085+0000","last_undegraded":"2024-04-20T10:27:38.108085+0000","last_fullsized":"2024-04-20T10:27:38.108085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109186+0000","last_change":"2024-04-20T10:27:32.079095+0000","last_active":"2024-04-20T10:27:38.109186+0000","last_peered":"2024-04-20T10:27:38.109186+0000","last_clean":"2024-04-20T10:27:38.109186+0000","last_became_active":"2024-04-20T10:27:32.074687+0000","last_became_peered":"2024-04-20T10:27:32.074687+0000","last_unstale":"2024-04-20T10:27:38.109186+0000","last_undegraded":"2024-04-20T10:27:38.109186+0000","last_fullsized":"2024-04-20T10:27:38.109186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106749+0000","last_change":"2024-04-20T10:27:32.080751+0000","last_active":"2024-04-20T10:27:38.106749+0000","last_peered":"2024-04-20T10:27:38.106749+0000","last_clean":"2024-04-20T10:27:38.106749+0000","last_became_active":"2024-04-20T10:27:32.078604+0000","last_became_peered":"2024-04-20T10:27:32.078604+0000","last_unstale":"2024-04-20T10:27:38.106749+0000","last_undegraded":"2024-04-20T10:27:38.106749+0000","last_fullsized":"2024-04-20T10:27:38.106749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:10.874813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390089+0000","last_change":"2024-04-20T10:27:33.024156+0000","last_active":"2024-04-20T10:27:38.390089+0000","last_peered":"2024-04-20T10:27:38.390089+0000","last_clean":"2024-04-20T10:27:38.390089+0000","last_became_active":"2024-04-20T10:27:33.023639+0000","last_became_peered":"2024-04-20T10:27:33.023639+0000","last_unstale":"2024-04-20T10:27:38.390089+0000","last_undegraded":"2024-04-20T10:27:38.390089+0000","last_fullsized":"2024-04-20T10:27:38.390089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:45.132088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391353+0000","last_change":"2024-04-20T10:27:37.136584+0000","last_active":"2024-04-20T10:27:38.391353+0000","last_peered":"2024-04-20T10:27:38.391353+0000","last_clean":"2024-04-20T10:27:38.391353+0000","last_became_active":"2024-04-20T10:27:37.135997+0000","last_became_peered":"2024-04-20T10:27:37.135997+0000","last_unstale":"2024-04-20T10:27:38.391353+0000","last_undegraded":"2024-04-20T10:27:38.391353+0000","last_fullsized":"2024-04-20T10:27:38.391353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:27.651374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111655+0000","last_change":"2024-04-20T10:27:32.090018+0000","last_active":"2024-04-20T10:27:38.111655+0000","last_peered":"2024-04-20T10:27:38.111655+0000","last_clean":"2024-04-20T10:27:38.111655+0000","last_became_active":"2024-04-20T10:27:32.089144+0000","last_became_peered":"2024-04-20T10:27:32.089144+0000","last_unstale":"2024-04-20T10:27:38.111655+0000","last_undegraded":"2024-04-20T10:27:38.111655+0000","last_fullsized":"2024-04-20T10:27:38.111655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390744+0000","last_change":"2024-04-20T10:27:33.010778+0000","last_active":"2024-04-20T10:27:38.390744+0000","last_peered":"2024-04-20T10:27:38.390744+0000","last_clean":"2024-04-20T10:27:38.390744+0000","last_became_active":"2024-04-20T10:27:33.010403+0000","last_became_peered":"2024-04-20T10:27:33.010403+0000","last_unstale":"2024-04-20T10:27:38.390744+0000","last_undegraded":"2024-04-20T10:27:38.390744+0000","last_fullsized":"2024-04-20T10:27:38.390744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393172+0000","last_change":"2024-04-20T10:27:37.134702+0000","last_active":"2024-04-20T10:27:38.393172+0000","last_peered":"2024-04-20T10:27:38.393172+0000","last_clean":"2024-04-20T10:27:38.393172+0000","last_became_active":"2024-04-20T10:27:37.134178+0000","last_became_peered":"2024-04-20T10:27:37.134178+0000","last_unstale":"2024-04-20T10:27:38.393172+0000","last_undegraded":"2024-04-20T10:27:38.393172+0000","last_fullsized":"2024-04-20T10:27:38.393172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:48.722800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390536+0000","last_change":"2024-04-20T10:27:32.998168+0000","last_active":"2024-04-20T10:27:38.390536+0000","last_peered":"2024-04-20T10:27:38.390536+0000","last_clean":"2024-04-20T10:27:38.390536+0000","last_became_active":"2024-04-20T10:27:32.996843+0000","last_became_peered":"2024-04-20T10:27:32.996843+0000","last_unstale":"2024-04-20T10:27:38.390536+0000","last_undegraded":"2024-04-20T10:27:38.390536+0000","last_fullsized":"2024-04-20T10:27:38.390536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:13.408788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391052+0000","last_change":"2024-04-20T10:27:33.020874+0000","last_active":"2024-04-20T10:27:38.391052+0000","last_peered":"2024-04-20T10:27:38.391052+0000","last_clean":"2024-04-20T10:27:38.391052+0000","last_became_active":"2024-04-20T10:27:33.016375+0000","last_became_peered":"2024-04-20T10:27:33.016375+0000","last_unstale":"2024-04-20T10:27:38.391052+0000","last_undegraded":"2024-04-20T10:27:38.391052+0000","last_fullsized":"2024-04-20T10:27:38.391052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105932+0000","last_change":"2024-04-20T10:27:33.011331+0000","last_active":"2024-04-20T10:27:38.105932+0000","last_peered":"2024-04-20T10:27:38.105932+0000","last_clean":"2024-04-20T10:27:38.105932+0000","last_became_active":"2024-04-20T10:27:33.010956+0000","last_became_peered":"2024-04-20T10:27:33.010956+0000","last_unstale":"2024-04-20T10:27:38.105932+0000","last_undegraded":"2024-04-20T10:27:38.105932+0000","last_fullsized":"2024-04-20T10:27:38.105932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:16.511398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111379+0000","last_change":"2024-04-20T10:27:33.009386+0000","last_active":"2024-04-20T10:27:38.111379+0000","last_peered":"2024-04-20T10:27:38.111379+0000","last_clean":"2024-04-20T10:27:38.111379+0000","last_became_active":"2024-04-20T10:27:33.008596+0000","last_became_peered":"2024-04-20T10:27:33.008596+0000","last_unstale":"2024-04-20T10:27:38.111379+0000","last_undegraded":"2024-04-20T10:27:38.111379+0000","last_fullsized":"2024-04-20T10:27:38.111379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109220+0000","last_change":"2024-04-20T10:27:33.001780+0000","last_active":"2024-04-20T10:27:38.109220+0000","last_peered":"2024-04-20T10:27:38.109220+0000","last_clean":"2024-04-20T10:27:38.109220+0000","last_became_active":"2024-04-20T10:27:33.000852+0000","last_became_peered":"2024-04-20T10:27:33.000852+0000","last_unstale":"2024-04-20T10:27:38.109220+0000","last_undegraded":"2024-04-20T10:27:38.109220+0000","last_fullsized":"2024-04-20T10:27:38.109220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:52.213855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111488+0000","last_change":"2024-04-20T10:27:33.010234+0000","last_active":"2024-04-20T10:27:38.111488+0000","last_peered":"2024-04-20T10:27:38.111488+0000","last_clean":"2024-04-20T10:27:38.111488+0000","last_became_active":"2024-04-20T10:27:33.009934+0000","last_became_peered":"2024-04-20T10:27:33.009934+0000","last_unstale":"2024-04-20T10:27:38.111488+0000","last_undegraded":"2024-04-20T10:27:38.111488+0000","last_fullsized":"2024-04-20T10:27:38.111488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392171+0000","last_change":"2024-04-20T10:27:33.006041+0000","last_active":"2024-04-20T10:27:38.392171+0000","last_peered":"2024-04-20T10:27:38.392171+0000","last_clean":"2024-04-20T10:27:38.392171+0000","last_became_active":"2024-04-20T10:27:33.005259+0000","last_became_peered":"2024-04-20T10:27:33.005259+0000","last_unstale":"2024-04-20T10:27:38.392171+0000","last_undegraded":"2024-04-20T10:27:38.392171+0000","last_fullsized":"2024-04-20T10:27:38.392171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112860+0000","last_change":"2024-04-20T10:27:33.015530+0000","last_active":"2024-04-20T10:27:38.112860+0000","last_peered":"2024-04-20T10:27:38.112860+0000","last_clean":"2024-04-20T10:27:38.112860+0000","last_became_active":"2024-04-20T10:27:33.015240+0000","last_became_peered":"2024-04-20T10:27:33.015240+0000","last_unstale":"2024-04-20T10:27:38.112860+0000","last_undegraded":"2024-04-20T10:27:38.112860+0000","last_fullsized":"2024-04-20T10:27:38.112860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105792+0000","last_change":"2024-04-20T10:27:33.015916+0000","last_active":"2024-04-20T10:27:38.105792+0000","last_peered":"2024-04-20T10:27:38.105792+0000","last_clean":"2024-04-20T10:27:38.105792+0000","last_became_active":"2024-04-20T10:27:33.015585+0000","last_became_peered":"2024-04-20T10:27:33.015585+0000","last_unstale":"2024-04-20T10:27:38.105792+0000","last_undegraded":"2024-04-20T10:27:38.105792+0000","last_fullsized":"2024-04-20T10:27:38.105792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112167+0000","last_change":"2024-04-20T10:27:33.014307+0000","last_active":"2024-04-20T10:27:38.112167+0000","last_peered":"2024-04-20T10:27:38.112167+0000","last_clean":"2024-04-20T10:27:38.112167+0000","last_became_active":"2024-04-20T10:27:33.013891+0000","last_became_peered":"2024-04-20T10:27:33.013891+0000","last_unstale":"2024-04-20T10:27:38.112167+0000","last_undegraded":"2024-04-20T10:27:38.112167+0000","last_fullsized":"2024-04-20T10:27:38.112167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:32.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390622+0000","last_change":"2024-04-20T10:27:33.010713+0000","last_active":"2024-04-20T10:27:38.390622+0000","last_peered":"2024-04-20T10:27:38.390622+0000","last_clean":"2024-04-20T10:27:38.390622+0000","last_became_active":"2024-04-20T10:27:33.009349+0000","last_became_peered":"2024-04-20T10:27:33.009349+0000","last_unstale":"2024-04-20T10:27:38.390622+0000","last_undegraded":"2024-04-20T10:27:38.390622+0000","last_fullsized":"2024-04-20T10:27:38.390622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389798+0000","last_change":"2024-04-20T10:27:33.018481+0000","last_active":"2024-04-20T10:27:38.389798+0000","last_peered":"2024-04-20T10:27:38.389798+0000","last_clean":"2024-04-20T10:27:38.389798+0000","last_became_active":"2024-04-20T10:27:33.015328+0000","last_became_peered":"2024-04-20T10:27:33.015328+0000","last_unstale":"2024-04-20T10:27:38.389798+0000","last_undegraded":"2024-04-20T10:27:38.389798+0000","last_fullsized":"2024-04-20T10:27:38.389798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:43:48.722150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388831+0000","last_change":"2024-04-20T10:27:33.006877+0000","last_active":"2024-04-20T10:27:38.388831+0000","last_peered":"2024-04-20T10:27:38.388831+0000","last_clean":"2024-04-20T10:27:38.388831+0000","last_became_active":"2024-04-20T10:27:33.006143+0000","last_became_peered":"2024-04-20T10:27:33.006143+0000","last_unstale":"2024-04-20T10:27:38.388831+0000","last_undegraded":"2024-04-20T10:27:38.388831+0000","last_fullsized":"2024-04-20T10:27:38.388831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108600+0000","last_change":"2024-04-20T10:27:33.016666+0000","last_active":"2024-04-20T10:27:38.108600+0000","last_peered":"2024-04-20T10:27:38.108600+0000","last_clean":"2024-04-20T10:27:38.108600+0000","last_became_active":"2024-04-20T10:27:33.016268+0000","last_became_peered":"2024-04-20T10:27:33.016268+0000","last_unstale":"2024-04-20T10:27:38.108600+0000","last_undegraded":"2024-04-20T10:27:38.108600+0000","last_fullsized":"2024-04-20T10:27:38.108600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:07.386901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392668+0000","last_change":"2024-04-20T10:27:33.014742+0000","last_active":"2024-04-20T10:27:38.392668+0000","last_peered":"2024-04-20T10:27:38.392668+0000","last_clean":"2024-04-20T10:27:38.392668+0000","last_became_active":"2024-04-20T10:27:33.014443+0000","last_became_peered":"2024-04-20T10:27:33.014443+0000","last_unstale":"2024-04-20T10:27:38.392668+0000","last_undegraded":"2024-04-20T10:27:38.392668+0000","last_fullsized":"2024-04-20T10:27:38.392668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108659+0000","last_change":"2024-04-20T10:27:33.002993+0000","last_active":"2024-04-20T10:27:38.108659+0000","last_peered":"2024-04-20T10:27:38.108659+0000","last_clean":"2024-04-20T10:27:38.108659+0000","last_became_active":"2024-04-20T10:27:33.002685+0000","last_became_peered":"2024-04-20T10:27:33.002685+0000","last_unstale":"2024-04-20T10:27:38.108659+0000","last_undegraded":"2024-04-20T10:27:38.108659+0000","last_fullsized":"2024-04-20T10:27:38.108659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:38.509857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105782+0000","last_change":"2024-04-20T10:27:33.014659+0000","last_active":"2024-04-20T10:27:38.105782+0000","last_peered":"2024-04-20T10:27:38.105782+0000","last_clean":"2024-04-20T10:27:38.105782+0000","last_became_active":"2024-04-20T10:27:33.014188+0000","last_became_peered":"2024-04-20T10:27:33.014188+0000","last_unstale":"2024-04-20T10:27:38.105782+0000","last_undegraded":"2024-04-20T10:27:38.105782+0000","last_fullsized":"2024-04-20T10:27:38.105782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107794+0000","last_change":"2024-04-20T10:27:33.011041+0000","last_active":"2024-04-20T10:27:38.107794+0000","last_peered":"2024-04-20T10:27:38.107794+0000","last_clean":"2024-04-20T10:27:38.107794+0000","last_became_active":"2024-04-20T10:27:33.010751+0000","last_became_peered":"2024-04-20T10:27:33.010751+0000","last_unstale":"2024-04-20T10:27:38.107794+0000","last_undegraded":"2024-04-20T10:27:38.107794+0000","last_fullsized":"2024-04-20T10:27:38.107794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110132+0000","last_change":"2024-04-20T10:27:33.005787+0000","last_active":"2024-04-20T10:27:38.110132+0000","last_peered":"2024-04-20T10:27:38.110132+0000","last_clean":"2024-04-20T10:27:38.110132+0000","last_became_active":"2024-04-20T10:27:33.004692+0000","last_became_peered":"2024-04-20T10:27:33.004692+0000","last_unstale":"2024-04-20T10:27:38.110132+0000","last_undegraded":"2024-04-20T10:27:38.110132+0000","last_fullsized":"2024-04-20T10:27:38.110132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:08.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112010+0000","last_change":"2024-04-20T10:27:33.010711+0000","last_active":"2024-04-20T10:27:38.112010+0000","last_peered":"2024-04-20T10:27:38.112010+0000","last_clean":"2024-04-20T10:27:38.112010+0000","last_became_active":"2024-04-20T10:27:33.010282+0000","last_became_peered":"2024-04-20T10:27:33.010282+0000","last_unstale":"2024-04-20T10:27:38.112010+0000","last_undegraded":"2024-04-20T10:27:38.112010+0000","last_fullsized":"2024-04-20T10:27:38.112010+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:06.224357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388721+0000","last_change":"2024-04-20T10:27:33.008425+0000","last_active":"2024-04-20T10:27:38.388721+0000","last_peered":"2024-04-20T10:27:38.388721+0000","last_clean":"2024-04-20T10:27:38.388721+0000","last_became_active":"2024-04-20T10:27:33.006532+0000","last_became_peered":"2024-04-20T10:27:33.006532+0000","last_unstale":"2024-04-20T10:27:38.388721+0000","last_undegraded":"2024-04-20T10:27:38.388721+0000","last_fullsized":"2024-04-20T10:27:38.388721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388300+0000","last_change":"2024-04-20T10:27:33.018484+0000","last_active":"2024-04-20T10:27:38.388300+0000","last_peered":"2024-04-20T10:27:38.388300+0000","last_clean":"2024-04-20T10:27:38.388300+0000","last_became_active":"2024-04-20T10:27:33.018257+0000","last_became_peered":"2024-04-20T10:27:33.018257+0000","last_unstale":"2024-04-20T10:27:38.388300+0000","last_undegraded":"2024-04-20T10:27:38.388300+0000","last_fullsized":"2024-04-20T10:27:38.388300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388715+0000","last_change":"2024-04-20T10:27:33.022995+0000","last_active":"2024-04-20T10:27:38.388715+0000","last_peered":"2024-04-20T10:27:38.388715+0000","last_clean":"2024-04-20T10:27:38.388715+0000","last_became_active":"2024-04-20T10:27:33.017740+0000","last_became_peered":"2024-04-20T10:27:33.017740+0000","last_unstale":"2024-04-20T10:27:38.388715+0000","last_undegraded":"2024-04-20T10:27:38.388715+0000","last_fullsized":"2024-04-20T10:27:38.388715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391590+0000","last_change":"2024-04-20T10:27:33.008312+0000","last_active":"2024-04-20T10:27:38.391590+0000","last_peered":"2024-04-20T10:27:38.391590+0000","last_clean":"2024-04-20T10:27:38.391590+0000","last_became_active":"2024-04-20T10:27:33.002597+0000","last_became_peered":"2024-04-20T10:27:33.002597+0000","last_unstale":"2024-04-20T10:27:38.391590+0000","last_undegraded":"2024-04-20T10:27:38.391590+0000","last_fullsized":"2024-04-20T10:27:38.391590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107489+0000","last_change":"2024-04-20T10:27:33.010109+0000","last_active":"2024-04-20T10:27:38.107489+0000","last_peered":"2024-04-20T10:27:38.107489+0000","last_clean":"2024-04-20T10:27:38.107489+0000","last_became_active":"2024-04-20T10:27:33.009838+0000","last_became_peered":"2024-04-20T10:27:33.009838+0000","last_unstale":"2024-04-20T10:27:38.107489+0000","last_undegraded":"2024-04-20T10:27:38.107489+0000","last_fullsized":"2024-04-20T10:27:38.107489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387903+0000","last_change":"2024-04-20T10:27:33.024641+0000","last_active":"2024-04-20T10:27:38.387903+0000","last_peered":"2024-04-20T10:27:38.387903+0000","last_clean":"2024-04-20T10:27:38.387903+0000","last_became_active":"2024-04-20T10:27:33.024410+0000","last_became_peered":"2024-04-20T10:27:33.024410+0000","last_unstale":"2024-04-20T10:27:38.387903+0000","last_undegraded":"2024-04-20T10:27:38.387903+0000","last_fullsized":"2024-04-20T10:27:38.387903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391138+0000","last_change":"2024-04-20T10:27:33.025226+0000","last_active":"2024-04-20T10:27:38.391138+0000","last_peered":"2024-04-20T10:27:38.391138+0000","last_clean":"2024-04-20T10:27:38.391138+0000","last_became_active":"2024-04-20T10:27:33.024311+0000","last_became_peered":"2024-04-20T10:27:33.024311+0000","last_unstale":"2024-04-20T10:27:38.391138+0000","last_undegraded":"2024-04-20T10:27:38.391138+0000","last_fullsized":"2024-04-20T10:27:38.391138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389207+0000","last_change":"2024-04-20T10:27:33.022586+0000","last_active":"2024-04-20T10:27:38.389207+0000","last_peered":"2024-04-20T10:27:38.389207+0000","last_clean":"2024-04-20T10:27:38.389207+0000","last_became_active":"2024-04-20T10:27:33.022293+0000","last_became_peered":"2024-04-20T10:27:33.022293+0000","last_unstale":"2024-04-20T10:27:38.389207+0000","last_undegraded":"2024-04-20T10:27:38.389207+0000","last_fullsized":"2024-04-20T10:27:38.389207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390946+0000","last_change":"2024-04-20T10:27:33.019615+0000","last_active":"2024-04-20T10:27:38.390946+0000","last_peered":"2024-04-20T10:27:38.390946+0000","last_clean":"2024-04-20T10:27:38.390946+0000","last_became_active":"2024-04-20T10:27:33.019288+0000","last_became_peered":"2024-04-20T10:27:33.019288+0000","last_unstale":"2024-04-20T10:27:38.390946+0000","last_undegraded":"2024-04-20T10:27:38.390946+0000","last_fullsized":"2024-04-20T10:27:38.390946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111457+0000","last_change":"2024-04-20T10:27:33.019815+0000","last_active":"2024-04-20T10:27:38.111457+0000","last_peered":"2024-04-20T10:27:38.111457+0000","last_clean":"2024-04-20T10:27:38.111457+0000","last_became_active":"2024-04-20T10:27:33.019369+0000","last_became_peered":"2024-04-20T10:27:33.019369+0000","last_unstale":"2024-04-20T10:27:38.111457+0000","last_undegraded":"2024-04-20T10:27:38.111457+0000","last_fullsized":"2024-04-20T10:27:38.111457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112358+0000","last_change":"2024-04-20T10:27:33.015773+0000","last_active":"2024-04-20T10:27:38.112358+0000","last_peered":"2024-04-20T10:27:38.112358+0000","last_clean":"2024-04-20T10:27:38.112358+0000","last_became_active":"2024-04-20T10:27:33.015430+0000","last_became_peered":"2024-04-20T10:27:33.015430+0000","last_unstale":"2024-04-20T10:27:38.112358+0000","last_undegraded":"2024-04-20T10:27:38.112358+0000","last_fullsized":"2024-04-20T10:27:38.112358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389282+0000","last_change":"2024-04-20T10:27:33.024105+0000","last_active":"2024-04-20T10:27:38.389282+0000","last_peered":"2024-04-20T10:27:38.389282+0000","last_clean":"2024-04-20T10:27:38.389282+0000","last_became_active":"2024-04-20T10:27:33.023418+0000","last_became_peered":"2024-04-20T10:27:33.023418+0000","last_unstale":"2024-04-20T10:27:38.389282+0000","last_undegraded":"2024-04-20T10:27:38.389282+0000","last_fullsized":"2024-04-20T10:27:38.389282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":110867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":700723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":114799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:28:01.421 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:28:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.668+0000 7f4b97e00640 1 Processor -- start 2024-04-20T10:28:01.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.669+0000 7f4b97e00640 1 -- start start 2024-04-20T10:28:01.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.670+0000 7f4b97e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4b98159e10 0x7f4b98071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.670+0000 7f4b97e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b98160e50 con 0x7f4b9815a1f0 2024-04-20T10:28:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.670+0000 7f4b97e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b9815cc90 con 0x7f4b9806da90 2024-04-20T10:28:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.670+0000 7f4b97e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4b98072900 con 0x7f4b98159e10 2024-04-20T10:28:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.671+0000 7f4b8e000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4b9815a1f0 0x7f4b98160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33796/0 (socket says 172.21.15.47:33796) 2024-04-20T10:28:01.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.671+0000 7f4b8e000640 1 -- 172.21.15.47:0/1908686065 learned_addr learned my addr 172.21.15.47:0/1908686065 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.671+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1908686065 >> v2:172.21.15.175:3300/0 conn(0x7f4b98159e10 0x7f4b98071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.671+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 820243946 0 0) 0x7f4b98160e50 con 0x7f4b9815a1f0 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.671+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4b7c003520 con 0x7f4b9815a1f0 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 63526539 0 0) 0x7f4b7c003520 con 0x7f4b9815a1f0 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 >> v2:172.21.15.175:3300/0 conn(0x7f4b98159e10 msgr2=0x7f4b98071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 --2- 172.21.15.47:0/1908686065 >> v2:172.21.15.175:3300/0 conn(0x7f4b98159e10 0x7f4b98071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 >> v1:172.21.15.175:6789/0 conn(0x7f4b9806da90 legacy=0x7f4b98160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9815cf70 con 0x7f4b9815a1f0 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f4b80001670 con 0x7f4b9815a1f0 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 >> v1:172.21.15.47:6789/0 conn(0x7f4b9815a1f0 legacy=0x7f4b98160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c0039b0 0x7f4b7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b7c004800 0x7f4b7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0051c0 0x7f4b7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c00c030 con 0x7f4b7c0039b0 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b7c00c1e0 con 0x7f4b7c0051c0 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b7c00c3e0 con 0x7f4b7c004800 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.672+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c0039b0 0x7f4b7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b97400640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0051c0 0x7f4b7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b8e000640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b7c004800 0x7f4b7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b96a00640 1 -- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b7c004800 msgr2=0x7f4b7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b7c004800 0x7f4b7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b96a00640 1 -- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0051c0 msgr2=0x7f4b7c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0051c0 0x7f4b7c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b96a00640 1 -- 172.21.15.47:0/1908686065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7c003520 con 0x7f4b7c0039b0 2024-04-20T10:28:01.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.673+0000 7f4b97400640 1 -- 172.21.15.47:0/1908686065 reap_dead start 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.674+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c0039b0 0x7f4b7c003dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fc0 tx=0x7f4b7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8147a1e706c5673 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.674+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88039070 con 0x7f4b7c0039b0 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.674+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b8802cc20 con 0x7f4b7c0039b0 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.674+0000 7f4b96000640 1 -- 172.21.15.47:0/1908686065 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b88009c10 con 0x7f4b7c0039b0 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.675+0000 7f4b97e00640 1 -- 172.21.15.47:0/1908686065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c0039b0 msgr2=0x7f4b7c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.675+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c0039b0 0x7f4b7c003dd0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4b88009fc0 tx=0x7f4b7c005830 comp rx=0 tx=0).stop 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.675+0000 7f4b97e00640 1 -- 172.21.15.47:0/1908686065 shutdown_connections 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.675+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1908686065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b7c0039b0 0x7f4b7c003dd0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.675+0000 7f4b97e00640 1 -- 172.21.15.47:0/1908686065 >> 172.21.15.47:0/1908686065 conn(0x7f4b980d0040 msgr2=0x7f4b980d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.675+0000 7f4b97e00640 1 -- 172.21.15.47:0/1908686065 shutdown_connections 2024-04-20T10:28:01.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.675+0000 7f4b97e00640 1 -- 172.21.15.47:0/1908686065 wait complete. 2024-04-20T10:28:01.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.677+0000 7f4b97e00640 1 Processor -- start 2024-04-20T10:28:01.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.677+0000 7f4b97e00640 1 -- start start 2024-04-20T10:28:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b97e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0039b0 0x7f4b9814cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b97e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98157fc0 0x7f4b980cae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b97e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b980ca250 0x7f4b980b40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b97e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9815d530 con 0x7f4b98157fc0 2024-04-20T10:28:01.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b97e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b9815ca40 con 0x7f4b7c0039b0 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b97e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b9815d810 con 0x7f4b980ca250 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b8e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98157fc0 0x7f4b980cae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b8e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98157fc0 0x7f4b980cae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56236/0 (socket says 172.21.15.47:56236) 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b8e000640 1 -- 172.21.15.47:0/1293396096 learned_addr learned my addr 172.21.15.47:0/1293396096 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.678+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0039b0 0x7f4b9814cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b97400640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b980ca250 0x7f4b980b40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8e000640 1 -- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b980ca250 msgr2=0x7f4b980b40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8e000640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b980ca250 0x7f4b980b40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8e000640 1 -- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0039b0 msgr2=0x7f4b9814cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8e000640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0039b0 0x7f4b9814cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8e000640 1 -- 172.21.15.47:0/1293396096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b9814d060 con 0x7f4b98157fc0 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0039b0 0x7f4b9814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8e000640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98157fc0 0x7f4b980cae30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000e710 tx=0x7f4b8000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:01.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8f400640 1 -- 172.21.15.47:0/1293396096 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000ee20 con 0x7f4b98157fc0 2024-04-20T10:28:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8f400640 1 -- 172.21.15.47:0/1293396096 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b80018070 con 0x7f4b98157fc0 2024-04-20T10:28:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b8f400640 1 -- 172.21.15.47:0/1293396096 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b8000cab0 con 0x7f4b98157fc0 2024-04-20T10:28:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b980c9490 con 0x7f4b98157fc0 2024-04-20T10:28:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.679+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b980c9a50 con 0x7f4b98157fc0 2024-04-20T10:28:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.681+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b980727e0 con 0x7f4b98157fc0 2024-04-20T10:28:01.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.681+0000 7f4b8f400640 1 -- 172.21.15.47:0/1293396096 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b8000cc50 con 0x7f4b98157fc0 2024-04-20T10:28:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.682+0000 7f4b8f400640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4b7406c8f0 0x7f4b7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:01.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.682+0000 7f4b8f400640 1 -- 172.21.15.47:0/1293396096 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f4b8008bd60 con 0x7f4b98157fc0 2024-04-20T10:28:01.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.683+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4b7406c8f0 0x7f4b7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:01.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.684+0000 7f4b96a00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4b7406c8f0 0x7f4b7406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b88036000 tx=0x7f4b8802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:01.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.687+0000 7f4b8f400640 1 -- 172.21.15.47:0/1293396096 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4b980727e0 con 0x7f4b98157fc0 2024-04-20T10:28:01.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.880+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4b980b0240 con 0x7f4b7406c8f0 2024-04-20T10:28:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.903+0000 7f4b8f400640 1 -- 172.21.15.47:0/1293396096 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332534 (secure 0 0 0) 0x7f4b980b0240 con 0x7f4b7406c8f0 2024-04-20T10:28:01.908 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:28:01.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.909+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4b7406c8f0 msgr2=0x7f4b7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.909+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4b7406c8f0 0x7f4b7406edb0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4b88036000 tx=0x7f4b8802e000 comp rx=0 tx=0).stop 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.909+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98157fc0 msgr2=0x7f4b980cae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.909+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98157fc0 0x7f4b980cae30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4b8000e710 tx=0x7f4b8000ebd0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 shutdown_connections 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b980ca250 0x7f4b980b40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4b7406c8f0 0x7f4b7406edb0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b98157fc0 0x7f4b980cae30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 --2- 172.21.15.47:0/1293396096 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b7c0039b0 0x7f4b9814cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 >> 172.21.15.47:0/1293396096 conn(0x7f4b980d0040 msgr2=0x7f4b98071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 shutdown_connections 2024-04-20T10:28:01.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:01.910+0000 7f4b97e00640 1 -- 172.21.15.47:0/1293396096 wait complete. 2024-04-20T10:28:01.928 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2024-04-20T10:28:01.074553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":227932,"kb_used_data":3432,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754746788,"statfs":{"total":773094113280,"available":772860710912,"internally_reserved":0,"allocated":3514368,"data_stored":2072296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":229756464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1781,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005662"},"pg_stats":[{"pgid":"2.1f","version":"22'4","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392064+0000","last_change":"2024-04-20T10:27:32.090082+0000","last_active":"2024-04-20T10:27:38.392064+0000","last_peered":"2024-04-20T10:27:38.392064+0000","last_clean":"2024-04-20T10:27:38.392064+0000","last_became_active":"2024-04-20T10:27:32.089819+0000","last_became_peered":"2024-04-20T10:27:32.089819+0000","last_unstale":"2024-04-20T10:27:38.392064+0000","last_undegraded":"2024-04-20T10:27:38.392064+0000","last_fullsized":"2024-04-20T10:27:38.392064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:50:16.568087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112567+0000","last_change":"2024-04-20T10:27:33.015363+0000","last_active":"2024-04-20T10:27:38.112567+0000","last_peered":"2024-04-20T10:27:38.112567+0000","last_clean":"2024-04-20T10:27:38.112567+0000","last_became_active":"2024-04-20T10:27:33.013421+0000","last_became_peered":"2024-04-20T10:27:33.013421+0000","last_unstale":"2024-04-20T10:27:38.112567+0000","last_undegraded":"2024-04-20T10:27:38.112567+0000","last_fullsized":"2024-04-20T10:27:38.112567+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391215+0000","last_change":"2024-04-20T10:27:37.132637+0000","last_active":"2024-04-20T10:27:38.391215+0000","last_peered":"2024-04-20T10:27:38.391215+0000","last_clean":"2024-04-20T10:27:38.391215+0000","last_became_active":"2024-04-20T10:27:37.132147+0000","last_became_peered":"2024-04-20T10:27:37.132147+0000","last_unstale":"2024-04-20T10:27:38.391215+0000","last_undegraded":"2024-04-20T10:27:38.391215+0000","last_fullsized":"2024-04-20T10:27:38.391215+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:13.081027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4,1],"acting":[7,6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112843+0000","last_change":"2024-04-20T10:27:32.093624+0000","last_active":"2024-04-20T10:27:38.112843+0000","last_peered":"2024-04-20T10:27:38.112843+0000","last_clean":"2024-04-20T10:27:38.112843+0000","last_became_active":"2024-04-20T10:27:32.093349+0000","last_became_peered":"2024-04-20T10:27:32.093349+0000","last_unstale":"2024-04-20T10:27:38.112843+0000","last_undegraded":"2024-04-20T10:27:38.112843+0000","last_fullsized":"2024-04-20T10:27:38.112843+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113178+0000","last_change":"2024-04-20T10:27:33.019078+0000","last_active":"2024-04-20T10:27:38.113178+0000","last_peered":"2024-04-20T10:27:38.113178+0000","last_clean":"2024-04-20T10:27:38.113178+0000","last_became_active":"2024-04-20T10:27:33.018362+0000","last_became_peered":"2024-04-20T10:27:33.018362+0000","last_unstale":"2024-04-20T10:27:38.113178+0000","last_undegraded":"2024-04-20T10:27:38.113178+0000","last_fullsized":"2024-04-20T10:27:38.113178+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390250+0000","last_change":"2024-04-20T10:27:37.118407+0000","last_active":"2024-04-20T10:27:38.390250+0000","last_peered":"2024-04-20T10:27:38.390250+0000","last_clean":"2024-04-20T10:27:38.390250+0000","last_became_active":"2024-04-20T10:27:37.117850+0000","last_became_peered":"2024-04-20T10:27:37.117850+0000","last_unstale":"2024-04-20T10:27:38.390250+0000","last_undegraded":"2024-04-20T10:27:38.390250+0000","last_fullsized":"2024-04-20T10:27:38.390250+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:36.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4,6],"acting":[2,7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388629+0000","last_change":"2024-04-20T10:27:32.080920+0000","last_active":"2024-04-20T10:27:38.388629+0000","last_peered":"2024-04-20T10:27:38.388629+0000","last_clean":"2024-04-20T10:27:38.388629+0000","last_became_active":"2024-04-20T10:27:32.080614+0000","last_became_peered":"2024-04-20T10:27:32.080614+0000","last_unstale":"2024-04-20T10:27:38.388629+0000","last_undegraded":"2024-04-20T10:27:38.388629+0000","last_fullsized":"2024-04-20T10:27:38.388629+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:51.883497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392440+0000","last_change":"2024-04-20T10:27:33.006318+0000","last_active":"2024-04-20T10:27:38.392440+0000","last_peered":"2024-04-20T10:27:38.392440+0000","last_clean":"2024-04-20T10:27:38.392440+0000","last_became_active":"2024-04-20T10:27:33.006040+0000","last_became_peered":"2024-04-20T10:27:33.006040+0000","last_unstale":"2024-04-20T10:27:38.392440+0000","last_undegraded":"2024-04-20T10:27:38.392440+0000","last_fullsized":"2024-04-20T10:27:38.392440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391743+0000","last_change":"2024-04-20T10:27:37.123688+0000","last_active":"2024-04-20T10:27:38.391743+0000","last_peered":"2024-04-20T10:27:38.391743+0000","last_clean":"2024-04-20T10:27:38.391743+0000","last_became_active":"2024-04-20T10:27:37.123316+0000","last_became_peered":"2024-04-20T10:27:37.123316+0000","last_unstale":"2024-04-20T10:27:38.391743+0000","last_undegraded":"2024-04-20T10:27:38.391743+0000","last_fullsized":"2024-04-20T10:27:38.391743+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3,6],"acting":[2,5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388939+0000","last_change":"2024-04-20T10:27:32.082282+0000","last_active":"2024-04-20T10:27:38.388939+0000","last_peered":"2024-04-20T10:27:38.388939+0000","last_clean":"2024-04-20T10:27:38.388939+0000","last_became_active":"2024-04-20T10:27:32.081943+0000","last_became_peered":"2024-04-20T10:27:32.081943+0000","last_unstale":"2024-04-20T10:27:38.388939+0000","last_undegraded":"2024-04-20T10:27:38.388939+0000","last_fullsized":"2024-04-20T10:27:38.388939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:12.394383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108312+0000","last_change":"2024-04-20T10:27:33.003700+0000","last_active":"2024-04-20T10:27:38.108312+0000","last_peered":"2024-04-20T10:27:38.108312+0000","last_clean":"2024-04-20T10:27:38.108312+0000","last_became_active":"2024-04-20T10:27:33.003443+0000","last_became_peered":"2024-04-20T10:27:33.003443+0000","last_unstale":"2024-04-20T10:27:38.108312+0000","last_undegraded":"2024-04-20T10:27:38.108312+0000","last_fullsized":"2024-04-20T10:27:38.108312+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:33.760650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390259+0000","last_change":"2024-04-20T10:27:37.127775+0000","last_active":"2024-04-20T10:27:38.390259+0000","last_peered":"2024-04-20T10:27:38.390259+0000","last_clean":"2024-04-20T10:27:38.390259+0000","last_became_active":"2024-04-20T10:27:37.126837+0000","last_became_peered":"2024-04-20T10:27:37.126837+0000","last_unstale":"2024-04-20T10:27:38.390259+0000","last_undegraded":"2024-04-20T10:27:38.390259+0000","last_fullsized":"2024-04-20T10:27:38.390259+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:31.369897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5,7],"acting":[2,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106458+0000","last_change":"2024-04-20T10:27:33.004497+0000","last_active":"2024-04-20T10:27:38.106458+0000","last_peered":"2024-04-20T10:27:38.106458+0000","last_clean":"2024-04-20T10:27:38.106458+0000","last_became_active":"2024-04-20T10:27:33.004228+0000","last_became_peered":"2024-04-20T10:27:33.004228+0000","last_unstale":"2024-04-20T10:27:38.106458+0000","last_undegraded":"2024-04-20T10:27:38.106458+0000","last_fullsized":"2024-04-20T10:27:38.106458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:01:04.936311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106698+0000","last_change":"2024-04-20T10:27:32.069803+0000","last_active":"2024-04-20T10:27:38.106698+0000","last_peered":"2024-04-20T10:27:38.106698+0000","last_clean":"2024-04-20T10:27:38.106698+0000","last_became_active":"2024-04-20T10:27:32.069542+0000","last_became_peered":"2024-04-20T10:27:32.069542+0000","last_unstale":"2024-04-20T10:27:38.106698+0000","last_undegraded":"2024-04-20T10:27:38.106698+0000","last_fullsized":"2024-04-20T10:27:38.106698+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107502+0000","last_change":"2024-04-20T10:27:37.122386+0000","last_active":"2024-04-20T10:27:38.107502+0000","last_peered":"2024-04-20T10:27:38.107502+0000","last_clean":"2024-04-20T10:27:38.107502+0000","last_became_active":"2024-04-20T10:27:37.121943+0000","last_became_peered":"2024-04-20T10:27:37.121943+0000","last_unstale":"2024-04-20T10:27:38.107502+0000","last_undegraded":"2024-04-20T10:27:38.107502+0000","last_fullsized":"2024-04-20T10:27:38.107502+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:53:31.071316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,5],"acting":[3,1,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391786+0000","last_change":"2024-04-20T10:27:33.015242+0000","last_active":"2024-04-20T10:27:38.391786+0000","last_peered":"2024-04-20T10:27:38.391786+0000","last_clean":"2024-04-20T10:27:38.391786+0000","last_became_active":"2024-04-20T10:27:33.014003+0000","last_became_peered":"2024-04-20T10:27:33.014003+0000","last_unstale":"2024-04-20T10:27:38.391786+0000","last_undegraded":"2024-04-20T10:27:38.391786+0000","last_fullsized":"2024-04-20T10:27:38.391786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110379+0000","last_change":"2024-04-20T10:27:32.087085+0000","last_active":"2024-04-20T10:27:38.110379+0000","last_peered":"2024-04-20T10:27:38.110379+0000","last_clean":"2024-04-20T10:27:38.110379+0000","last_became_active":"2024-04-20T10:27:32.086777+0000","last_became_peered":"2024-04-20T10:27:32.086777+0000","last_unstale":"2024-04-20T10:27:38.110379+0000","last_undegraded":"2024-04-20T10:27:38.110379+0000","last_fullsized":"2024-04-20T10:27:38.110379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388751+0000","last_change":"2024-04-20T10:27:37.123374+0000","last_active":"2024-04-20T10:27:38.388751+0000","last_peered":"2024-04-20T10:27:38.388751+0000","last_clean":"2024-04-20T10:27:38.388751+0000","last_became_active":"2024-04-20T10:27:37.118855+0000","last_became_peered":"2024-04-20T10:27:37.118855+0000","last_unstale":"2024-04-20T10:27:38.388751+0000","last_undegraded":"2024-04-20T10:27:38.388751+0000","last_fullsized":"2024-04-20T10:27:38.388751+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,6,1],"acting":[0,4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389358+0000","last_change":"2024-04-20T10:27:33.013100+0000","last_active":"2024-04-20T10:27:38.389358+0000","last_peered":"2024-04-20T10:27:38.389358+0000","last_clean":"2024-04-20T10:27:38.389358+0000","last_became_active":"2024-04-20T10:27:33.012720+0000","last_became_peered":"2024-04-20T10:27:33.012720+0000","last_unstale":"2024-04-20T10:27:38.389358+0000","last_undegraded":"2024-04-20T10:27:38.389358+0000","last_fullsized":"2024-04-20T10:27:38.389358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:40:52.950066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390484+0000","last_change":"2024-04-20T10:27:32.082330+0000","last_active":"2024-04-20T10:27:38.390484+0000","last_peered":"2024-04-20T10:27:38.390484+0000","last_clean":"2024-04-20T10:27:38.390484+0000","last_became_active":"2024-04-20T10:27:32.080602+0000","last_became_peered":"2024-04-20T10:27:32.080602+0000","last_unstale":"2024-04-20T10:27:38.390484+0000","last_undegraded":"2024-04-20T10:27:38.390484+0000","last_fullsized":"2024-04-20T10:27:38.390484+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:16:59.067061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392568+0000","last_change":"2024-04-20T10:27:37.125949+0000","last_active":"2024-04-20T10:27:38.392568+0000","last_peered":"2024-04-20T10:27:38.392568+0000","last_clean":"2024-04-20T10:27:38.392568+0000","last_became_active":"2024-04-20T10:27:37.125481+0000","last_became_peered":"2024-04-20T10:27:37.125481+0000","last_unstale":"2024-04-20T10:27:38.392568+0000","last_undegraded":"2024-04-20T10:27:38.392568+0000","last_fullsized":"2024-04-20T10:27:38.392568+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:15.992852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2,3],"acting":[7,4,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105786+0000","last_change":"2024-04-20T10:27:33.015612+0000","last_active":"2024-04-20T10:27:38.105786+0000","last_peered":"2024-04-20T10:27:38.105786+0000","last_clean":"2024-04-20T10:27:38.105786+0000","last_became_active":"2024-04-20T10:27:33.014370+0000","last_became_peered":"2024-04-20T10:27:33.014370+0000","last_unstale":"2024-04-20T10:27:38.105786+0000","last_undegraded":"2024-04-20T10:27:38.105786+0000","last_fullsized":"2024-04-20T10:27:38.105786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387559+0000","last_change":"2024-04-20T10:27:32.068081+0000","last_active":"2024-04-20T10:27:38.387559+0000","last_peered":"2024-04-20T10:27:38.387559+0000","last_clean":"2024-04-20T10:27:38.387559+0000","last_became_active":"2024-04-20T10:27:32.067727+0000","last_became_peered":"2024-04-20T10:27:32.067727+0000","last_unstale":"2024-04-20T10:27:38.387559+0000","last_undegraded":"2024-04-20T10:27:38.387559+0000","last_fullsized":"2024-04-20T10:27:38.387559+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:10:59.773254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392244+0000","last_change":"2024-04-20T10:27:37.119884+0000","last_active":"2024-04-20T10:27:38.392244+0000","last_peered":"2024-04-20T10:27:38.392244+0000","last_clean":"2024-04-20T10:27:38.392244+0000","last_became_active":"2024-04-20T10:27:37.119461+0000","last_became_peered":"2024-04-20T10:27:37.119461+0000","last_unstale":"2024-04-20T10:27:38.392244+0000","last_undegraded":"2024-04-20T10:27:38.392244+0000","last_fullsized":"2024-04-20T10:27:38.392244+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,7],"acting":[2,1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391977+0000","last_change":"2024-04-20T10:27:33.009139+0000","last_active":"2024-04-20T10:27:38.391977+0000","last_peered":"2024-04-20T10:27:38.391977+0000","last_clean":"2024-04-20T10:27:38.391977+0000","last_became_active":"2024-04-20T10:27:33.008830+0000","last_became_peered":"2024-04-20T10:27:33.008830+0000","last_unstale":"2024-04-20T10:27:38.391977+0000","last_undegraded":"2024-04-20T10:27:38.391977+0000","last_fullsized":"2024-04-20T10:27:38.391977+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:16:06.157535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108669+0000","last_change":"2024-04-20T10:27:32.086356+0000","last_active":"2024-04-20T10:27:38.108669+0000","last_peered":"2024-04-20T10:27:38.108669+0000","last_clean":"2024-04-20T10:27:38.108669+0000","last_became_active":"2024-04-20T10:27:32.077640+0000","last_became_peered":"2024-04-20T10:27:32.077640+0000","last_unstale":"2024-04-20T10:27:38.108669+0000","last_undegraded":"2024-04-20T10:27:38.108669+0000","last_fullsized":"2024-04-20T10:27:38.108669+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:44.469214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390797+0000","last_change":"2024-04-20T10:27:37.124270+0000","last_active":"2024-04-20T10:27:38.390797+0000","last_peered":"2024-04-20T10:27:38.390797+0000","last_clean":"2024-04-20T10:27:38.390797+0000","last_became_active":"2024-04-20T10:27:37.123953+0000","last_became_peered":"2024-04-20T10:27:37.123953+0000","last_unstale":"2024-04-20T10:27:38.390797+0000","last_undegraded":"2024-04-20T10:27:38.390797+0000","last_fullsized":"2024-04-20T10:27:38.390797+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:27.841628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,3,1],"acting":[2,6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390487+0000","last_change":"2024-04-20T10:27:33.017825+0000","last_active":"2024-04-20T10:27:38.390487+0000","last_peered":"2024-04-20T10:27:38.390487+0000","last_clean":"2024-04-20T10:27:38.390487+0000","last_became_active":"2024-04-20T10:27:33.015635+0000","last_became_peered":"2024-04-20T10:27:33.015635+0000","last_unstale":"2024-04-20T10:27:38.390487+0000","last_undegraded":"2024-04-20T10:27:38.390487+0000","last_fullsized":"2024-04-20T10:27:38.390487+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:06:58.914910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392232+0000","last_change":"2024-04-20T10:27:32.082908+0000","last_active":"2024-04-20T10:27:38.392232+0000","last_peered":"2024-04-20T10:27:38.392232+0000","last_clean":"2024-04-20T10:27:38.392232+0000","last_became_active":"2024-04-20T10:27:32.082507+0000","last_became_peered":"2024-04-20T10:27:32.082507+0000","last_unstale":"2024-04-20T10:27:38.392232+0000","last_undegraded":"2024-04-20T10:27:38.392232+0000","last_fullsized":"2024-04-20T10:27:38.392232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:55:52.804365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390908+0000","last_change":"2024-04-20T10:27:37.126093+0000","last_active":"2024-04-20T10:27:38.390908+0000","last_peered":"2024-04-20T10:27:38.390908+0000","last_clean":"2024-04-20T10:27:38.390908+0000","last_became_active":"2024-04-20T10:27:37.125681+0000","last_became_peered":"2024-04-20T10:27:37.125681+0000","last_unstale":"2024-04-20T10:27:38.390908+0000","last_undegraded":"2024-04-20T10:27:38.390908+0000","last_fullsized":"2024-04-20T10:27:38.390908+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:13:36.622877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,6,3],"acting":[4,1,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109415+0000","last_change":"2024-04-20T10:27:33.011662+0000","last_active":"2024-04-20T10:27:38.109415+0000","last_peered":"2024-04-20T10:27:38.109415+0000","last_clean":"2024-04-20T10:27:38.109415+0000","last_became_active":"2024-04-20T10:27:33.010842+0000","last_became_peered":"2024-04-20T10:27:33.010842+0000","last_unstale":"2024-04-20T10:27:38.109415+0000","last_undegraded":"2024-04-20T10:27:38.109415+0000","last_fullsized":"2024-04-20T10:27:38.109415+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108664+0000","last_change":"2024-04-20T10:27:32.083502+0000","last_active":"2024-04-20T10:27:38.108664+0000","last_peered":"2024-04-20T10:27:38.108664+0000","last_clean":"2024-04-20T10:27:38.108664+0000","last_became_active":"2024-04-20T10:27:32.083217+0000","last_became_peered":"2024-04-20T10:27:32.083217+0000","last_unstale":"2024-04-20T10:27:38.108664+0000","last_undegraded":"2024-04-20T10:27:38.108664+0000","last_fullsized":"2024-04-20T10:27:38.108664+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:41.953724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109193+0000","last_change":"2024-04-20T10:27:37.110995+0000","last_active":"2024-04-20T10:27:38.109193+0000","last_peered":"2024-04-20T10:27:38.109193+0000","last_clean":"2024-04-20T10:27:38.109193+0000","last_became_active":"2024-04-20T10:27:37.110619+0000","last_became_peered":"2024-04-20T10:27:37.110619+0000","last_unstale":"2024-04-20T10:27:38.109193+0000","last_undegraded":"2024-04-20T10:27:38.109193+0000","last_fullsized":"2024-04-20T10:27:38.109193+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:33.173033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2,7],"acting":[6,0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391490+0000","last_change":"2024-04-20T10:27:32.997361+0000","last_active":"2024-04-20T10:27:38.391490+0000","last_peered":"2024-04-20T10:27:38.391490+0000","last_clean":"2024-04-20T10:27:38.391490+0000","last_became_active":"2024-04-20T10:27:32.996366+0000","last_became_peered":"2024-04-20T10:27:32.996366+0000","last_unstale":"2024-04-20T10:27:38.391490+0000","last_undegraded":"2024-04-20T10:27:38.391490+0000","last_fullsized":"2024-04-20T10:27:38.391490+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110460+0000","last_change":"2024-04-20T10:27:32.087311+0000","last_active":"2024-04-20T10:27:38.110460+0000","last_peered":"2024-04-20T10:27:38.110460+0000","last_clean":"2024-04-20T10:27:38.110460+0000","last_became_active":"2024-04-20T10:27:32.086959+0000","last_became_peered":"2024-04-20T10:27:32.086959+0000","last_unstale":"2024-04-20T10:27:38.110460+0000","last_undegraded":"2024-04-20T10:27:38.110460+0000","last_fullsized":"2024-04-20T10:27:38.110460+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110080+0000","last_change":"2024-04-20T10:27:37.121913+0000","last_active":"2024-04-20T10:27:38.110080+0000","last_peered":"2024-04-20T10:27:38.110080+0000","last_clean":"2024-04-20T10:27:38.110080+0000","last_became_active":"2024-04-20T10:27:37.121491+0000","last_became_peered":"2024-04-20T10:27:37.121491+0000","last_unstale":"2024-04-20T10:27:38.110080+0000","last_undegraded":"2024-04-20T10:27:38.110080+0000","last_fullsized":"2024-04-20T10:27:38.110080+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:28.925537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5,7],"acting":[6,3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388610+0000","last_change":"2024-04-20T10:27:33.004607+0000","last_active":"2024-04-20T10:27:38.388610+0000","last_peered":"2024-04-20T10:27:38.388610+0000","last_clean":"2024-04-20T10:27:38.388610+0000","last_became_active":"2024-04-20T10:27:33.004299+0000","last_became_peered":"2024-04-20T10:27:33.004299+0000","last_unstale":"2024-04-20T10:27:38.388610+0000","last_undegraded":"2024-04-20T10:27:38.388610+0000","last_fullsized":"2024-04-20T10:27:38.388610+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107052+0000","last_change":"2024-04-20T10:27:32.082735+0000","last_active":"2024-04-20T10:27:38.107052+0000","last_peered":"2024-04-20T10:27:38.107052+0000","last_clean":"2024-04-20T10:27:38.107052+0000","last_became_active":"2024-04-20T10:27:32.079695+0000","last_became_peered":"2024-04-20T10:27:32.079695+0000","last_unstale":"2024-04-20T10:27:38.107052+0000","last_undegraded":"2024-04-20T10:27:38.107052+0000","last_fullsized":"2024-04-20T10:27:38.107052+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:07.962987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109079+0000","last_change":"2024-04-20T10:27:37.131952+0000","last_active":"2024-04-20T10:27:38.109079+0000","last_peered":"2024-04-20T10:27:38.109079+0000","last_clean":"2024-04-20T10:27:38.109079+0000","last_became_active":"2024-04-20T10:27:37.130721+0000","last_became_peered":"2024-04-20T10:27:37.130721+0000","last_unstale":"2024-04-20T10:27:38.109079+0000","last_undegraded":"2024-04-20T10:27:38.109079+0000","last_fullsized":"2024-04-20T10:27:38.109079+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:45.776795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0,3],"acting":[6,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107777+0000","last_change":"2024-04-20T10:27:33.013537+0000","last_active":"2024-04-20T10:27:38.107777+0000","last_peered":"2024-04-20T10:27:38.107777+0000","last_clean":"2024-04-20T10:27:38.107777+0000","last_became_active":"2024-04-20T10:27:33.013223+0000","last_became_peered":"2024-04-20T10:27:33.013223+0000","last_unstale":"2024-04-20T10:27:38.107777+0000","last_undegraded":"2024-04-20T10:27:38.107777+0000","last_fullsized":"2024-04-20T10:27:38.107777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391320+0000","last_change":"2024-04-20T10:27:32.086422+0000","last_active":"2024-04-20T10:27:38.391320+0000","last_peered":"2024-04-20T10:27:38.391320+0000","last_clean":"2024-04-20T10:27:38.391320+0000","last_became_active":"2024-04-20T10:27:32.085353+0000","last_became_peered":"2024-04-20T10:27:32.085353+0000","last_unstale":"2024-04-20T10:27:38.391320+0000","last_undegraded":"2024-04-20T10:27:38.391320+0000","last_fullsized":"2024-04-20T10:27:38.391320+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112264+0000","last_change":"2024-04-20T10:27:37.113304+0000","last_active":"2024-04-20T10:27:38.112264+0000","last_peered":"2024-04-20T10:27:38.112264+0000","last_clean":"2024-04-20T10:27:38.112264+0000","last_became_active":"2024-04-20T10:27:37.112828+0000","last_became_peered":"2024-04-20T10:27:37.112828+0000","last_unstale":"2024-04-20T10:27:38.112264+0000","last_undegraded":"2024-04-20T10:27:38.112264+0000","last_fullsized":"2024-04-20T10:27:38.112264+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:19.178189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,7],"acting":[6,0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108899+0000","last_change":"2024-04-20T10:27:32.989233+0000","last_active":"2024-04-20T10:27:38.108899+0000","last_peered":"2024-04-20T10:27:38.108899+0000","last_clean":"2024-04-20T10:27:38.108899+0000","last_became_active":"2024-04-20T10:27:32.988092+0000","last_became_peered":"2024-04-20T10:27:32.988092+0000","last_unstale":"2024-04-20T10:27:38.108899+0000","last_undegraded":"2024-04-20T10:27:38.108899+0000","last_fullsized":"2024-04-20T10:27:38.108899+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:11.889218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106243+0000","last_change":"2024-04-20T10:27:32.092257+0000","last_active":"2024-04-20T10:27:38.106243+0000","last_peered":"2024-04-20T10:27:38.106243+0000","last_clean":"2024-04-20T10:27:38.106243+0000","last_became_active":"2024-04-20T10:27:32.091944+0000","last_became_peered":"2024-04-20T10:27:32.091944+0000","last_unstale":"2024-04-20T10:27:38.106243+0000","last_undegraded":"2024-04-20T10:27:38.106243+0000","last_fullsized":"2024-04-20T10:27:38.106243+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:56.139394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112022+0000","last_change":"2024-04-20T10:27:37.124420+0000","last_active":"2024-04-20T10:27:38.112022+0000","last_peered":"2024-04-20T10:27:38.112022+0000","last_clean":"2024-04-20T10:27:38.112022+0000","last_became_active":"2024-04-20T10:27:37.124048+0000","last_became_peered":"2024-04-20T10:27:37.124048+0000","last_unstale":"2024-04-20T10:27:38.112022+0000","last_undegraded":"2024-04-20T10:27:38.112022+0000","last_fullsized":"2024-04-20T10:27:38.112022+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:37.864284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6,1],"acting":[5,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390325+0000","last_change":"2024-04-20T10:27:33.016860+0000","last_active":"2024-04-20T10:27:38.390325+0000","last_peered":"2024-04-20T10:27:38.390325+0000","last_clean":"2024-04-20T10:27:38.390325+0000","last_became_active":"2024-04-20T10:27:33.016323+0000","last_became_peered":"2024-04-20T10:27:33.016323+0000","last_unstale":"2024-04-20T10:27:38.390325+0000","last_undegraded":"2024-04-20T10:27:38.390325+0000","last_fullsized":"2024-04-20T10:27:38.390325+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:11:39.053198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107037+0000","last_change":"2024-04-20T10:27:32.069209+0000","last_active":"2024-04-20T10:27:38.107037+0000","last_peered":"2024-04-20T10:27:38.107037+0000","last_clean":"2024-04-20T10:27:38.107037+0000","last_became_active":"2024-04-20T10:27:32.068711+0000","last_became_peered":"2024-04-20T10:27:32.068711+0000","last_unstale":"2024-04-20T10:27:38.107037+0000","last_undegraded":"2024-04-20T10:27:38.107037+0000","last_fullsized":"2024-04-20T10:27:38.107037+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:55.256665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":178,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392446+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:27:38.392446+0000","last_peered":"2024-04-20T10:27:38.392446+0000","last_clean":"2024-04-20T10:27:38.392446+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:27:38.392446+0000","last_undegraded":"2024-04-20T10:27:38.392446+0000","last_fullsized":"2024-04-20T10:27:38.392446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108582+0000","last_change":"2024-04-20T10:27:37.103228+0000","last_active":"2024-04-20T10:27:38.108582+0000","last_peered":"2024-04-20T10:27:38.108582+0000","last_clean":"2024-04-20T10:27:38.108582+0000","last_became_active":"2024-04-20T10:27:37.102768+0000","last_became_peered":"2024-04-20T10:27:37.102768+0000","last_unstale":"2024-04-20T10:27:38.108582+0000","last_undegraded":"2024-04-20T10:27:38.108582+0000","last_fullsized":"2024-04-20T10:27:38.108582+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:09:13.639017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2,0],"acting":[3,1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108024+0000","last_change":"2024-04-20T10:27:33.014266+0000","last_active":"2024-04-20T10:27:38.108024+0000","last_peered":"2024-04-20T10:27:38.108024+0000","last_clean":"2024-04-20T10:27:38.108024+0000","last_became_active":"2024-04-20T10:27:33.013935+0000","last_became_peered":"2024-04-20T10:27:33.013935+0000","last_unstale":"2024-04-20T10:27:38.108024+0000","last_undegraded":"2024-04-20T10:27:38.108024+0000","last_fullsized":"2024-04-20T10:27:38.108024+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:03.292296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109594+0000","last_change":"2024-04-20T10:27:32.078144+0000","last_active":"2024-04-20T10:27:38.109594+0000","last_peered":"2024-04-20T10:27:38.109594+0000","last_clean":"2024-04-20T10:27:38.109594+0000","last_became_active":"2024-04-20T10:27:32.075588+0000","last_became_peered":"2024-04-20T10:27:32.075588+0000","last_unstale":"2024-04-20T10:27:38.109594+0000","last_undegraded":"2024-04-20T10:27:38.109594+0000","last_fullsized":"2024-04-20T10:27:38.109594+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390435+0000","last_change":"2024-04-20T10:27:37.128340+0000","last_active":"2024-04-20T10:27:38.390435+0000","last_peered":"2024-04-20T10:27:38.390435+0000","last_clean":"2024-04-20T10:27:38.390435+0000","last_became_active":"2024-04-20T10:27:37.127609+0000","last_became_peered":"2024-04-20T10:27:37.127609+0000","last_unstale":"2024-04-20T10:27:38.390435+0000","last_undegraded":"2024-04-20T10:27:38.390435+0000","last_fullsized":"2024-04-20T10:27:38.390435+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:41.767940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7,4],"acting":[2,3,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388918+0000","last_change":"2024-04-20T10:27:33.007624+0000","last_active":"2024-04-20T10:27:38.388918+0000","last_peered":"2024-04-20T10:27:38.388918+0000","last_clean":"2024-04-20T10:27:38.388918+0000","last_became_active":"2024-04-20T10:27:33.007319+0000","last_became_peered":"2024-04-20T10:27:33.007319+0000","last_unstale":"2024-04-20T10:27:38.388918+0000","last_undegraded":"2024-04-20T10:27:38.388918+0000","last_fullsized":"2024-04-20T10:27:38.388918+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112959+0000","last_change":"2024-04-20T10:27:32.087470+0000","last_active":"2024-04-20T10:27:38.112959+0000","last_peered":"2024-04-20T10:27:38.112959+0000","last_clean":"2024-04-20T10:27:38.112959+0000","last_became_active":"2024-04-20T10:27:32.086045+0000","last_became_peered":"2024-04-20T10:27:32.086045+0000","last_unstale":"2024-04-20T10:27:38.112959+0000","last_undegraded":"2024-04-20T10:27:38.112959+0000","last_fullsized":"2024-04-20T10:27:38.112959+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111234+0000","last_change":"2024-04-20T10:27:37.118303+0000","last_active":"2024-04-20T10:27:38.111234+0000","last_peered":"2024-04-20T10:27:38.111234+0000","last_clean":"2024-04-20T10:27:38.111234+0000","last_became_active":"2024-04-20T10:27:37.117875+0000","last_became_peered":"2024-04-20T10:27:37.117875+0000","last_unstale":"2024-04-20T10:27:38.111234+0000","last_undegraded":"2024-04-20T10:27:38.111234+0000","last_fullsized":"2024-04-20T10:27:38.111234+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:59.680449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1,3],"acting":[5,4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389469+0000","last_change":"2024-04-20T10:27:33.021039+0000","last_active":"2024-04-20T10:27:38.389469+0000","last_peered":"2024-04-20T10:27:38.389469+0000","last_clean":"2024-04-20T10:27:38.389469+0000","last_became_active":"2024-04-20T10:27:33.020767+0000","last_became_peered":"2024-04-20T10:27:33.020767+0000","last_unstale":"2024-04-20T10:27:38.389469+0000","last_undegraded":"2024-04-20T10:27:38.389469+0000","last_fullsized":"2024-04-20T10:27:38.389469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392137+0000","last_change":"2024-04-20T10:27:32.091910+0000","last_active":"2024-04-20T10:27:38.392137+0000","last_peered":"2024-04-20T10:27:38.392137+0000","last_clean":"2024-04-20T10:27:38.392137+0000","last_became_active":"2024-04-20T10:27:32.091500+0000","last_became_peered":"2024-04-20T10:27:32.091500+0000","last_unstale":"2024-04-20T10:27:38.392137+0000","last_undegraded":"2024-04-20T10:27:38.392137+0000","last_fullsized":"2024-04-20T10:27:38.392137+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:53:02.468594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105575+0000","last_change":"2024-04-20T10:27:37.119655+0000","last_active":"2024-04-20T10:27:38.105575+0000","last_peered":"2024-04-20T10:27:38.105575+0000","last_clean":"2024-04-20T10:27:38.105575+0000","last_became_active":"2024-04-20T10:27:37.119274+0000","last_became_peered":"2024-04-20T10:27:37.119274+0000","last_unstale":"2024-04-20T10:27:38.105575+0000","last_undegraded":"2024-04-20T10:27:38.105575+0000","last_fullsized":"2024-04-20T10:27:38.105575+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:56:33.147822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392306+0000","last_change":"2024-04-20T10:27:33.003557+0000","last_active":"2024-04-20T10:27:38.392306+0000","last_peered":"2024-04-20T10:27:38.392306+0000","last_clean":"2024-04-20T10:27:38.392306+0000","last_became_active":"2024-04-20T10:27:33.001763+0000","last_became_peered":"2024-04-20T10:27:33.001763+0000","last_unstale":"2024-04-20T10:27:38.392306+0000","last_undegraded":"2024-04-20T10:27:38.392306+0000","last_fullsized":"2024-04-20T10:27:38.392306+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:41:45.483619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389540+0000","last_change":"2024-04-20T10:27:32.090042+0000","last_active":"2024-04-20T10:27:38.389540+0000","last_peered":"2024-04-20T10:27:38.389540+0000","last_clean":"2024-04-20T10:27:38.389540+0000","last_became_active":"2024-04-20T10:27:32.089694+0000","last_became_peered":"2024-04-20T10:27:32.089694+0000","last_unstale":"2024-04-20T10:27:38.389540+0000","last_undegraded":"2024-04-20T10:27:38.389540+0000","last_fullsized":"2024-04-20T10:27:38.389540+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393350+0000","last_change":"2024-04-20T10:27:37.125234+0000","last_active":"2024-04-20T10:27:38.393350+0000","last_peered":"2024-04-20T10:27:38.393350+0000","last_clean":"2024-04-20T10:27:38.393350+0000","last_became_active":"2024-04-20T10:27:37.124838+0000","last_became_peered":"2024-04-20T10:27:37.124838+0000","last_unstale":"2024-04-20T10:27:38.393350+0000","last_undegraded":"2024-04-20T10:27:38.393350+0000","last_fullsized":"2024-04-20T10:27:38.393350+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:47.071764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,2],"acting":[1,5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105563+0000","last_change":"2024-04-20T10:27:33.013769+0000","last_active":"2024-04-20T10:27:38.105563+0000","last_peered":"2024-04-20T10:27:38.105563+0000","last_clean":"2024-04-20T10:27:38.105563+0000","last_became_active":"2024-04-20T10:27:33.005662+0000","last_became_peered":"2024-04-20T10:27:33.005662+0000","last_unstale":"2024-04-20T10:27:38.105563+0000","last_undegraded":"2024-04-20T10:27:38.105563+0000","last_fullsized":"2024-04-20T10:27:38.105563+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:09.144127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389725+0000","last_change":"2024-04-20T10:27:32.080478+0000","last_active":"2024-04-20T10:27:38.389725+0000","last_peered":"2024-04-20T10:27:38.389725+0000","last_clean":"2024-04-20T10:27:38.389725+0000","last_became_active":"2024-04-20T10:27:32.080148+0000","last_became_peered":"2024-04-20T10:27:32.080148+0000","last_unstale":"2024-04-20T10:27:38.389725+0000","last_undegraded":"2024-04-20T10:27:38.389725+0000","last_fullsized":"2024-04-20T10:27:38.389725+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:10:17.004529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112793+0000","last_change":"2024-04-20T10:27:37.126177+0000","last_active":"2024-04-20T10:27:38.112793+0000","last_peered":"2024-04-20T10:27:38.112793+0000","last_clean":"2024-04-20T10:27:38.112793+0000","last_became_active":"2024-04-20T10:27:37.125862+0000","last_became_peered":"2024-04-20T10:27:37.125862+0000","last_unstale":"2024-04-20T10:27:38.112793+0000","last_undegraded":"2024-04-20T10:27:38.112793+0000","last_fullsized":"2024-04-20T10:27:38.112793+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:05:07.395930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1,6],"acting":[5,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391792+0000","last_change":"2024-04-20T10:27:33.016106+0000","last_active":"2024-04-20T10:27:38.391792+0000","last_peered":"2024-04-20T10:27:38.391792+0000","last_clean":"2024-04-20T10:27:38.391792+0000","last_became_active":"2024-04-20T10:27:33.015787+0000","last_became_peered":"2024-04-20T10:27:33.015787+0000","last_unstale":"2024-04-20T10:27:38.391792+0000","last_undegraded":"2024-04-20T10:27:38.391792+0000","last_fullsized":"2024-04-20T10:27:38.391792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388035+0000","last_change":"2024-04-20T10:27:32.075511+0000","last_active":"2024-04-20T10:27:38.388035+0000","last_peered":"2024-04-20T10:27:38.388035+0000","last_clean":"2024-04-20T10:27:38.388035+0000","last_became_active":"2024-04-20T10:27:32.072523+0000","last_became_peered":"2024-04-20T10:27:32.072523+0000","last_unstale":"2024-04-20T10:27:38.388035+0000","last_undegraded":"2024-04-20T10:27:38.388035+0000","last_fullsized":"2024-04-20T10:27:38.388035+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:04:38.961231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113117+0000","last_change":"2024-04-20T10:27:37.121568+0000","last_active":"2024-04-20T10:27:38.113117+0000","last_peered":"2024-04-20T10:27:38.113117+0000","last_clean":"2024-04-20T10:27:38.113117+0000","last_became_active":"2024-04-20T10:27:37.121105+0000","last_became_peered":"2024-04-20T10:27:37.121105+0000","last_unstale":"2024-04-20T10:27:38.113117+0000","last_undegraded":"2024-04-20T10:27:38.113117+0000","last_fullsized":"2024-04-20T10:27:38.113117+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:20.615032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6,4],"acting":[5,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107307+0000","last_change":"2024-04-20T10:27:33.008601+0000","last_active":"2024-04-20T10:27:38.107307+0000","last_peered":"2024-04-20T10:27:38.107307+0000","last_clean":"2024-04-20T10:27:38.107307+0000","last_became_active":"2024-04-20T10:27:33.008317+0000","last_became_peered":"2024-04-20T10:27:33.008317+0000","last_unstale":"2024-04-20T10:27:38.107307+0000","last_undegraded":"2024-04-20T10:27:38.107307+0000","last_fullsized":"2024-04-20T10:27:38.107307+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:42:55.576815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106751+0000","last_change":"2024-04-20T10:27:32.081278+0000","last_active":"2024-04-20T10:27:38.106751+0000","last_peered":"2024-04-20T10:27:38.106751+0000","last_clean":"2024-04-20T10:27:38.106751+0000","last_became_active":"2024-04-20T10:27:32.080466+0000","last_became_peered":"2024-04-20T10:27:32.080466+0000","last_unstale":"2024-04-20T10:27:38.106751+0000","last_undegraded":"2024-04-20T10:27:38.106751+0000","last_fullsized":"2024-04-20T10:27:38.106751+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:51.199449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109133+0000","last_change":"2024-04-20T10:27:37.132802+0000","last_active":"2024-04-20T10:27:38.109133+0000","last_peered":"2024-04-20T10:27:38.109133+0000","last_clean":"2024-04-20T10:27:38.109133+0000","last_became_active":"2024-04-20T10:27:37.129193+0000","last_became_peered":"2024-04-20T10:27:37.129193+0000","last_unstale":"2024-04-20T10:27:38.109133+0000","last_undegraded":"2024-04-20T10:27:38.109133+0000","last_fullsized":"2024-04-20T10:27:38.109133+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:42.200300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0,7],"acting":[6,4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110211+0000","last_change":"2024-04-20T10:27:33.010100+0000","last_active":"2024-04-20T10:27:38.110211+0000","last_peered":"2024-04-20T10:27:38.110211+0000","last_clean":"2024-04-20T10:27:38.110211+0000","last_became_active":"2024-04-20T10:27:33.009703+0000","last_became_peered":"2024-04-20T10:27:33.009703+0000","last_unstale":"2024-04-20T10:27:38.110211+0000","last_undegraded":"2024-04-20T10:27:38.110211+0000","last_fullsized":"2024-04-20T10:27:38.110211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391899+0000","last_change":"2024-04-20T10:27:32.078724+0000","last_active":"2024-04-20T10:27:38.391899+0000","last_peered":"2024-04-20T10:27:38.391899+0000","last_clean":"2024-04-20T10:27:38.391899+0000","last_became_active":"2024-04-20T10:27:32.078093+0000","last_became_peered":"2024-04-20T10:27:32.078093+0000","last_unstale":"2024-04-20T10:27:38.391899+0000","last_undegraded":"2024-04-20T10:27:38.391899+0000","last_fullsized":"2024-04-20T10:27:38.391899+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107857+0000","last_change":"2024-04-20T10:27:37.121822+0000","last_active":"2024-04-20T10:27:38.107857+0000","last_peered":"2024-04-20T10:27:38.107857+0000","last_clean":"2024-04-20T10:27:38.107857+0000","last_became_active":"2024-04-20T10:27:37.121423+0000","last_became_peered":"2024-04-20T10:27:37.121423+0000","last_unstale":"2024-04-20T10:27:38.107857+0000","last_undegraded":"2024-04-20T10:27:38.107857+0000","last_fullsized":"2024-04-20T10:27:38.107857+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:21:35.974697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5,6],"acting":[3,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388424+0000","last_change":"2024-04-20T10:27:33.001752+0000","last_active":"2024-04-20T10:27:38.388424+0000","last_peered":"2024-04-20T10:27:38.388424+0000","last_clean":"2024-04-20T10:27:38.388424+0000","last_became_active":"2024-04-20T10:27:33.001201+0000","last_became_peered":"2024-04-20T10:27:33.001201+0000","last_unstale":"2024-04-20T10:27:38.388424+0000","last_undegraded":"2024-04-20T10:27:38.388424+0000","last_fullsized":"2024-04-20T10:27:38.388424+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"27'10","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:51.040746+0000","last_change":"2024-04-20T10:27:32.080328+0000","last_active":"2024-04-20T10:27:51.040746+0000","last_peered":"2024-04-20T10:27:51.040746+0000","last_clean":"2024-04-20T10:27:51.040746+0000","last_became_active":"2024-04-20T10:27:32.080011+0000","last_became_peered":"2024-04-20T10:27:32.080011+0000","last_unstale":"2024-04-20T10:27:51.040746+0000","last_undegraded":"2024-04-20T10:27:51.040746+0000","last_fullsized":"2024-04-20T10:27:51.040746+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:22:14.097676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389991+0000","last_change":"2024-04-20T10:27:37.126061+0000","last_active":"2024-04-20T10:27:38.389991+0000","last_peered":"2024-04-20T10:27:38.389991+0000","last_clean":"2024-04-20T10:27:38.389991+0000","last_became_active":"2024-04-20T10:27:37.121461+0000","last_became_peered":"2024-04-20T10:27:37.121461+0000","last_unstale":"2024-04-20T10:27:38.389991+0000","last_undegraded":"2024-04-20T10:27:38.389991+0000","last_fullsized":"2024-04-20T10:27:38.389991+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:10:13.289328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5,0],"acting":[4,3,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107516+0000","last_change":"2024-04-20T10:27:33.007572+0000","last_active":"2024-04-20T10:27:38.107516+0000","last_peered":"2024-04-20T10:27:38.107516+0000","last_clean":"2024-04-20T10:27:38.107516+0000","last_became_active":"2024-04-20T10:27:33.007206+0000","last_became_peered":"2024-04-20T10:27:33.007206+0000","last_unstale":"2024-04-20T10:27:38.107516+0000","last_undegraded":"2024-04-20T10:27:38.107516+0000","last_fullsized":"2024-04-20T10:27:38.107516+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:43:21.846828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393065+0000","last_change":"2024-04-20T10:27:32.080090+0000","last_active":"2024-04-20T10:27:38.393065+0000","last_peered":"2024-04-20T10:27:38.393065+0000","last_clean":"2024-04-20T10:27:38.393065+0000","last_became_active":"2024-04-20T10:27:32.079763+0000","last_became_peered":"2024-04-20T10:27:32.079763+0000","last_unstale":"2024-04-20T10:27:38.393065+0000","last_undegraded":"2024-04-20T10:27:38.393065+0000","last_fullsized":"2024-04-20T10:27:38.393065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:19:43.651615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390646+0000","last_change":"2024-04-20T10:27:37.113593+0000","last_active":"2024-04-20T10:27:38.390646+0000","last_peered":"2024-04-20T10:27:38.390646+0000","last_clean":"2024-04-20T10:27:38.390646+0000","last_became_active":"2024-04-20T10:27:37.113016+0000","last_became_peered":"2024-04-20T10:27:37.113016+0000","last_unstale":"2024-04-20T10:27:38.390646+0000","last_undegraded":"2024-04-20T10:27:38.390646+0000","last_fullsized":"2024-04-20T10:27:38.390646+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:27.394166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,1,7],"acting":[2,3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.113229+0000","last_change":"2024-04-20T10:27:33.020099+0000","last_active":"2024-04-20T10:27:38.113229+0000","last_peered":"2024-04-20T10:27:38.113229+0000","last_clean":"2024-04-20T10:27:38.113229+0000","last_became_active":"2024-04-20T10:27:33.018743+0000","last_became_peered":"2024-04-20T10:27:33.018743+0000","last_unstale":"2024-04-20T10:27:38.113229+0000","last_undegraded":"2024-04-20T10:27:38.113229+0000","last_fullsized":"2024-04-20T10:27:38.113229+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389020+0000","last_change":"2024-04-20T10:27:32.081599+0000","last_active":"2024-04-20T10:27:38.389020+0000","last_peered":"2024-04-20T10:27:38.389020+0000","last_clean":"2024-04-20T10:27:38.389020+0000","last_became_active":"2024-04-20T10:27:32.081288+0000","last_became_peered":"2024-04-20T10:27:32.081288+0000","last_unstale":"2024-04-20T10:27:38.389020+0000","last_undegraded":"2024-04-20T10:27:38.389020+0000","last_fullsized":"2024-04-20T10:27:38.389020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:04.575146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391304+0000","last_change":"2024-04-20T10:27:37.134883+0000","last_active":"2024-04-20T10:27:38.391304+0000","last_peered":"2024-04-20T10:27:38.391304+0000","last_clean":"2024-04-20T10:27:38.391304+0000","last_became_active":"2024-04-20T10:27:37.134221+0000","last_became_peered":"2024-04-20T10:27:37.134221+0000","last_unstale":"2024-04-20T10:27:38.391304+0000","last_undegraded":"2024-04-20T10:27:38.391304+0000","last_fullsized":"2024-04-20T10:27:38.391304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:10.327991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,3,2],"acting":[7,1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392241+0000","last_change":"2024-04-20T10:27:33.006819+0000","last_active":"2024-04-20T10:27:38.392241+0000","last_peered":"2024-04-20T10:27:38.392241+0000","last_clean":"2024-04-20T10:27:38.392241+0000","last_became_active":"2024-04-20T10:27:33.005657+0000","last_became_peered":"2024-04-20T10:27:33.005657+0000","last_unstale":"2024-04-20T10:27:38.392241+0000","last_undegraded":"2024-04-20T10:27:38.392241+0000","last_fullsized":"2024-04-20T10:27:38.392241+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391630+0000","last_change":"2024-04-20T10:27:32.083786+0000","last_active":"2024-04-20T10:27:38.391630+0000","last_peered":"2024-04-20T10:27:38.391630+0000","last_clean":"2024-04-20T10:27:38.391630+0000","last_became_active":"2024-04-20T10:27:32.083256+0000","last_became_peered":"2024-04-20T10:27:32.083256+0000","last_unstale":"2024-04-20T10:27:38.391630+0000","last_undegraded":"2024-04-20T10:27:38.391630+0000","last_fullsized":"2024-04-20T10:27:38.391630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:30:11.023664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390075+0000","last_change":"2024-04-20T10:27:37.131051+0000","last_active":"2024-04-20T10:27:38.390075+0000","last_peered":"2024-04-20T10:27:38.390075+0000","last_clean":"2024-04-20T10:27:38.390075+0000","last_became_active":"2024-04-20T10:27:37.130723+0000","last_became_peered":"2024-04-20T10:27:37.130723+0000","last_unstale":"2024-04-20T10:27:38.390075+0000","last_undegraded":"2024-04-20T10:27:38.390075+0000","last_fullsized":"2024-04-20T10:27:38.390075+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:28.980316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,3,7],"acting":[4,0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391821+0000","last_change":"2024-04-20T10:27:37.124930+0000","last_active":"2024-04-20T10:27:38.391821+0000","last_peered":"2024-04-20T10:27:38.391821+0000","last_clean":"2024-04-20T10:27:38.391821+0000","last_became_active":"2024-04-20T10:27:37.123089+0000","last_became_peered":"2024-04-20T10:27:37.123089+0000","last_unstale":"2024-04-20T10:27:38.391821+0000","last_undegraded":"2024-04-20T10:27:38.391821+0000","last_fullsized":"2024-04-20T10:27:38.391821+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:28:05.113291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,5,3],"acting":[4,1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390231+0000","last_change":"2024-04-20T10:27:33.020003+0000","last_active":"2024-04-20T10:27:38.390231+0000","last_peered":"2024-04-20T10:27:38.390231+0000","last_clean":"2024-04-20T10:27:38.390231+0000","last_became_active":"2024-04-20T10:27:33.019709+0000","last_became_peered":"2024-04-20T10:27:33.019709+0000","last_unstale":"2024-04-20T10:27:38.390231+0000","last_undegraded":"2024-04-20T10:27:38.390231+0000","last_fullsized":"2024-04-20T10:27:38.390231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:06.663893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106161+0000","last_change":"2024-04-20T10:27:32.070067+0000","last_active":"2024-04-20T10:27:38.106161+0000","last_peered":"2024-04-20T10:27:38.106161+0000","last_clean":"2024-04-20T10:27:38.106161+0000","last_became_active":"2024-04-20T10:27:32.069761+0000","last_became_peered":"2024-04-20T10:27:32.069761+0000","last_unstale":"2024-04-20T10:27:38.106161+0000","last_undegraded":"2024-04-20T10:27:38.106161+0000","last_fullsized":"2024-04-20T10:27:38.106161+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:06.166496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108148+0000","last_change":"2024-04-20T10:27:37.125295+0000","last_active":"2024-04-20T10:27:38.108148+0000","last_peered":"2024-04-20T10:27:38.108148+0000","last_clean":"2024-04-20T10:27:38.108148+0000","last_became_active":"2024-04-20T10:27:37.124922+0000","last_became_peered":"2024-04-20T10:27:37.124922+0000","last_unstale":"2024-04-20T10:27:38.108148+0000","last_undegraded":"2024-04-20T10:27:38.108148+0000","last_fullsized":"2024-04-20T10:27:38.108148+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:38:45.803688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5,1],"acting":[3,2,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108085+0000","last_change":"2024-04-20T10:27:33.009010+0000","last_active":"2024-04-20T10:27:38.108085+0000","last_peered":"2024-04-20T10:27:38.108085+0000","last_clean":"2024-04-20T10:27:38.108085+0000","last_became_active":"2024-04-20T10:27:33.008701+0000","last_became_peered":"2024-04-20T10:27:33.008701+0000","last_unstale":"2024-04-20T10:27:38.108085+0000","last_undegraded":"2024-04-20T10:27:38.108085+0000","last_fullsized":"2024-04-20T10:27:38.108085+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:27.266263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109186+0000","last_change":"2024-04-20T10:27:32.079095+0000","last_active":"2024-04-20T10:27:38.109186+0000","last_peered":"2024-04-20T10:27:38.109186+0000","last_clean":"2024-04-20T10:27:38.109186+0000","last_became_active":"2024-04-20T10:27:32.074687+0000","last_became_peered":"2024-04-20T10:27:32.074687+0000","last_unstale":"2024-04-20T10:27:38.109186+0000","last_undegraded":"2024-04-20T10:27:38.109186+0000","last_fullsized":"2024-04-20T10:27:38.109186+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:12:20.810691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.106749+0000","last_change":"2024-04-20T10:27:32.080751+0000","last_active":"2024-04-20T10:27:38.106749+0000","last_peered":"2024-04-20T10:27:38.106749+0000","last_clean":"2024-04-20T10:27:38.106749+0000","last_became_active":"2024-04-20T10:27:32.078604+0000","last_became_peered":"2024-04-20T10:27:32.078604+0000","last_unstale":"2024-04-20T10:27:38.106749+0000","last_undegraded":"2024-04-20T10:27:38.106749+0000","last_fullsized":"2024-04-20T10:27:38.106749+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:48:10.874813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390089+0000","last_change":"2024-04-20T10:27:33.024156+0000","last_active":"2024-04-20T10:27:38.390089+0000","last_peered":"2024-04-20T10:27:38.390089+0000","last_clean":"2024-04-20T10:27:38.390089+0000","last_became_active":"2024-04-20T10:27:33.023639+0000","last_became_peered":"2024-04-20T10:27:33.023639+0000","last_unstale":"2024-04-20T10:27:38.390089+0000","last_undegraded":"2024-04-20T10:27:38.390089+0000","last_fullsized":"2024-04-20T10:27:38.390089+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:02:45.132088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391353+0000","last_change":"2024-04-20T10:27:37.136584+0000","last_active":"2024-04-20T10:27:38.391353+0000","last_peered":"2024-04-20T10:27:38.391353+0000","last_clean":"2024-04-20T10:27:38.391353+0000","last_became_active":"2024-04-20T10:27:37.135997+0000","last_became_peered":"2024-04-20T10:27:37.135997+0000","last_unstale":"2024-04-20T10:27:38.391353+0000","last_undegraded":"2024-04-20T10:27:38.391353+0000","last_fullsized":"2024-04-20T10:27:38.391353+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:03:27.651374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,7],"acting":[4,6,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111655+0000","last_change":"2024-04-20T10:27:32.090018+0000","last_active":"2024-04-20T10:27:38.111655+0000","last_peered":"2024-04-20T10:27:38.111655+0000","last_clean":"2024-04-20T10:27:38.111655+0000","last_became_active":"2024-04-20T10:27:32.089144+0000","last_became_peered":"2024-04-20T10:27:32.089144+0000","last_unstale":"2024-04-20T10:27:38.111655+0000","last_undegraded":"2024-04-20T10:27:38.111655+0000","last_fullsized":"2024-04-20T10:27:38.111655+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:30.948915+0000","last_clean_scrub_stamp":"2024-04-20T10:27:30.948915+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:43:53.691511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390744+0000","last_change":"2024-04-20T10:27:33.010778+0000","last_active":"2024-04-20T10:27:38.390744+0000","last_peered":"2024-04-20T10:27:38.390744+0000","last_clean":"2024-04-20T10:27:38.390744+0000","last_became_active":"2024-04-20T10:27:33.010403+0000","last_became_peered":"2024-04-20T10:27:33.010403+0000","last_unstale":"2024-04-20T10:27:38.390744+0000","last_undegraded":"2024-04-20T10:27:38.390744+0000","last_fullsized":"2024-04-20T10:27:38.390744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:07.180902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.393172+0000","last_change":"2024-04-20T10:27:37.134702+0000","last_active":"2024-04-20T10:27:38.393172+0000","last_peered":"2024-04-20T10:27:38.393172+0000","last_clean":"2024-04-20T10:27:38.393172+0000","last_became_active":"2024-04-20T10:27:37.134178+0000","last_became_peered":"2024-04-20T10:27:37.134178+0000","last_unstale":"2024-04-20T10:27:38.393172+0000","last_undegraded":"2024-04-20T10:27:38.393172+0000","last_fullsized":"2024-04-20T10:27:38.393172+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:36.066954+0000","last_clean_scrub_stamp":"2024-04-20T10:27:36.066954+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:48.722800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1,0],"acting":[7,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390536+0000","last_change":"2024-04-20T10:27:32.998168+0000","last_active":"2024-04-20T10:27:38.390536+0000","last_peered":"2024-04-20T10:27:38.390536+0000","last_clean":"2024-04-20T10:27:38.390536+0000","last_became_active":"2024-04-20T10:27:32.996843+0000","last_became_peered":"2024-04-20T10:27:32.996843+0000","last_unstale":"2024-04-20T10:27:38.390536+0000","last_undegraded":"2024-04-20T10:27:38.390536+0000","last_fullsized":"2024-04-20T10:27:38.390536+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:13.408788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391052+0000","last_change":"2024-04-20T10:27:33.020874+0000","last_active":"2024-04-20T10:27:38.391052+0000","last_peered":"2024-04-20T10:27:38.391052+0000","last_clean":"2024-04-20T10:27:38.391052+0000","last_became_active":"2024-04-20T10:27:33.016375+0000","last_became_peered":"2024-04-20T10:27:33.016375+0000","last_unstale":"2024-04-20T10:27:38.391052+0000","last_undegraded":"2024-04-20T10:27:38.391052+0000","last_fullsized":"2024-04-20T10:27:38.391052+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105932+0000","last_change":"2024-04-20T10:27:33.011331+0000","last_active":"2024-04-20T10:27:38.105932+0000","last_peered":"2024-04-20T10:27:38.105932+0000","last_clean":"2024-04-20T10:27:38.105932+0000","last_became_active":"2024-04-20T10:27:33.010956+0000","last_became_peered":"2024-04-20T10:27:33.010956+0000","last_unstale":"2024-04-20T10:27:38.105932+0000","last_undegraded":"2024-04-20T10:27:38.105932+0000","last_fullsized":"2024-04-20T10:27:38.105932+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:35:16.511398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111379+0000","last_change":"2024-04-20T10:27:33.009386+0000","last_active":"2024-04-20T10:27:38.111379+0000","last_peered":"2024-04-20T10:27:38.111379+0000","last_clean":"2024-04-20T10:27:38.111379+0000","last_became_active":"2024-04-20T10:27:33.008596+0000","last_became_peered":"2024-04-20T10:27:33.008596+0000","last_unstale":"2024-04-20T10:27:38.111379+0000","last_undegraded":"2024-04-20T10:27:38.111379+0000","last_fullsized":"2024-04-20T10:27:38.111379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:23.737994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.109220+0000","last_change":"2024-04-20T10:27:33.001780+0000","last_active":"2024-04-20T10:27:38.109220+0000","last_peered":"2024-04-20T10:27:38.109220+0000","last_clean":"2024-04-20T10:27:38.109220+0000","last_became_active":"2024-04-20T10:27:33.000852+0000","last_became_peered":"2024-04-20T10:27:33.000852+0000","last_unstale":"2024-04-20T10:27:38.109220+0000","last_undegraded":"2024-04-20T10:27:38.109220+0000","last_fullsized":"2024-04-20T10:27:38.109220+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:17:52.213855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111488+0000","last_change":"2024-04-20T10:27:33.010234+0000","last_active":"2024-04-20T10:27:38.111488+0000","last_peered":"2024-04-20T10:27:38.111488+0000","last_clean":"2024-04-20T10:27:38.111488+0000","last_became_active":"2024-04-20T10:27:33.009934+0000","last_became_peered":"2024-04-20T10:27:33.009934+0000","last_unstale":"2024-04-20T10:27:38.111488+0000","last_undegraded":"2024-04-20T10:27:38.111488+0000","last_fullsized":"2024-04-20T10:27:38.111488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:16:11.492557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392171+0000","last_change":"2024-04-20T10:27:33.006041+0000","last_active":"2024-04-20T10:27:38.392171+0000","last_peered":"2024-04-20T10:27:38.392171+0000","last_clean":"2024-04-20T10:27:38.392171+0000","last_became_active":"2024-04-20T10:27:33.005259+0000","last_became_peered":"2024-04-20T10:27:33.005259+0000","last_unstale":"2024-04-20T10:27:38.392171+0000","last_undegraded":"2024-04-20T10:27:38.392171+0000","last_fullsized":"2024-04-20T10:27:38.392171+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112860+0000","last_change":"2024-04-20T10:27:33.015530+0000","last_active":"2024-04-20T10:27:38.112860+0000","last_peered":"2024-04-20T10:27:38.112860+0000","last_clean":"2024-04-20T10:27:38.112860+0000","last_became_active":"2024-04-20T10:27:33.015240+0000","last_became_peered":"2024-04-20T10:27:33.015240+0000","last_unstale":"2024-04-20T10:27:38.112860+0000","last_undegraded":"2024-04-20T10:27:38.112860+0000","last_fullsized":"2024-04-20T10:27:38.112860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105792+0000","last_change":"2024-04-20T10:27:33.015916+0000","last_active":"2024-04-20T10:27:38.105792+0000","last_peered":"2024-04-20T10:27:38.105792+0000","last_clean":"2024-04-20T10:27:38.105792+0000","last_became_active":"2024-04-20T10:27:33.015585+0000","last_became_peered":"2024-04-20T10:27:33.015585+0000","last_unstale":"2024-04-20T10:27:38.105792+0000","last_undegraded":"2024-04-20T10:27:38.105792+0000","last_fullsized":"2024-04-20T10:27:38.105792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112167+0000","last_change":"2024-04-20T10:27:33.014307+0000","last_active":"2024-04-20T10:27:38.112167+0000","last_peered":"2024-04-20T10:27:38.112167+0000","last_clean":"2024-04-20T10:27:38.112167+0000","last_became_active":"2024-04-20T10:27:33.013891+0000","last_became_peered":"2024-04-20T10:27:33.013891+0000","last_unstale":"2024-04-20T10:27:38.112167+0000","last_undegraded":"2024-04-20T10:27:38.112167+0000","last_fullsized":"2024-04-20T10:27:38.112167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:32.733184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390622+0000","last_change":"2024-04-20T10:27:33.010713+0000","last_active":"2024-04-20T10:27:38.390622+0000","last_peered":"2024-04-20T10:27:38.390622+0000","last_clean":"2024-04-20T10:27:38.390622+0000","last_became_active":"2024-04-20T10:27:33.009349+0000","last_became_peered":"2024-04-20T10:27:33.009349+0000","last_unstale":"2024-04-20T10:27:38.390622+0000","last_undegraded":"2024-04-20T10:27:38.390622+0000","last_fullsized":"2024-04-20T10:27:38.390622+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389798+0000","last_change":"2024-04-20T10:27:33.018481+0000","last_active":"2024-04-20T10:27:38.389798+0000","last_peered":"2024-04-20T10:27:38.389798+0000","last_clean":"2024-04-20T10:27:38.389798+0000","last_became_active":"2024-04-20T10:27:33.015328+0000","last_became_peered":"2024-04-20T10:27:33.015328+0000","last_unstale":"2024-04-20T10:27:38.389798+0000","last_undegraded":"2024-04-20T10:27:38.389798+0000","last_fullsized":"2024-04-20T10:27:38.389798+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:43:48.722150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388831+0000","last_change":"2024-04-20T10:27:33.006877+0000","last_active":"2024-04-20T10:27:38.388831+0000","last_peered":"2024-04-20T10:27:38.388831+0000","last_clean":"2024-04-20T10:27:38.388831+0000","last_became_active":"2024-04-20T10:27:33.006143+0000","last_became_peered":"2024-04-20T10:27:33.006143+0000","last_unstale":"2024-04-20T10:27:38.388831+0000","last_undegraded":"2024-04-20T10:27:38.388831+0000","last_fullsized":"2024-04-20T10:27:38.388831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:38.096666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108600+0000","last_change":"2024-04-20T10:27:33.016666+0000","last_active":"2024-04-20T10:27:38.108600+0000","last_peered":"2024-04-20T10:27:38.108600+0000","last_clean":"2024-04-20T10:27:38.108600+0000","last_became_active":"2024-04-20T10:27:33.016268+0000","last_became_peered":"2024-04-20T10:27:33.016268+0000","last_unstale":"2024-04-20T10:27:38.108600+0000","last_undegraded":"2024-04-20T10:27:38.108600+0000","last_fullsized":"2024-04-20T10:27:38.108600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:07.386901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.392668+0000","last_change":"2024-04-20T10:27:33.014742+0000","last_active":"2024-04-20T10:27:38.392668+0000","last_peered":"2024-04-20T10:27:38.392668+0000","last_clean":"2024-04-20T10:27:38.392668+0000","last_became_active":"2024-04-20T10:27:33.014443+0000","last_became_peered":"2024-04-20T10:27:33.014443+0000","last_unstale":"2024-04-20T10:27:38.392668+0000","last_undegraded":"2024-04-20T10:27:38.392668+0000","last_fullsized":"2024-04-20T10:27:38.392668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:15:57.153800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.108659+0000","last_change":"2024-04-20T10:27:33.002993+0000","last_active":"2024-04-20T10:27:38.108659+0000","last_peered":"2024-04-20T10:27:38.108659+0000","last_clean":"2024-04-20T10:27:38.108659+0000","last_became_active":"2024-04-20T10:27:33.002685+0000","last_became_peered":"2024-04-20T10:27:33.002685+0000","last_unstale":"2024-04-20T10:27:38.108659+0000","last_undegraded":"2024-04-20T10:27:38.108659+0000","last_fullsized":"2024-04-20T10:27:38.108659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:18:38.509857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.105782+0000","last_change":"2024-04-20T10:27:33.014659+0000","last_active":"2024-04-20T10:27:38.105782+0000","last_peered":"2024-04-20T10:27:38.105782+0000","last_clean":"2024-04-20T10:27:38.105782+0000","last_became_active":"2024-04-20T10:27:33.014188+0000","last_became_peered":"2024-04-20T10:27:33.014188+0000","last_unstale":"2024-04-20T10:27:38.105782+0000","last_undegraded":"2024-04-20T10:27:38.105782+0000","last_fullsized":"2024-04-20T10:27:38.105782+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107794+0000","last_change":"2024-04-20T10:27:33.011041+0000","last_active":"2024-04-20T10:27:38.107794+0000","last_peered":"2024-04-20T10:27:38.107794+0000","last_clean":"2024-04-20T10:27:38.107794+0000","last_became_active":"2024-04-20T10:27:33.010751+0000","last_became_peered":"2024-04-20T10:27:33.010751+0000","last_unstale":"2024-04-20T10:27:38.107794+0000","last_undegraded":"2024-04-20T10:27:38.107794+0000","last_fullsized":"2024-04-20T10:27:38.107794+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:43:14.531021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.110132+0000","last_change":"2024-04-20T10:27:33.005787+0000","last_active":"2024-04-20T10:27:38.110132+0000","last_peered":"2024-04-20T10:27:38.110132+0000","last_clean":"2024-04-20T10:27:38.110132+0000","last_became_active":"2024-04-20T10:27:33.004692+0000","last_became_peered":"2024-04-20T10:27:33.004692+0000","last_unstale":"2024-04-20T10:27:38.110132+0000","last_undegraded":"2024-04-20T10:27:38.110132+0000","last_fullsized":"2024-04-20T10:27:38.110132+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:08.977393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112010+0000","last_change":"2024-04-20T10:27:33.010711+0000","last_active":"2024-04-20T10:27:38.112010+0000","last_peered":"2024-04-20T10:27:38.112010+0000","last_clean":"2024-04-20T10:27:38.112010+0000","last_became_active":"2024-04-20T10:27:33.010282+0000","last_became_peered":"2024-04-20T10:27:33.010282+0000","last_unstale":"2024-04-20T10:27:38.112010+0000","last_undegraded":"2024-04-20T10:27:38.112010+0000","last_fullsized":"2024-04-20T10:27:38.112010+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:52:06.224357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388721+0000","last_change":"2024-04-20T10:27:33.008425+0000","last_active":"2024-04-20T10:27:38.388721+0000","last_peered":"2024-04-20T10:27:38.388721+0000","last_clean":"2024-04-20T10:27:38.388721+0000","last_became_active":"2024-04-20T10:27:33.006532+0000","last_became_peered":"2024-04-20T10:27:33.006532+0000","last_unstale":"2024-04-20T10:27:38.388721+0000","last_undegraded":"2024-04-20T10:27:38.388721+0000","last_fullsized":"2024-04-20T10:27:38.388721+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:41.673162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388300+0000","last_change":"2024-04-20T10:27:33.018484+0000","last_active":"2024-04-20T10:27:38.388300+0000","last_peered":"2024-04-20T10:27:38.388300+0000","last_clean":"2024-04-20T10:27:38.388300+0000","last_became_active":"2024-04-20T10:27:33.018257+0000","last_became_peered":"2024-04-20T10:27:33.018257+0000","last_unstale":"2024-04-20T10:27:38.388300+0000","last_undegraded":"2024-04-20T10:27:38.388300+0000","last_fullsized":"2024-04-20T10:27:38.388300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:32:37.664306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.388715+0000","last_change":"2024-04-20T10:27:33.022995+0000","last_active":"2024-04-20T10:27:38.388715+0000","last_peered":"2024-04-20T10:27:38.388715+0000","last_clean":"2024-04-20T10:27:38.388715+0000","last_became_active":"2024-04-20T10:27:33.017740+0000","last_became_peered":"2024-04-20T10:27:33.017740+0000","last_unstale":"2024-04-20T10:27:38.388715+0000","last_undegraded":"2024-04-20T10:27:38.388715+0000","last_fullsized":"2024-04-20T10:27:38.388715+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:16:15.074555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391590+0000","last_change":"2024-04-20T10:27:33.008312+0000","last_active":"2024-04-20T10:27:38.391590+0000","last_peered":"2024-04-20T10:27:38.391590+0000","last_clean":"2024-04-20T10:27:38.391590+0000","last_became_active":"2024-04-20T10:27:33.002597+0000","last_became_peered":"2024-04-20T10:27:33.002597+0000","last_unstale":"2024-04-20T10:27:38.391590+0000","last_undegraded":"2024-04-20T10:27:38.391590+0000","last_fullsized":"2024-04-20T10:27:38.391590+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:35:56.271071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.107489+0000","last_change":"2024-04-20T10:27:33.010109+0000","last_active":"2024-04-20T10:27:38.107489+0000","last_peered":"2024-04-20T10:27:38.107489+0000","last_clean":"2024-04-20T10:27:38.107489+0000","last_became_active":"2024-04-20T10:27:33.009838+0000","last_became_peered":"2024-04-20T10:27:33.009838+0000","last_unstale":"2024-04-20T10:27:38.107489+0000","last_undegraded":"2024-04-20T10:27:38.107489+0000","last_fullsized":"2024-04-20T10:27:38.107489+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:09:44.619166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.387903+0000","last_change":"2024-04-20T10:27:33.024641+0000","last_active":"2024-04-20T10:27:38.387903+0000","last_peered":"2024-04-20T10:27:38.387903+0000","last_clean":"2024-04-20T10:27:38.387903+0000","last_became_active":"2024-04-20T10:27:33.024410+0000","last_became_peered":"2024-04-20T10:27:33.024410+0000","last_unstale":"2024-04-20T10:27:38.387903+0000","last_undegraded":"2024-04-20T10:27:38.387903+0000","last_fullsized":"2024-04-20T10:27:38.387903+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.391138+0000","last_change":"2024-04-20T10:27:33.025226+0000","last_active":"2024-04-20T10:27:38.391138+0000","last_peered":"2024-04-20T10:27:38.391138+0000","last_clean":"2024-04-20T10:27:38.391138+0000","last_became_active":"2024-04-20T10:27:33.024311+0000","last_became_peered":"2024-04-20T10:27:33.024311+0000","last_unstale":"2024-04-20T10:27:38.391138+0000","last_undegraded":"2024-04-20T10:27:38.391138+0000","last_fullsized":"2024-04-20T10:27:38.391138+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:45:05.589551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389207+0000","last_change":"2024-04-20T10:27:33.022586+0000","last_active":"2024-04-20T10:27:38.389207+0000","last_peered":"2024-04-20T10:27:38.389207+0000","last_clean":"2024-04-20T10:27:38.389207+0000","last_became_active":"2024-04-20T10:27:33.022293+0000","last_became_peered":"2024-04-20T10:27:33.022293+0000","last_unstale":"2024-04-20T10:27:38.389207+0000","last_undegraded":"2024-04-20T10:27:38.389207+0000","last_fullsized":"2024-04-20T10:27:38.389207+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:05:47.564499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.390946+0000","last_change":"2024-04-20T10:27:33.019615+0000","last_active":"2024-04-20T10:27:38.390946+0000","last_peered":"2024-04-20T10:27:38.390946+0000","last_clean":"2024-04-20T10:27:38.390946+0000","last_became_active":"2024-04-20T10:27:33.019288+0000","last_became_peered":"2024-04-20T10:27:33.019288+0000","last_unstale":"2024-04-20T10:27:38.390946+0000","last_undegraded":"2024-04-20T10:27:38.390946+0000","last_fullsized":"2024-04-20T10:27:38.390946+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:19:52.897903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.111457+0000","last_change":"2024-04-20T10:27:33.019815+0000","last_active":"2024-04-20T10:27:38.111457+0000","last_peered":"2024-04-20T10:27:38.111457+0000","last_clean":"2024-04-20T10:27:38.111457+0000","last_became_active":"2024-04-20T10:27:33.019369+0000","last_became_peered":"2024-04-20T10:27:33.019369+0000","last_unstale":"2024-04-20T10:27:38.111457+0000","last_undegraded":"2024-04-20T10:27:38.111457+0000","last_fullsized":"2024-04-20T10:27:38.111457+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:49:24.821903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.112358+0000","last_change":"2024-04-20T10:27:33.015773+0000","last_active":"2024-04-20T10:27:38.112358+0000","last_peered":"2024-04-20T10:27:38.112358+0000","last_clean":"2024-04-20T10:27:38.112358+0000","last_became_active":"2024-04-20T10:27:33.015430+0000","last_became_peered":"2024-04-20T10:27:33.015430+0000","last_unstale":"2024-04-20T10:27:38.112358+0000","last_undegraded":"2024-04-20T10:27:38.112358+0000","last_fullsized":"2024-04-20T10:27:38.112358+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:42:23.290532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-20T10:27:38.389282+0000","last_change":"2024-04-20T10:27:33.024105+0000","last_active":"2024-04-20T10:27:38.389282+0000","last_peered":"2024-04-20T10:27:38.389282+0000","last_clean":"2024-04-20T10:27:38.389282+0000","last_became_active":"2024-04-20T10:27:33.023418+0000","last_became_peered":"2024-04-20T10:27:33.023418+0000","last_unstale":"2024-04-20T10:27:38.389282+0000","last_undegraded":"2024-04-20T10:27:38.389282+0000","last_fullsized":"2024-04-20T10:27:38.389282+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:31.963320+0000","last_clean_scrub_stamp":"2024-04-20T10:27:31.963320+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:48:29.069399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509451,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":110285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":700745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509451,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31468,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340372,"statfs":{"total":96636764160,"available":96604540928,"internally_reserved":0,"allocated":286720,"data_stored":110867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509451,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":294912,"data_stored":114127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":880640,"data_stored":700723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509451,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":290816,"data_stored":110341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542155,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":296,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":303104,"data_stored":114799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":4514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:28:01.933 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T10:28:01.933 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-20T10:28:01.934 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-20T10:28:01.934 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:28:02.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.178+0000 7f0f57200640 1 Processor -- start 2024-04-20T10:28:02.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.179+0000 7f0f57200640 1 -- start start 2024-04-20T10:28:02.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.180+0000 7f0f57200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0f58161d50 0x7f0f580d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.180+0000 7f0f57200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f580b3330 0x7f0f580b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.180+0000 7f0f57200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0f5806c190 con 0x7f0f5806c6d0 2024-04-20T10:28:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.180+0000 7f0f57200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0f58162130 con 0x7f0f58161d50 2024-04-20T10:28:02.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.180+0000 7f0f57200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5806c470 con 0x7f0f580b3330 2024-04-20T10:28:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.181+0000 7f0f55e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0f58161d50 0x7f0f580d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.181+0000 7f0f55400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f580b3330 0x7f0f580b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.181+0000 7f0f55e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0f58161d50 0x7f0f580d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:41126/0 (socket says 172.21.15.47:41126) 2024-04-20T10:28:02.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.181+0000 7f0f55e00640 1 -- 172.21.15.47:0/2545637971 learned_addr learned my addr 172.21.15.47:0/2545637971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.182+0000 7f0f55e00640 1 -- 172.21.15.47:0/2545637971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f580b3330 msgr2=0x7f0f580b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.182+0000 7f0f55e00640 1 --2- 172.21.15.47:0/2545637971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f580b3330 0x7f0f580b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.182+0000 7f0f55e00640 1 -- 172.21.15.47:0/2545637971 >> v1:172.21.15.175:6789/0 conn(0x7f0f5806c6d0 legacy=0x7f0f580cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.182+0000 7f0f55e00640 1 -- 172.21.15.47:0/2545637971 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f580d2e00 con 0x7f0f58161d50 2024-04-20T10:28:02.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.183+0000 7f0f55e00640 1 --2- 172.21.15.47:0/2545637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f58161d50 0x7f0f580d38e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009870 tx=0x7f0f4802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3bbcdb04e8e3032 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.183+0000 7f0f54a00640 1 -- 172.21.15.47:0/2545637971 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48034070 con 0x7f0f58161d50 2024-04-20T10:28:02.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.183+0000 7f0f54a00640 1 -- 172.21.15.47:0/2545637971 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f48030840 con 0x7f0f58161d50 2024-04-20T10:28:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.183+0000 7f0f54a00640 1 -- 172.21.15.47:0/2545637971 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48030c10 con 0x7f0f58161d50 2024-04-20T10:28:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 -- 172.21.15.47:0/2545637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f58161d50 msgr2=0x7f0f580d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 --2- 172.21.15.47:0/2545637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f58161d50 0x7f0f580d38e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0f48009870 tx=0x7f0f4802b980 comp rx=0 tx=0).stop 2024-04-20T10:28:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 -- 172.21.15.47:0/2545637971 shutdown_connections 2024-04-20T10:28:02.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 --2- 172.21.15.47:0/2545637971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f580b3330 0x7f0f580b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 --2- 172.21.15.47:0/2545637971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f58161d50 0x7f0f580d38e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 -- 172.21.15.47:0/2545637971 >> 172.21.15.47:0/2545637971 conn(0x7f0f580d4450 msgr2=0x7f0f580d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:02.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 -- 172.21.15.47:0/2545637971 shutdown_connections 2024-04-20T10:28:02.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.184+0000 7f0f57200640 1 -- 172.21.15.47:0/2545637971 wait complete. 2024-04-20T10:28:02.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.186+0000 7f0f57200640 1 Processor -- start 2024-04-20T10:28:02.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.186+0000 7f0f57200640 1 -- start start 2024-04-20T10:28:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f57200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f5806c6d0 0x7f0f5814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f57200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f580b3330 0x7f0f580cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f57200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f58161d50 0x7f0f580cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f57200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5806cb60 con 0x7f0f5806c6d0 2024-04-20T10:28:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f57200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f580d33c0 con 0x7f0f580b3330 2024-04-20T10:28:02.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f57200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5806cce0 con 0x7f0f58161d50 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f55e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f5806c6d0 0x7f0f5814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f55e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f5806c6d0 0x7f0f5814a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56240/0 (socket says 172.21.15.47:56240) 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.187+0000 7f0f55e00640 1 -- 172.21.15.47:0/138698070 learned_addr learned my addr 172.21.15.47:0/138698070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55400640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f580b3330 0x7f0f580cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f56800640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f58161d50 0x7f0f580cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55e00640 1 -- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f58161d50 msgr2=0x7f0f580cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55e00640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f58161d50 0x7f0f580cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55e00640 1 -- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f580b3330 msgr2=0x7f0f580cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55e00640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f580b3330 0x7f0f580cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55e00640 1 -- 172.21.15.47:0/138698070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5815d8e0 con 0x7f0f5806c6d0 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55400640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f580b3330 0x7f0f580cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f56800640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f58161d50 0x7f0f580cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f55e00640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f5806c6d0 0x7f0f5814a8c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802b9c0 tx=0x7f0f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.188+0000 7f0f4e000640 1 -- 172.21.15.47:0/138698070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48034040 con 0x7f0f5806c6d0 2024-04-20T10:28:02.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.189+0000 7f0f4e000640 1 -- 172.21.15.47:0/138698070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f48039d90 con 0x7f0f5806c6d0 2024-04-20T10:28:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.189+0000 7f0f4e000640 1 -- 172.21.15.47:0/138698070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48043630 con 0x7f0f5806c6d0 2024-04-20T10:28:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.189+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5815dbc0 con 0x7f0f5806c6d0 2024-04-20T10:28:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.189+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f580c8780 con 0x7f0f5806c6d0 2024-04-20T10:28:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.190+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f58075b60 con 0x7f0f5806c6d0 2024-04-20T10:28:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.191+0000 7f0f4e000640 1 -- 172.21.15.47:0/138698070 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0f48037070 con 0x7f0f5806c6d0 2024-04-20T10:28:02.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.192+0000 7f0f4e000640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0f3406c8f0 0x7f0f3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.192+0000 7f0f4e000640 1 -- 172.21.15.47:0/138698070 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0f480adff0 con 0x7f0f5806c6d0 2024-04-20T10:28:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.192+0000 7f0f55400640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0f3406c8f0 0x7f0f3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.193+0000 7f0f55400640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0f3406c8f0 0x7f0f3406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0f5815bd80 tx=0x7f0f4000b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:02.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.196+0000 7f0f4e000640 1 -- 172.21.15.47:0/138698070 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0f4803e070 con 0x7f0f5806c6d0 2024-04-20T10:28:02.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.436+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f0f580676c0 con 0x7f0f5806c6d0 2024-04-20T10:28:02.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.437+0000 7f0f4e000640 1 -- 172.21.15.47:0/138698070 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f0f4807c1a0 con 0x7f0f5806c6d0 2024-04-20T10:28:02.437 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:02.437 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.439+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0f3406c8f0 msgr2=0x7f0f3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.439+0000 7f0f57200640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0f3406c8f0 0x7f0f3406edb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f0f5815bd80 tx=0x7f0f4000b040 comp rx=0 tx=0).stop 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.439+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f5806c6d0 msgr2=0x7f0f5814a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.439+0000 7f0f57200640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f5806c6d0 0x7f0f5814a8c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0f4802b9c0 tx=0x7f0f480046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 shutdown_connections 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f58161d50 0x7f0f580cd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f580b3330 0x7f0f580cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0f3406c8f0 0x7f0f3406edb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 --2- 172.21.15.47:0/138698070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0f5806c6d0 0x7f0f5814a8c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 >> 172.21.15.47:0/138698070 conn(0x7f0f580d4450 msgr2=0x7f0f580d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:02.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 shutdown_connections 2024-04-20T10:28:02.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.440+0000 7f0f57200640 1 -- 172.21.15.47:0/138698070 wait complete. 2024-04-20T10:28:02.458 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-20T10:28:02.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:02.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.647+0000 7f019dc00640 1 Processor -- start 2024-04-20T10:28:02.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.647+0000 7f019dc00640 1 -- start start 2024-04-20T10:28:02.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.648+0000 7f019dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 0x7f0198071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.648+0000 7f019dc00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01980ade40 con 0x7f019815b3b0 2024-04-20T10:28:02.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.648+0000 7f019dc00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01980ae0e0 con 0x7f0198072090 2024-04-20T10:28:02.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.648+0000 7f019dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f019806f1b0 con 0x7f019806dcb0 2024-04-20T10:28:02.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0196a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f019815b3b0 0x7f019806eaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33808/0 (socket says 172.21.15.47:33808) 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0196a00640 1 -- 172.21.15.47:0/3895991935 learned_addr learned my addr 172.21.15.47:0/3895991935 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0196000640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 0x7f0198071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1230757263 0 0) 0x7f01980ade40 con 0x7f019815b3b0 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f017c003520 con 0x7f019815b3b0 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4004282854 0 0) 0x7f017c003520 con 0x7f019815b3b0 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 msgr2=0x7f0198071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.649+0000 7f0195600640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 0x7f0198071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 >> v1:172.21.15.175:6789/0 conn(0x7f0198072090 legacy=0x7f01980d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019815b790 con 0x7f019815b3b0 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f0180001690 con 0x7f019815b3b0 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 >> v1:172.21.15.47:6789/0 conn(0x7f019815b3b0 legacy=0x7f019806eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c0039e0 0x7f017c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 0x7f017c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f017c004d40 0x7f017c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f019806f1b0 con 0x7f017c004d40 2024-04-20T10:28:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f017c00ba70 con 0x7f017c0039e0 2024-04-20T10:28:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f017c00bc70 con 0x7f019806dcb0 2024-04-20T10:28:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0197400640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f017c004d40 0x7f017c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0196a00640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c0039e0 0x7f017c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.650+0000 7f0196000640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 0x7f017c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.651+0000 7f0197400640 1 -- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 msgr2=0x7f017c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.651+0000 7f0197400640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 0x7f017c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.651+0000 7f0197400640 1 -- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c0039e0 msgr2=0x7f017c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.651+0000 7f0197400640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c0039e0 0x7f017c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.651+0000 7f0197400640 1 -- 172.21.15.47:0/3895991935 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f017c003520 con 0x7f017c004d40 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.651+0000 7f0196a00640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c0039e0 0x7f017c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.651+0000 7f0196000640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f019806dcb0 0x7f017c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.652+0000 7f0197400640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f017c004d40 0x7f017c0051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f01980b3120 tx=0x7f017c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4d8e96f263e3fa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.652+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190001a70 con 0x7f017c004d40 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.652+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0190003b70 con 0x7f017c004d40 2024-04-20T10:28:02.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.652+0000 7f0195600640 1 -- 172.21.15.47:0/3895991935 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01900050d0 con 0x7f017c004d40 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.653+0000 7f019dc00640 1 -- 172.21.15.47:0/3895991935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f017c004d40 msgr2=0x7f017c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.653+0000 7f019dc00640 1 --2- 172.21.15.47:0/3895991935 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f017c004d40 0x7f017c0051c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f01980b3120 tx=0x7f017c005830 comp rx=0 tx=0).stop 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.653+0000 7f0197400640 1 -- 172.21.15.47:0/3895991935 reap_dead start 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.653+0000 7f019dc00640 1 -- 172.21.15.47:0/3895991935 shutdown_connections 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.653+0000 7f019dc00640 1 -- 172.21.15.47:0/3895991935 >> 172.21.15.47:0/3895991935 conn(0x7f01980b3350 msgr2=0x7f01980b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.653+0000 7f019dc00640 1 -- 172.21.15.47:0/3895991935 shutdown_connections 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.653+0000 7f019dc00640 1 -- 172.21.15.47:0/3895991935 wait complete. 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.655+0000 7f019dc00640 1 Processor -- start 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.655+0000 7f019dc00640 1 -- start start 2024-04-20T10:28:02.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f019dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c004d40 0x7f019809d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f019dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0198151ea0 0x7f0198153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f019dc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981522e0 0x7f0198153f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f019dc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01980ae940 con 0x7f01981522e0 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f019dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01980ae6e0 con 0x7f017c004d40 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f019dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01980aec20 con 0x7f0198151ea0 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f0197400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981522e0 0x7f0198153f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f0197400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981522e0 0x7f0198153f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56264/0 (socket says 172.21.15.47:56264) 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f0197400640 1 -- 172.21.15.47:0/2250799053 learned_addr learned my addr 172.21.15.47:0/2250799053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f0196a00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c004d40 0x7f019809d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.656+0000 7f0197400640 1 -- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0198151ea0 msgr2=0x7f0198153a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f0197400640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0198151ea0 0x7f0198153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f0197400640 1 -- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c004d40 msgr2=0x7f019809d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f0197400640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c004d40 0x7f019809d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f0197400640 1 -- 172.21.15.47:0/2250799053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019814f2f0 con 0x7f01981522e0 2024-04-20T10:28:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f0196a00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c004d40 0x7f019809d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f0197400640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981522e0 0x7f0198153f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f019002bf30 tx=0x7f01900014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f018ea00640 1 -- 172.21.15.47:0/2250799053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190001a70 con 0x7f01981522e0 2024-04-20T10:28:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f018ea00640 1 -- 172.21.15.47:0/2250799053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0190003970 con 0x7f01981522e0 2024-04-20T10:28:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f018ea00640 1 -- 172.21.15.47:0/2250799053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190006d50 con 0x7f01981522e0 2024-04-20T10:28:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019814f5d0 con 0x7f01981522e0 2024-04-20T10:28:02.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.657+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019814fb90 con 0x7f01981522e0 2024-04-20T10:28:02.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.658+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0198068980 con 0x7f01981522e0 2024-04-20T10:28:02.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.659+0000 7f018ea00640 1 -- 172.21.15.47:0/2250799053 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0190007030 con 0x7f01981522e0 2024-04-20T10:28:02.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.660+0000 7f018ea00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f017406c8f0 0x7f017406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:02.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.660+0000 7f018ea00640 1 -- 172.21.15.47:0/2250799053 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f01900acf10 con 0x7f01981522e0 2024-04-20T10:28:02.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.661+0000 7f0196a00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f017406c8f0 0x7f017406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:02.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.662+0000 7f0196a00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f017406c8f0 0x7f017406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0180001220 tx=0x7f0180001190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:02.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.665+0000 7f018ea00640 1 -- 172.21.15.47:0/2250799053 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0190033020 con 0x7f01981522e0 2024-04-20T10:28:02.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.871+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0198151b60 con 0x7f01981522e0 2024-04-20T10:28:02.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.874+0000 7f018ea00640 1 -- 172.21.15.47:0/2250799053 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7f019007fe20 con 0x7f01981522e0 2024-04-20T10:28:02.874 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:02.875 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:02.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f017406c8f0 msgr2=0x7f017406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f017406c8f0 0x7f017406edb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0180001220 tx=0x7f0180001190 comp rx=0 tx=0).stop 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981522e0 msgr2=0x7f0198153f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981522e0 0x7f0198153f40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f019002bf30 tx=0x7f01900014a0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 shutdown_connections 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f017406c8f0 0x7f017406edb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01981522e0 0x7f0198153f40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0198151ea0 0x7f0198153a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 --2- 172.21.15.47:0/2250799053 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f017c004d40 0x7f019809d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 >> 172.21.15.47:0/2250799053 conn(0x7f01980b3350 msgr2=0x7f0198157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 shutdown_connections 2024-04-20T10:28:02.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:02.877+0000 7f019dc00640 1 -- 172.21.15.47:0/2250799053 wait complete. 2024-04-20T10:28:02.896 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:02.897 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:03.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.110+0000 7f9289000640 1 Processor -- start 2024-04-20T10:28:03.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.111+0000 7f9289000640 1 -- start start 2024-04-20T10:28:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.112+0000 7f9289000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f92840b11c0 0x7f92840d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.112+0000 7f9289000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92840d23b0 con 0x7f928415e9c0 2024-04-20T10:28:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.112+0000 7f9289000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f92840d2690 con 0x7f92840d4930 2024-04-20T10:28:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.112+0000 7f9289000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f92840d0f20 con 0x7f92840b11c0 2024-04-20T10:28:03.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.112+0000 7f9281e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f928415e9c0 0x7f92840d0740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33822/0 (socket says 172.21.15.47:33822) 2024-04-20T10:28:03.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.112+0000 7f9281e00640 1 -- 172.21.15.47:0/1267674125 learned_addr learned my addr 172.21.15.47:0/1267674125 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:03.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.113+0000 7f927be00640 1 --2- 172.21.15.47:0/1267674125 >> v2:172.21.15.47:3300/0 conn(0x7f92840b11c0 0x7f92840d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.113+0000 7f927be00640 1 -- 172.21.15.47:0/1267674125 >> v1:172.21.15.175:6789/0 conn(0x7f92840d4930 legacy=0x7f92840d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.113+0000 7f927be00640 1 -- 172.21.15.47:0/1267674125 >> v1:172.21.15.47:6789/0 conn(0x7f928415e9c0 legacy=0x7f92840d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.113+0000 7f9281400640 1 -- 172.21.15.47:0/1267674125 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3399357147 0 0) 0x7f92840d23b0 con 0x7f928415e9c0 2024-04-20T10:28:03.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.113+0000 7f927be00640 1 -- 172.21.15.47:0/1267674125 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92840d2970 con 0x7f92840b11c0 2024-04-20T10:28:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.114+0000 7f927be00640 1 --2- 172.21.15.47:0/1267674125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92840b11c0 0x7f92840d1ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f926c0097a0 tx=0x7f926c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73b26f8f40ac6ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9281400640 1 -- 172.21.15.47:0/1267674125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c035020 con 0x7f92840b11c0 2024-04-20T10:28:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9281400640 1 -- 172.21.15.47:0/1267674125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f926c031830 con 0x7f92840b11c0 2024-04-20T10:28:03.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9281400640 1 -- 172.21.15.47:0/1267674125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f926c031c00 con 0x7f92840b11c0 2024-04-20T10:28:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9289000640 1 -- 172.21.15.47:0/1267674125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92840b11c0 msgr2=0x7f92840d1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9289000640 1 --2- 172.21.15.47:0/1267674125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92840b11c0 0x7f92840d1ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f926c0097a0 tx=0x7f926c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9289000640 1 -- 172.21.15.47:0/1267674125 shutdown_connections 2024-04-20T10:28:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9289000640 1 --2- 172.21.15.47:0/1267674125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f92840b11c0 0x7f92840d1ba0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.115+0000 7f9289000640 1 -- 172.21.15.47:0/1267674125 >> 172.21.15.47:0/1267674125 conn(0x7f92840b9600 msgr2=0x7f92840b9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.116+0000 7f9289000640 1 -- 172.21.15.47:0/1267674125 shutdown_connections 2024-04-20T10:28:03.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.116+0000 7f9289000640 1 -- 172.21.15.47:0/1267674125 wait complete. 2024-04-20T10:28:03.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.117+0000 7f9289000640 1 Processor -- start 2024-04-20T10:28:03.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.117+0000 7f9289000640 1 -- start start 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.118+0000 7f9289000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92840b11c0 0x7f92840b6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.118+0000 7f9289000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92840d4930 0x7f928415a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.118+0000 7f9289000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f928415e9c0 0x7f92840b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.118+0000 7f9289000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f92840d5170 con 0x7f928415e9c0 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.118+0000 7f9289000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92840d4ff0 con 0x7f92840b11c0 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.118+0000 7f9289000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f92840d2ce0 con 0x7f92840d4930 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f928415e9c0 0x7f92840b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f928415e9c0 0x7f92840b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56286/0 (socket says 172.21.15.47:56286) 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 -- 172.21.15.47:0/1795430855 learned_addr learned my addr 172.21.15.47:0/1795430855 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f927be00640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92840d4930 0x7f928415a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9281e00640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92840b11c0 0x7f92840b6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 -- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92840d4930 msgr2=0x7f928415a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92840d4930 0x7f928415a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 -- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92840b11c0 msgr2=0x7f92840b6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92840b11c0 0x7f92840b6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.119+0000 7f9282800640 1 -- 172.21.15.47:0/1795430855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928415f0f0 con 0x7f928415e9c0 2024-04-20T10:28:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.120+0000 7f9282800640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f928415e9c0 0x7f92840b5900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c7a0 tx=0x7f927c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.120+0000 7f927a000640 1 -- 172.21.15.47:0/1795430855 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c003b70 con 0x7f928415e9c0 2024-04-20T10:28:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.120+0000 7f927a000640 1 -- 172.21.15.47:0/1795430855 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f927c003d30 con 0x7f928415e9c0 2024-04-20T10:28:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.120+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f928415f3b0 con 0x7f928415e9c0 2024-04-20T10:28:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.120+0000 7f927a000640 1 -- 172.21.15.47:0/1795430855 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f927c00aa90 con 0x7f928415e9c0 2024-04-20T10:28:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.120+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f928415aec0 con 0x7f928415e9c0 2024-04-20T10:28:03.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.121+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9284068980 con 0x7f928415e9c0 2024-04-20T10:28:03.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.122+0000 7f927a000640 1 -- 172.21.15.47:0/1795430855 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f927c00ac30 con 0x7f928415e9c0 2024-04-20T10:28:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.123+0000 7f927a000640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f926006c8f0 0x7f926006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.123+0000 7f927a000640 1 -- 172.21.15.47:0/1795430855 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f927c08be40 con 0x7f928415e9c0 2024-04-20T10:28:03.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.124+0000 7f9281e00640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f926006c8f0 0x7f926006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.125+0000 7f9281e00640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f926006c8f0 0x7f926006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9274000f60 tx=0x7f9274007d20 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:03.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.128+0000 7f927a000640 1 -- 172.21.15.47:0/1795430855 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f927c059f20 con 0x7f928415e9c0 2024-04-20T10:28:03.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.332+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f928406efb0 con 0x7f928415e9c0 2024-04-20T10:28:03.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.333+0000 7f927a000640 1 -- 172.21.15.47:0/1795430855 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f927c059820 con 0x7f928415e9c0 2024-04-20T10:28:03.333 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.335+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f926006c8f0 msgr2=0x7f926006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.335+0000 7f9289000640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f926006c8f0 0x7f926006edb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f9274000f60 tx=0x7f9274007d20 comp rx=0 tx=0).stop 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.335+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f928415e9c0 msgr2=0x7f92840b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.335+0000 7f9289000640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f928415e9c0 0x7f92840b5900 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f927c00c7a0 tx=0x7f927c00cc60 comp rx=0 tx=0).stop 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 shutdown_connections 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f926006c8f0 0x7f926006edb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f928415e9c0 0x7f92840b5900 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92840d4930 0x7f928415a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 --2- 172.21.15.47:0/1795430855 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f92840b11c0 0x7f92840b6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 >> 172.21.15.47:0/1795430855 conn(0x7f92840b9600 msgr2=0x7f92840b3280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 shutdown_connections 2024-04-20T10:28:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.336+0000 7f9289000640 1 -- 172.21.15.47:0/1795430855 wait complete. 2024-04-20T10:28:03.355 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:03.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.572+0000 7f8c4c000640 1 Processor -- start 2024-04-20T10:28:03.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.572+0000 7f8c4c000640 1 -- start start 2024-04-20T10:28:03.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.573+0000 7f8c4c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 0x7f8c440d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.573+0000 7f8c4c000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c4406df70 con 0x7f8c4406fc30 2024-04-20T10:28:03.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.573+0000 7f8c4c000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8c4406c720 con 0x7f8c4406f090 2024-04-20T10:28:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.573+0000 7f8c4c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c44162fe0 con 0x7f8c44162c00 2024-04-20T10:28:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.574+0000 7f8c4a000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8c4406fc30 0x7f8c4406c010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33826/0 (socket says 172.21.15.47:33826) 2024-04-20T10:28:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.574+0000 7f8c4a000640 1 -- 172.21.15.47:0/3247422318 learned_addr learned my addr 172.21.15.47:0/3247422318 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:03.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.574+0000 7f8c49600640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 0x7f8c440d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.574+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 125832268 0 0) 0x7f8c4406df70 con 0x7f8c4406fc30 2024-04-20T10:28:03.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8c2c003520 con 0x7f8c4406fc30 2024-04-20T10:28:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4058325991 0 0) 0x7f8c2c003520 con 0x7f8c4406fc30 2024-04-20T10:28:03.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 msgr2=0x7f8c440d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 0x7f8c440d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 >> v1:172.21.15.175:6789/0 conn(0x7f8c4406f090 legacy=0x7f8c4406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c4406e310 con 0x7f8c4406fc30 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f8c3c001670 con 0x7f8c4406fc30 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 >> v1:172.21.15.47:6789/0 conn(0x7f8c4406fc30 legacy=0x7f8c4406c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c2c0039e0 0x7f8c2c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2c004800 0x7f8c2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 0x7f8c2c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c00bb50 con 0x7f8c2c0039e0 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c2c00bcd0 con 0x7f8c2c004800 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c2c00bed0 con 0x7f8c44162c00 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.575+0000 7f8c49600640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c2c0039e0 0x7f8c2c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c48c00640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2c004800 0x7f8c2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c4a000640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 0x7f8c2c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c49600640 1 -- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 msgr2=0x7f8c2c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c49600640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 0x7f8c2c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c49600640 1 -- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2c004800 msgr2=0x7f8c2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c49600640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2c004800 0x7f8c2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c49600640 1 -- 172.21.15.47:0/3247422318 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c2c003520 con 0x7f8c2c0039e0 2024-04-20T10:28:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c4a000640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c44162c00 0x7f8c2c005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:03.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.576+0000 7f8c48c00640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c2c004800 0x7f8c2c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.577+0000 7f8c49600640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c2c0039e0 0x7f8c2c003e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009fc0 tx=0x7f8c2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31342ec0ed02fc8f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.577+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38039070 con 0x7f8c2c0039e0 2024-04-20T10:28:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.577+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3802cbf0 con 0x7f8c2c0039e0 2024-04-20T10:28:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.577+0000 7f8c43e00640 1 -- 172.21.15.47:0/3247422318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c38009c10 con 0x7f8c2c0039e0 2024-04-20T10:28:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.578+0000 7f8c4c000640 1 -- 172.21.15.47:0/3247422318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c2c0039e0 msgr2=0x7f8c2c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.578+0000 7f8c4c000640 1 --2- 172.21.15.47:0/3247422318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c2c0039e0 0x7f8c2c003e00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8c38009fc0 tx=0x7f8c2c005830 comp rx=0 tx=0).stop 2024-04-20T10:28:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.578+0000 7f8c4a000640 1 -- 172.21.15.47:0/3247422318 reap_dead start 2024-04-20T10:28:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.578+0000 7f8c4c000640 1 -- 172.21.15.47:0/3247422318 shutdown_connections 2024-04-20T10:28:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.578+0000 7f8c4c000640 1 -- 172.21.15.47:0/3247422318 >> 172.21.15.47:0/3247422318 conn(0x7f8c440b3350 msgr2=0x7f8c440b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.578+0000 7f8c4c000640 1 -- 172.21.15.47:0/3247422318 shutdown_connections 2024-04-20T10:28:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.578+0000 7f8c4c000640 1 -- 172.21.15.47:0/3247422318 wait complete. 2024-04-20T10:28:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.580+0000 7f8c4c000640 1 Processor -- start 2024-04-20T10:28:03.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.580+0000 7f8c4c000640 1 -- start start 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c2c0039e0 0x7f8c44156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c44165440 0x7f8c440ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4c000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c441fb320 0x7f8c441fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4c000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4406e890 con 0x7f8c441fb320 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c4406ec00 con 0x7f8c44165440 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c4406dbb0 con 0x7f8c2c0039e0 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c441fb320 0x7f8c441fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c441fb320 0x7f8c441fb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:56306/0 (socket says 172.21.15.47:56306) 2024-04-20T10:28:03.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.581+0000 7f8c4a000640 1 -- 172.21.15.47:0/3449920446 learned_addr learned my addr 172.21.15.47:0/3449920446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c49600640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c2c0039e0 0x7f8c44156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c48c00640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c44165440 0x7f8c440ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c4a000640 1 -- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c2c0039e0 msgr2=0x7f8c44156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c4a000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c2c0039e0 0x7f8c44156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c4a000640 1 -- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c44165440 msgr2=0x7f8c440ae7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c4a000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c44165440 0x7f8c440ae7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c4a000640 1 -- 172.21.15.47:0/3449920446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c441fbd10 con 0x7f8c441fb320 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c48c00640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c44165440 0x7f8c440ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:03.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c4a000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c441fb320 0x7f8c441fb7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c004780 tx=0x7f8c3c00d900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c41600640 1 -- 172.21.15.47:0/3449920446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c0074e0 con 0x7f8c441fb320 2024-04-20T10:28:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c41600640 1 -- 172.21.15.47:0/3449920446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8c3c0047e0 con 0x7f8c441fb320 2024-04-20T10:28:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c41600640 1 -- 172.21.15.47:0/3449920446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c3c004bd0 con 0x7f8c441fb320 2024-04-20T10:28:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.582+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c441658a0 con 0x7f8c441fb320 2024-04-20T10:28:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.583+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c44157750 con 0x7f8c441fb320 2024-04-20T10:28:03.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.584+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c44068980 con 0x7f8c441fb320 2024-04-20T10:28:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.585+0000 7f8c41600640 1 -- 172.21.15.47:0/3449920446 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8c3c0133d0 con 0x7f8c441fb320 2024-04-20T10:28:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.585+0000 7f8c41600640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8c2406c8f0 0x7f8c2406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:03.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.586+0000 7f8c41600640 1 -- 172.21.15.47:0/3449920446 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f8c3c08ba60 con 0x7f8c441fb320 2024-04-20T10:28:03.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.586+0000 7f8c49600640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8c2406c8f0 0x7f8c2406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:03.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.587+0000 7f8c49600640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8c2406c8f0 0x7f8c2406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8c38037040 tx=0x7f8c3802e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:03.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.590+0000 7f8c41600640 1 -- 172.21.15.47:0/3449920446 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8c441fbd10 con 0x7f8c441fb320 2024-04-20T10:28:03.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.795+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8c440b0f30 con 0x7f8c441fb320 2024-04-20T10:28:03.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.796+0000 7f8c41600640 1 -- 172.21.15.47:0/3449920446 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f8c3c059b70 con 0x7f8c441fb320 2024-04-20T10:28:03.796 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:03.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8c2406c8f0 msgr2=0x7f8c2406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8c2406c8f0 0x7f8c2406edb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8c38037040 tx=0x7f8c3802e000 comp rx=0 tx=0).stop 2024-04-20T10:28:03.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c441fb320 msgr2=0x7f8c441fb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c441fb320 0x7f8c441fb7d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c004780 tx=0x7f8c3c00d900 comp rx=0 tx=0).stop 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 shutdown_connections 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8c2406c8f0 0x7f8c2406edb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8c441fb320 0x7f8c441fb7d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c44165440 0x7f8c440ae7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 --2- 172.21.15.47:0/3449920446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c2c0039e0 0x7f8c44156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 >> 172.21.15.47:0/3449920446 conn(0x7f8c440b3350 msgr2=0x7f8c440af1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 shutdown_connections 2024-04-20T10:28:03.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:03.799+0000 7f8c4c000640 1 -- 172.21.15.47:0/3449920446 wait complete. 2024-04-20T10:28:03.820 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.035+0000 7ff28b000640 1 Processor -- start 2024-04-20T10:28:04.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.035+0000 7ff28b000640 1 -- start start 2024-04-20T10:28:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.037+0000 7ff28b000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff28415a130 0x7ff284073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.037+0000 7ff28b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.037+0000 7ff28b000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff28415cc20 con 0x7ff284071a60 2024-04-20T10:28:04.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.037+0000 7ff28b000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff28415d7f0 con 0x7ff28415a130 2024-04-20T10:28:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.037+0000 7ff28b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff28415cec0 con 0x7ff28406f870 2024-04-20T10:28:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff289c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff28415a130 0x7ff284073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff289c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff28415a130 0x7ff284073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:55102/0 (socket says 172.21.15.47:55102) 2024-04-20T10:28:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff289c00640 1 -- 172.21.15.47:0/1870306679 learned_addr learned my addr 172.21.15.47:0/1870306679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:04.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff28a600640 1 --2- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff289c00640 1 -- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 msgr2=0x7ff2840739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff289c00640 1 --2- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff289c00640 1 -- 172.21.15.47:0/1870306679 >> v1:172.21.15.175:6789/0 conn(0x7ff284071a60 legacy=0x7ff28406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.038+0000 7ff289c00640 1 -- 172.21.15.47:0/1870306679 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28415d1a0 con 0x7ff28415a130 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.040+0000 7ff289c00640 1 --2- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff284073470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8149a6060f415fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.040+0000 7ff283e00640 1 -- 172.21.15.47:0/1870306679 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780317c0 con 0x7ff28415a130 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff283e00640 1 -- 172.21.15.47:0/1870306679 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff278037070 con 0x7ff28415a130 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff283e00640 1 -- 172.21.15.47:0/1870306679 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278031b80 con 0x7ff28415a130 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff28b000640 1 -- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 msgr2=0x7ff284073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff28b000640 1 --2- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff284073470 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ff278009870 tx=0x7ff27802b860 comp rx=0 tx=0).stop 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff28b000640 1 -- 172.21.15.47:0/1870306679 shutdown_connections 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff28b000640 1 --2- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff28b000640 1 --2- 172.21.15.47:0/1870306679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff284073470 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.041+0000 7ff28b000640 1 -- 172.21.15.47:0/1870306679 >> 172.21.15.47:0/1870306679 conn(0x7ff2840cffa0 msgr2=0x7ff2840d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.042+0000 7ff28b000640 1 -- 172.21.15.47:0/1870306679 shutdown_connections 2024-04-20T10:28:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.042+0000 7ff28b000640 1 -- 172.21.15.47:0/1870306679 wait complete. 2024-04-20T10:28:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.044+0000 7ff28b000640 1 Processor -- start 2024-04-20T10:28:04.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.044+0000 7ff28b000640 1 -- start start 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.045+0000 7ff28b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840cb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.045+0000 7ff28b000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284071a60 0x7ff28415b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff28b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff28415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff28b000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff28406ba80 con 0x7ff284071a60 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff28b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff28406b900 con 0x7ff28415a130 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff28b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff28406bc00 con 0x7ff28406f870 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff289200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284071a60 0x7ff28415b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff289200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284071a60 0x7ff28415b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57526/0 (socket says 172.21.15.47:57526) 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff289200640 1 -- 172.21.15.47:0/2744713766 learned_addr learned my addr 172.21.15.47:0/2744713766 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff28a600640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff28415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.046+0000 7ff289c00640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840cb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff289200640 1 -- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 msgr2=0x7ff2840cb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff289200640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840cb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff289200640 1 -- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 msgr2=0x7ff28415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff289200640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff28415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff289200640 1 -- 172.21.15.47:0/2744713766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2841f7f60 con 0x7ff284071a60 2024-04-20T10:28:04.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff28a600640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff28415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:04.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff289200640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284071a60 0x7ff28415b9a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff27400c7c0 tx=0x7ff27400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff281600640 1 -- 172.21.15.47:0/2744713766 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274016020 con 0x7ff284071a60 2024-04-20T10:28:04.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff281600640 1 -- 172.21.15.47:0/2744713766 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff274006d50 con 0x7ff284071a60 2024-04-20T10:28:04.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2840b5a00 con 0x7ff284071a60 2024-04-20T10:28:04.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.047+0000 7ff281600640 1 -- 172.21.15.47:0/2744713766 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274003bb0 con 0x7ff284071a60 2024-04-20T10:28:04.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.048+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2841f8500 con 0x7ff284071a60 2024-04-20T10:28:04.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.049+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff284075b30 con 0x7ff284071a60 2024-04-20T10:28:04.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.050+0000 7ff281600640 1 -- 172.21.15.47:0/2744713766 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff27400a850 con 0x7ff284071a60 2024-04-20T10:28:04.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.051+0000 7ff281600640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff26806c8f0 0x7ff26806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.051+0000 7ff281600640 1 -- 172.21.15.47:0/2744713766 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff27408bb00 con 0x7ff284071a60 2024-04-20T10:28:04.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.052+0000 7ff289c00640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff26806c8f0 0x7ff26806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.053+0000 7ff289c00640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff26806c8f0 0x7ff26806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff278036000 tx=0x7ff27803d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.055+0000 7ff281600640 1 -- 172.21.15.47:0/2744713766 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff274020020 con 0x7ff284071a60 2024-04-20T10:28:04.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.263+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff2841f8700 con 0x7ff284071a60 2024-04-20T10:28:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.266+0000 7ff281600640 1 -- 172.21.15.47:0/2744713766 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7ff2841f8700 con 0x7ff284071a60 2024-04-20T10:28:04.267 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff26806c8f0 msgr2=0x7ff26806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff26806c8f0 0x7ff26806edb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff278036000 tx=0x7ff27803d040 comp rx=0 tx=0).stop 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284071a60 msgr2=0x7ff28415b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284071a60 0x7ff28415b9a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff27400c7c0 tx=0x7ff27400cc80 comp rx=0 tx=0).stop 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 shutdown_connections 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff28415a130 0x7ff28415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff26806c8f0 0x7ff26806edb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff284071a60 0x7ff28415b9a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 --2- 172.21.15.47:0/2744713766 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28406f870 0x7ff2840cb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.269+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 >> 172.21.15.47:0/2744713766 conn(0x7ff2840cffa0 msgr2=0x7ff28406f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.270+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 shutdown_connections 2024-04-20T10:28:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.270+0000 7ff28b000640 1 -- 172.21.15.47:0/2744713766 wait complete. 2024-04-20T10:28:04.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.505+0000 7f8a0f800640 1 Processor -- start 2024-04-20T10:28:04.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.505+0000 7f8a0f800640 1 -- start start 2024-04-20T10:28:04.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.506+0000 7f8a0f800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8a0815e960 0x7f8a080d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.506+0000 7f8a0f800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a080b6830 0x7f8a080d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.506+0000 7f8a0f800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0815ed40 con 0x7f8a080d1c10 2024-04-20T10:28:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.506+0000 7f8a0f800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8a080b6c60 con 0x7f8a080b6830 2024-04-20T10:28:04.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.506+0000 7f8a0f800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8a080b6e90 con 0x7f8a0815e960 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.507+0000 7f8a07e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8a080d1c10 0x7f8a080b6120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60940/0 (socket says 172.21.15.47:60940) 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.507+0000 7f8a07e00640 1 -- 172.21.15.47:0/1076170035 learned_addr learned my addr 172.21.15.47:0/1076170035 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.507+0000 7f8a0d800640 1 --2- 172.21.15.47:0/1076170035 >> v2:172.21.15.47:3300/0 conn(0x7f8a080b6830 0x7f8a080d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.507+0000 7f8a0ce00640 1 --2- 172.21.15.47:0/1076170035 >> v2:172.21.15.175:3300/0 conn(0x7f8a0815e960 0x7f8a080d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.507+0000 7f8a07400640 1 -- 172.21.15.47:0/1076170035 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2591239944 0 0) 0x7f8a0815ed40 con 0x7f8a080d1c10 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.508+0000 7f8a07400640 1 -- 172.21.15.47:0/1076170035 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89ec003520 con 0x7f8a080d1c10 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.508+0000 7f8a0d800640 1 -- 172.21.15.47:0/1076170035 >> v2:172.21.15.175:3300/0 conn(0x7f8a0815e960 msgr2=0x7f8a080d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.508+0000 7f8a0d800640 1 --2- 172.21.15.47:0/1076170035 >> v2:172.21.15.175:3300/0 conn(0x7f8a0815e960 0x7f8a080d4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.508+0000 7f8a0d800640 1 -- 172.21.15.47:0/1076170035 >> v1:172.21.15.47:6789/0 conn(0x7f8a080d1c10 legacy=0x7f8a080b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.508+0000 7f8a0d800640 1 -- 172.21.15.47:0/1076170035 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a080d0e50 con 0x7f8a080b6830 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.509+0000 7f8a0d800640 1 --2- 172.21.15.47:0/1076170035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a080d4fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a08068070 tx=0x7f8a0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac123f3113caae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.509+0000 7f8a07400640 1 -- 172.21.15.47:0/1076170035 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00035020 con 0x7f8a080b6830 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.509+0000 7f8a07400640 1 -- 172.21.15.47:0/1076170035 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a00031830 con 0x7f8a080b6830 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.509+0000 7f8a07400640 1 -- 172.21.15.47:0/1076170035 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00031c00 con 0x7f8a080b6830 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 -- 172.21.15.47:0/1076170035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 msgr2=0x7f8a080d4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 --2- 172.21.15.47:0/1076170035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a080d4fe0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8a08068070 tx=0x7f8a0002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 -- 172.21.15.47:0/1076170035 shutdown_connections 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 --2- 172.21.15.47:0/1076170035 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a080d4fe0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 --2- 172.21.15.47:0/1076170035 >> v2:172.21.15.175:3300/0 conn(0x7f8a0815e960 0x7f8a080d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 -- 172.21.15.47:0/1076170035 >> 172.21.15.47:0/1076170035 conn(0x7f8a080b96f0 msgr2=0x7f8a080b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:04.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 -- 172.21.15.47:0/1076170035 shutdown_connections 2024-04-20T10:28:04.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.510+0000 7f8a0f800640 1 -- 172.21.15.47:0/1076170035 wait complete. 2024-04-20T10:28:04.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.512+0000 7f8a0f800640 1 Processor -- start 2024-04-20T10:28:04.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.512+0000 7f8a0f800640 1 -- start start 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0f800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a0815ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0f800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a080d1c10 0x7f8a0806df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0f800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815e960 0x7f8a08151890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0f800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8a080d16f0 con 0x7f8a080b6830 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0f800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a080d1410 con 0x7f8a080d1c10 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0f800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a080d3160 con 0x7f8a0815e960 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a0815ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a0815ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57540/0 (socket says 172.21.15.47:57540) 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a0ce00640 1 -- 172.21.15.47:0/3123172818 learned_addr learned my addr 172.21.15.47:0/3123172818 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.513+0000 7f8a07e00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a080d1c10 0x7f8a0806df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0ce00640 1 -- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815e960 msgr2=0x7f8a08151890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0d800640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815e960 0x7f8a08151890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0ce00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815e960 0x7f8a08151890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0ce00640 1 -- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a080d1c10 msgr2=0x7f8a0806df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0ce00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a080d1c10 0x7f8a0806df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0ce00640 1 -- 172.21.15.47:0/3123172818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0806e490 con 0x7f8a080b6830 2024-04-20T10:28:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a07e00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a080d1c10 0x7f8a0806df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:04.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0ce00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a0815ad80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f89f800efc0 tx=0x7f89f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a04c00640 1 -- 172.21.15.47:0/3123172818 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8018020 con 0x7f8a080b6830 2024-04-20T10:28:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a04c00640 1 -- 172.21.15.47:0/3123172818 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f89f8014480 con 0x7f8a080b6830 2024-04-20T10:28:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a04c00640 1 -- 172.21.15.47:0/3123172818 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89f8014850 con 0x7f8a080b6830 2024-04-20T10:28:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a08151dd0 con 0x7f8a080b6830 2024-04-20T10:28:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.514+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0806c720 con 0x7f8a080b6830 2024-04-20T10:28:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.516+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a08068980 con 0x7f8a080b6830 2024-04-20T10:28:04.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.517+0000 7f8a04c00640 1 -- 172.21.15.47:0/3123172818 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f89f8005d10 con 0x7f8a080b6830 2024-04-20T10:28:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.518+0000 7f8a04c00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89e806c8f0 0x7f89e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.518+0000 7f8a04c00640 1 -- 172.21.15.47:0/3123172818 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f89f808cc80 con 0x7f8a080b6830 2024-04-20T10:28:04.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.518+0000 7f8a07e00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89e806c8f0 0x7f89e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.520+0000 7f8a07e00640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89e806c8f0 0x7f89e806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a080cf600 tx=0x7f89f4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.522+0000 7f8a04c00640 1 -- 172.21.15.47:0/3123172818 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f89f8022030 con 0x7f8a080b6830 2024-04-20T10:28:04.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.728+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f8a080b39b0 con 0x7f8a080b6830 2024-04-20T10:28:04.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.729+0000 7f8a04c00640 1 -- 172.21.15.47:0/3123172818 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f89f8010040 con 0x7f8a080b6830 2024-04-20T10:28:04.729 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.731+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89e806c8f0 msgr2=0x7f89e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.731+0000 7f8a0f800640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89e806c8f0 0x7f89e806edb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8a080cf600 tx=0x7f89f4009040 comp rx=0 tx=0).stop 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.731+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 msgr2=0x7f8a0815ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.731+0000 7f8a0f800640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a0815ad80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f89f800efc0 tx=0x7f89f800c490 comp rx=0 tx=0).stop 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 shutdown_connections 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0815e960 0x7f8a08151890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a080d1c10 0x7f8a0806df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89e806c8f0 0x7f89e806edb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 --2- 172.21.15.47:0/3123172818 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a080b6830 0x7f8a0815ad80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 >> 172.21.15.47:0/3123172818 conn(0x7f8a080b96f0 msgr2=0x7f8a080cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:04.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 shutdown_connections 2024-04-20T10:28:04.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.732+0000 7f8a0f800640 1 -- 172.21.15.47:0/3123172818 wait complete. 2024-04-20T10:28:04.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:28:04.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.962+0000 7f3aa6600640 1 Processor -- start 2024-04-20T10:28:04.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.962+0000 7f3aa6600640 1 -- start start 2024-04-20T10:28:04.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa6600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3aa0159e10 0x7f3aa0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa6600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3aa015a1f0 0x7f3aa0071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa6600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3aa015cc90 con 0x7f3aa006da90 2024-04-20T10:28:04.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa6600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa0072900 con 0x7f3aa0159e10 2024-04-20T10:28:04.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa6600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3aa015ca40 con 0x7f3aa015a1f0 2024-04-20T10:28:04.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa5200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3aa0159e10 0x7f3aa0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa5200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3aa0159e10 0x7f3aa0071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57550/0 (socket says 172.21.15.47:57550) 2024-04-20T10:28:04.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.964+0000 7f3aa5200640 1 -- 172.21.15.47:0/566170623 learned_addr learned my addr 172.21.15.47:0/566170623 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:04.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.965+0000 7f3a9fe00640 1 --2- 172.21.15.47:0/566170623 >> v2:172.21.15.175:3300/0 conn(0x7f3aa015a1f0 0x7f3aa0071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.965+0000 7f3aa5200640 1 -- 172.21.15.47:0/566170623 >> v2:172.21.15.175:3300/0 conn(0x7f3aa015a1f0 msgr2=0x7f3aa0071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:04.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.965+0000 7f3aa5200640 1 --2- 172.21.15.47:0/566170623 >> v2:172.21.15.175:3300/0 conn(0x7f3aa015a1f0 0x7f3aa0071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.965+0000 7f3aa5200640 1 -- 172.21.15.47:0/566170623 >> v1:172.21.15.175:6789/0 conn(0x7f3aa006da90 legacy=0x7f3aa0160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:04.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.965+0000 7f3aa5200640 1 -- 172.21.15.47:0/566170623 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa015cf70 con 0x7f3aa0159e10 2024-04-20T10:28:04.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.966+0000 7f3aa5200640 1 --2- 172.21.15.47:0/566170623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa0159e10 0x7f3aa0071a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0050400 tx=0x7f3a9402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f674fad91a48b2b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3a9f400640 1 -- 172.21.15.47:0/566170623 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94034070 con 0x7f3aa0159e10 2024-04-20T10:28:04.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3a9f400640 1 -- 172.21.15.47:0/566170623 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94030860 con 0x7f3aa0159e10 2024-04-20T10:28:04.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3a9f400640 1 -- 172.21.15.47:0/566170623 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94030c30 con 0x7f3aa0159e10 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3aa6600640 1 -- 172.21.15.47:0/566170623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa0159e10 msgr2=0x7f3aa0071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3aa6600640 1 --2- 172.21.15.47:0/566170623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa0159e10 0x7f3aa0071a80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0050400 tx=0x7f3a9402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3aa6600640 1 -- 172.21.15.47:0/566170623 shutdown_connections 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3aa6600640 1 --2- 172.21.15.47:0/566170623 >> v2:172.21.15.175:3300/0 conn(0x7f3aa015a1f0 0x7f3aa0071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3aa6600640 1 --2- 172.21.15.47:0/566170623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa0159e10 0x7f3aa0071a80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3aa6600640 1 -- 172.21.15.47:0/566170623 >> 172.21.15.47:0/566170623 conn(0x7f3aa00d0040 msgr2=0x7f3aa00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.967+0000 7f3aa6600640 1 -- 172.21.15.47:0/566170623 shutdown_connections 2024-04-20T10:28:04.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.968+0000 7f3aa6600640 1 -- 172.21.15.47:0/566170623 wait complete. 2024-04-20T10:28:04.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.969+0000 7f3aa6600640 1 Processor -- start 2024-04-20T10:28:04.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.970+0000 7f3aa6600640 1 -- start start 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.970+0000 7f3aa6600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa006da90 0x7f3aa015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa0159e10 0x7f3aa014cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3aa015a1f0 0x7f3aa009aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa6600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa015d810 con 0x7f3aa006da90 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa015d530 con 0x7f3aa0159e10 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3aa006b900 con 0x7f3aa015a1f0 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa006da90 0x7f3aa015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa5200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa006da90 0x7f3aa015c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57556/0 (socket says 172.21.15.47:57556) 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa5200640 1 -- 172.21.15.47:0/560268678 learned_addr learned my addr 172.21.15.47:0/560268678 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3a9fe00640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa0159e10 0x7f3aa014cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa5200640 1 -- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3aa015a1f0 msgr2=0x7f3aa009aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa5c00640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3aa015a1f0 0x7f3aa009aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.971+0000 7f3aa5200640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3aa015a1f0 0x7f3aa009aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3aa5200640 1 -- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa0159e10 msgr2=0x7f3aa014cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3aa5200640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa0159e10 0x7f3aa014cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3aa5200640 1 -- 172.21.15.47:0/560268678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa015c6d0 con 0x7f3aa006da90 2024-04-20T10:28:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3aa5c00640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3aa015a1f0 0x7f3aa009aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3a9fe00640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa0159e10 0x7f3aa014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:04.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3aa5200640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa006da90 0x7f3aa015c040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3a9402b9e0 tx=0x7f3a94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3a9cc00640 1 -- 172.21.15.47:0/560268678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94048070 con 0x7f3aa006da90 2024-04-20T10:28:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3a9cc00640 1 -- 172.21.15.47:0/560268678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a94030660 con 0x7f3aa006da90 2024-04-20T10:28:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa009d240 con 0x7f3aa006da90 2024-04-20T10:28:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3a9cc00640 1 -- 172.21.15.47:0/560268678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a94034040 con 0x7f3aa006da90 2024-04-20T10:28:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.972+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa015eca0 con 0x7f3aa006da90 2024-04-20T10:28:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.973+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa0075b60 con 0x7f3aa006da90 2024-04-20T10:28:04.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.974+0000 7f3a9cc00640 1 -- 172.21.15.47:0/560268678 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a9404b070 con 0x7f3aa006da90 2024-04-20T10:28:04.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.975+0000 7f3a9cc00640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a8006c920 0x7f3a8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:04.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.975+0000 7f3a9cc00640 1 -- 172.21.15.47:0/560268678 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f3a940add60 con 0x7f3aa006da90 2024-04-20T10:28:04.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.976+0000 7f3a9fe00640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a8006c920 0x7f3a8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:04.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.977+0000 7f3a9fe00640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a8006c920 0x7f3a8006ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006fd0 tx=0x7f3a8c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:04.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:04.980+0000 7f3a9cc00640 1 -- 172.21.15.47:0/560268678 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3a9407bee0 con 0x7f3aa006da90 2024-04-20T10:28:05.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.232+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3aa006d1c0 con 0x7f3aa006da90 2024-04-20T10:28:05.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.233+0000 7f3a9cc00640 1 -- 172.21.15.47:0/560268678 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7f3a9407b7e0 con 0x7f3aa006da90 2024-04-20T10:28:05.234 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:05.234 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:28:05.234 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:28:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a8006c920 msgr2=0x7f3a8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a8006c920 0x7f3a8006ede0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3a8c006fd0 tx=0x7f3a8c008040 comp rx=0 tx=0).stop 2024-04-20T10:28:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa006da90 msgr2=0x7f3aa015c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa006da90 0x7f3aa015c040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3a9402b9e0 tx=0x7f3a94004730 comp rx=0 tx=0).stop 2024-04-20T10:28:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 shutdown_connections 2024-04-20T10:28:05.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3aa015a1f0 0x7f3aa009aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3aa0159e10 0x7f3aa014cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3a8006c920 0x7f3a8006ede0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 --2- 172.21.15.47:0/560268678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3aa006da90 0x7f3aa015c040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 >> 172.21.15.47:0/560268678 conn(0x7f3aa00d0040 msgr2=0x7f3aa006c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:05.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.236+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 shutdown_connections 2024-04-20T10:28:05.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.237+0000 7f3aa6600640 1 -- 172.21.15.47:0/560268678 wait complete. 2024-04-20T10:28:05.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:05.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.467+0000 7f5e98600640 1 Processor -- start 2024-04-20T10:28:05.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.468+0000 7f5e98600640 1 -- start start 2024-04-20T10:28:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.470+0000 7f5e98600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5e900cfa00 0x7f5e9015ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.470+0000 7f5e98600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e900ce4a0 con 0x7f5e900cfde0 2024-04-20T10:28:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.470+0000 7f5e98600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5e900ce780 con 0x7f5e90158780 2024-04-20T10:28:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.470+0000 7f5e98600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5e900d0680 con 0x7f5e900cfa00 2024-04-20T10:28:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.470+0000 7f5e95200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5e900cfde0 0x7f5e900cdcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60950/0 (socket says 172.21.15.47:60950) 2024-04-20T10:28:05.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.470+0000 7f5e95200640 1 -- 172.21.15.47:0/3134965276 learned_addr learned my addr 172.21.15.47:0/3134965276 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.470+0000 7f5e95c00640 1 --2- 172.21.15.47:0/3134965276 >> v2:172.21.15.47:3300/0 conn(0x7f5e900cfa00 0x7f5e9015ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.471+0000 7f5e8fe00640 1 -- 172.21.15.47:0/3134965276 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3272424338 0 0) 0x7f5e900ce4a0 con 0x7f5e900cfde0 2024-04-20T10:28:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.471+0000 7f5e8fe00640 1 -- 172.21.15.47:0/3134965276 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5e78003520 con 0x7f5e900cfde0 2024-04-20T10:28:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.471+0000 7f5e95c00640 1 -- 172.21.15.47:0/3134965276 >> v1:172.21.15.175:6789/0 conn(0x7f5e90158780 legacy=0x7f5e900b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.471+0000 7f5e95c00640 1 -- 172.21.15.47:0/3134965276 >> v1:172.21.15.47:6789/0 conn(0x7f5e900cfde0 legacy=0x7f5e900cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.471+0000 7f5e95c00640 1 -- 172.21.15.47:0/3134965276 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e900cea60 con 0x7f5e900cfa00 2024-04-20T10:28:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.472+0000 7f5e95c00640 1 --2- 172.21.15.47:0/3134965276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e900cfa00 0x7f5e9015ebe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e840097d0 tx=0x7f5e8402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c175d26c36bf6e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.472+0000 7f5e8fe00640 1 -- 172.21.15.47:0/3134965276 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84035020 con 0x7f5e900cfa00 2024-04-20T10:28:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.472+0000 7f5e8fe00640 1 -- 172.21.15.47:0/3134965276 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e84031810 con 0x7f5e900cfa00 2024-04-20T10:28:05.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.472+0000 7f5e8fe00640 1 -- 172.21.15.47:0/3134965276 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e84031be0 con 0x7f5e900cfa00 2024-04-20T10:28:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.473+0000 7f5e98600640 1 -- 172.21.15.47:0/3134965276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e900cfa00 msgr2=0x7f5e9015ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.473+0000 7f5e98600640 1 --2- 172.21.15.47:0/3134965276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e900cfa00 0x7f5e9015ebe0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5e840097d0 tx=0x7f5e8402b8b0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.473+0000 7f5e98600640 1 -- 172.21.15.47:0/3134965276 shutdown_connections 2024-04-20T10:28:05.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.473+0000 7f5e98600640 1 --2- 172.21.15.47:0/3134965276 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e900cfa00 0x7f5e9015ebe0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.473+0000 7f5e98600640 1 -- 172.21.15.47:0/3134965276 >> 172.21.15.47:0/3134965276 conn(0x7f5e90070450 msgr2=0x7f5e90070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.473+0000 7f5e98600640 1 -- 172.21.15.47:0/3134965276 shutdown_connections 2024-04-20T10:28:05.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.473+0000 7f5e98600640 1 -- 172.21.15.47:0/3134965276 wait complete. 2024-04-20T10:28:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.475+0000 7f5e98600640 1 Processor -- start 2024-04-20T10:28:05.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.475+0000 7f5e98600640 1 -- start start 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.476+0000 7f5e98600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e900cfa00 0x7f5e90141a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.476+0000 7f5e98600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e900cfde0 0x7f5e900b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.476+0000 7f5e98600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e90158780 0x7f5e900b0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.476+0000 7f5e98600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5e900b5880 con 0x7f5e90158780 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.476+0000 7f5e98600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e900b4350 con 0x7f5e900cfde0 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.476+0000 7f5e98600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5e900b5a00 con 0x7f5e900cfa00 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e90158780 0x7f5e900b0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e90158780 0x7f5e900b0f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57574/0 (socket says 172.21.15.47:57574) 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 -- 172.21.15.47:0/4146813742 learned_addr learned my addr 172.21.15.47:0/4146813742 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:05.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e95200640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e900cfde0 0x7f5e900b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e95c00640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e900cfa00 0x7f5e90141a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 -- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e900cfa00 msgr2=0x7f5e90141a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e900cfa00 0x7f5e90141a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 -- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e900cfde0 msgr2=0x7f5e900b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e900cfde0 0x7f5e900b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.477+0000 7f5e96600640 1 -- 172.21.15.47:0/4146813742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e901425b0 con 0x7f5e90158780 2024-04-20T10:28:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.478+0000 7f5e96600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e90158780 0x7f5e900b0f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800c7a0 tx=0x7f5e8800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.478+0000 7f5e8d600640 1 -- 172.21.15.47:0/4146813742 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e88003b70 con 0x7f5e90158780 2024-04-20T10:28:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.478+0000 7f5e8d600640 1 -- 172.21.15.47:0/4146813742 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5e88003d30 con 0x7f5e90158780 2024-04-20T10:28:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.478+0000 7f5e8d600640 1 -- 172.21.15.47:0/4146813742 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e8800aa90 con 0x7f5e90158780 2024-04-20T10:28:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.478+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e90142890 con 0x7f5e90158780 2024-04-20T10:28:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.478+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e90142e50 con 0x7f5e90158780 2024-04-20T10:28:05.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.479+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e90068980 con 0x7f5e90158780 2024-04-20T10:28:05.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.480+0000 7f5e8d600640 1 -- 172.21.15.47:0/4146813742 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5e8800ac30 con 0x7f5e90158780 2024-04-20T10:28:05.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.481+0000 7f5e8d600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5e7006c8f0 0x7f5e7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.481+0000 7f5e8d600640 1 -- 172.21.15.47:0/4146813742 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f5e8808bed0 con 0x7f5e90158780 2024-04-20T10:28:05.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.482+0000 7f5e95c00640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5e7006c8f0 0x7f5e7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.483+0000 7f5e95c00640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5e7006c8f0 0x7f5e7006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e84036000 tx=0x7f5e8403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:05.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.486+0000 7f5e8d600640 1 -- 172.21.15.47:0/4146813742 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5e88059f20 con 0x7f5e90158780 2024-04-20T10:28:05.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.692+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5e900b65a0 con 0x7f5e90158780 2024-04-20T10:28:05.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.695+0000 7f5e8d600640 1 -- 172.21.15.47:0/4146813742 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7f5e88059820 con 0x7f5e90158780 2024-04-20T10:28:05.695 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:05.695 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5e7006c8f0 msgr2=0x7f5e7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5e7006c8f0 0x7f5e7006edb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5e84036000 tx=0x7f5e8403c040 comp rx=0 tx=0).stop 2024-04-20T10:28:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e90158780 msgr2=0x7f5e900b0f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e90158780 0x7f5e900b0f20 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5e8800c7a0 tx=0x7f5e8800cc60 comp rx=0 tx=0).stop 2024-04-20T10:28:05.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 shutdown_connections 2024-04-20T10:28:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5e7006c8f0 0x7f5e7006edb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5e90158780 0x7f5e900b0f20 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e900cfde0 0x7f5e900b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 --2- 172.21.15.47:0/4146813742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e900cfa00 0x7f5e90141a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 >> 172.21.15.47:0/4146813742 conn(0x7f5e90070450 msgr2=0x7f5e9006cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 shutdown_connections 2024-04-20T10:28:05.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.698+0000 7f5e98600640 1 -- 172.21.15.47:0/4146813742 wait complete. 2024-04-20T10:28:05.718 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:05.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:05.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.933+0000 7f1b81600640 1 Processor -- start 2024-04-20T10:28:05.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.933+0000 7f1b81600640 1 -- start start 2024-04-20T10:28:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.935+0000 7f1b81600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b7c071b50 0x7f1b7c15abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.935+0000 7f1b81600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c072a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.935+0000 7f1b81600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b7c0ae9b0 con 0x7f1b7c0af800 2024-04-20T10:28:05.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.935+0000 7f1b81600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1b7c0d0a70 con 0x7f1b7c071b50 2024-04-20T10:28:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.935+0000 7f1b81600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c06eb00 con 0x7f1b7c0725b0 2024-04-20T10:28:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b7c071b50 0x7f1b7c15abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1b7c071b50 0x7f1b7c15abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57580/0 (socket says 172.21.15.47:57580) 2024-04-20T10:28:05.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7aa00640 1 -- 172.21.15.47:0/434944699 learned_addr learned my addr 172.21.15.47:0/434944699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:05.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7b400640 1 --2- 172.21.15.47:0/434944699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c072a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7aa00640 1 -- 172.21.15.47:0/434944699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 msgr2=0x7f1b7c072a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:05.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7aa00640 1 --2- 172.21.15.47:0/434944699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c072a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7aa00640 1 -- 172.21.15.47:0/434944699 >> v1:172.21.15.175:6789/0 conn(0x7f1b7c0af800 legacy=0x7f1b7c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.936+0000 7f1b7aa00640 1 -- 172.21.15.47:0/434944699 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c06e840 con 0x7f1b7c071b50 2024-04-20T10:28:05.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.938+0000 7f1b7aa00640 1 --2- 172.21.15.47:0/434944699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c15abb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009870 tx=0x7f1b6c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3e35405f3dc18b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:05.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.938+0000 7f1b7a000640 1 -- 172.21.15.47:0/434944699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c034070 con 0x7f1b7c071b50 2024-04-20T10:28:05.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.938+0000 7f1b7a000640 1 -- 172.21.15.47:0/434944699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c030840 con 0x7f1b7c071b50 2024-04-20T10:28:05.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.938+0000 7f1b7a000640 1 -- 172.21.15.47:0/434944699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c030c10 con 0x7f1b7c071b50 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 -- 172.21.15.47:0/434944699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 msgr2=0x7f1b7c15abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 --2- 172.21.15.47:0/434944699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c15abb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009870 tx=0x7f1b6c02b980 comp rx=0 tx=0).stop 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 -- 172.21.15.47:0/434944699 shutdown_connections 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 --2- 172.21.15.47:0/434944699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c072a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 --2- 172.21.15.47:0/434944699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c15abb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 -- 172.21.15.47:0/434944699 >> 172.21.15.47:0/434944699 conn(0x7f1b7c0b3350 msgr2=0x7f1b7c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 -- 172.21.15.47:0/434944699 shutdown_connections 2024-04-20T10:28:05.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.939+0000 7f1b81600640 1 -- 172.21.15.47:0/434944699 wait complete. 2024-04-20T10:28:05.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.941+0000 7f1b81600640 1 Processor -- start 2024-04-20T10:28:05.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.941+0000 7f1b81600640 1 -- start start 2024-04-20T10:28:05.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b81600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b81600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b81600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b7c0af800 0x7f1b7c155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b81600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c15bf60 con 0x7f1b7c071b50 2024-04-20T10:28:05.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b81600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c072f40 con 0x7f1b7c0af800 2024-04-20T10:28:05.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b81600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b7c15b0f0 con 0x7f1b7c0725b0 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b7aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b7aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c14bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57582/0 (socket says 172.21.15.47:57582) 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.942+0000 7f1b7aa00640 1 -- 172.21.15.47:0/2487002489 learned_addr learned my addr 172.21.15.47:0/2487002489 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b7b400640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b7c0af800 0x7f1b7c155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b72000640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b7aa00640 1 -- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 msgr2=0x7f1b7c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b7aa00640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c157070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b7aa00640 1 -- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b7c0af800 msgr2=0x7f1b7c155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:05.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b7aa00640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b7c0af800 0x7f1b7c155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b7aa00640 1 -- 172.21.15.47:0/2487002489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c0ce910 con 0x7f1b7c071b50 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b72000640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.943+0000 7f1b7aa00640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c14bbe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c02b9c0 tx=0x7f1b6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.944+0000 7f1b73400640 1 -- 172.21.15.47:0/2487002489 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c048070 con 0x7f1b7c071b50 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.944+0000 7f1b73400640 1 -- 172.21.15.47:0/2487002489 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b6c030640 con 0x7f1b7c071b50 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.944+0000 7f1b73400640 1 -- 172.21.15.47:0/2487002489 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c034040 con 0x7f1b7c071b50 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.944+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c0cebf0 con 0x7f1b7c071b50 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.944+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c14c120 con 0x7f1b7c071b50 2024-04-20T10:28:05.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.945+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b7c068980 con 0x7f1b7c071b50 2024-04-20T10:28:05.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.946+0000 7f1b73400640 1 -- 172.21.15.47:0/2487002489 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b6c037070 con 0x7f1b7c071b50 2024-04-20T10:28:05.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.947+0000 7f1b73400640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b5806c920 0x7f1b5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:05.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.947+0000 7f1b73400640 1 -- 172.21.15.47:0/2487002489 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1b6c0ada90 con 0x7f1b7c071b50 2024-04-20T10:28:05.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.948+0000 7f1b72000640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b5806c920 0x7f1b5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:05.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.949+0000 7f1b72000640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b5806c920 0x7f1b5806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c06c790 tx=0x7f1b64009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:05.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:05.952+0000 7f1b73400640 1 -- 172.21.15.47:0/2487002489 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b6c030810 con 0x7f1b7c071b50 2024-04-20T10:28:06.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.159+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1b7c0cfe90 con 0x7f1b7c071b50 2024-04-20T10:28:06.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.160+0000 7f1b73400640 1 -- 172.21.15.47:0/2487002489 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f1b6c07b640 con 0x7f1b7c071b50 2024-04-20T10:28:06.160 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b5806c920 msgr2=0x7f1b5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b5806c920 0x7f1b5806ede0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c06c790 tx=0x7f1b64009040 comp rx=0 tx=0).stop 2024-04-20T10:28:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 msgr2=0x7f1b7c14bbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c14bbe0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c02b9c0 tx=0x7f1b6c004730 comp rx=0 tx=0).stop 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 shutdown_connections 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b7c0af800 0x7f1b7c155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b7c0725b0 0x7f1b7c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b5806c920 0x7f1b5806ede0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 --2- 172.21.15.47:0/2487002489 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b7c071b50 0x7f1b7c14bbe0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 >> 172.21.15.47:0/2487002489 conn(0x7f1b7c0b3350 msgr2=0x7f1b7c15e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 shutdown_connections 2024-04-20T10:28:06.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.163+0000 7f1b81600640 1 -- 172.21.15.47:0/2487002489 wait complete. 2024-04-20T10:28:06.183 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:06.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.398+0000 7fc436600640 1 Processor -- start 2024-04-20T10:28:06.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.398+0000 7fc436600640 1 -- start start 2024-04-20T10:28:06.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.399+0000 7fc436600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc430165190 0x7fc430167580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.399+0000 7fc436600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43006b900 con 0x7fc4300aee60 2024-04-20T10:28:06.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.399+0000 7fc436600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc43006bbe0 con 0x7fc43015d4f0 2024-04-20T10:28:06.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.399+0000 7fc436600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc43006c510 con 0x7fc430165190 2024-04-20T10:28:06.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.400+0000 7fc435200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc4300aee60 0x7fc4300af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60954/0 (socket says 172.21.15.47:60954) 2024-04-20T10:28:06.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.400+0000 7fc435200640 1 -- 172.21.15.47:0/3904630062 learned_addr learned my addr 172.21.15.47:0/3904630062 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.400+0000 7fc435c00640 1 --2- 172.21.15.47:0/3904630062 >> v2:172.21.15.175:3300/0 conn(0x7fc430165190 0x7fc430167580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.400+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3346852140 0 0) 0x7fc43006b900 con 0x7fc4300aee60 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.400+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc418003520 con 0x7fc4300aee60 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2161837790 0 0) 0x7fc418003520 con 0x7fc4300aee60 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 >> v2:172.21.15.175:3300/0 conn(0x7fc430165190 msgr2=0x7fc430167580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 --2- 172.21.15.47:0/3904630062 >> v2:172.21.15.175:3300/0 conn(0x7fc430165190 0x7fc430167580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 >> v1:172.21.15.175:6789/0 conn(0x7fc43015d4f0 legacy=0x7fc4300ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc43006bec0 con 0x7fc4300aee60 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fc424001670 con 0x7fc4300aee60 2024-04-20T10:28:06.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 >> v1:172.21.15.47:6789/0 conn(0x7fc4300aee60 legacy=0x7fc4300af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc418003a00 0x7fc418003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc418004800 0x7fc418004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4180051c0 0x7fc41800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc43006c510 con 0x7fc418004800 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc41800c0c0 con 0x7fc4180051c0 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc41800c2c0 con 0x7fc418003a00 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.401+0000 7fc42fe00640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc418004800 0x7fc418004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc435c00640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4180051c0 0x7fc41800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc435200640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc418003a00 0x7fc418003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc42fe00640 1 -- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc418003a00 msgr2=0x7fc418003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc42fe00640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc418003a00 0x7fc418003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc42fe00640 1 -- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4180051c0 msgr2=0x7fc41800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc42fe00640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4180051c0 0x7fc41800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc42fe00640 1 -- 172.21.15.47:0/3904630062 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc418003520 con 0x7fc418004800 2024-04-20T10:28:06.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.402+0000 7fc435c00640 1 -- 172.21.15.47:0/3904630062 reap_dead start 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.403+0000 7fc42fe00640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc418004800 0x7fc418004c80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc41c001750 tx=0x7fc418005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f9d29144f8b366 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.403+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c001370 con 0x7fc418004800 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.403+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41c0052d0 con 0x7fc418004800 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.403+0000 7fc42f400640 1 -- 172.21.15.47:0/3904630062 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc41c003830 con 0x7fc418004800 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.404+0000 7fc436600640 1 -- 172.21.15.47:0/3904630062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc418004800 msgr2=0x7fc418004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.404+0000 7fc436600640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc418004800 0x7fc418004c80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc41c001750 tx=0x7fc418005830 comp rx=0 tx=0).stop 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.404+0000 7fc436600640 1 -- 172.21.15.47:0/3904630062 shutdown_connections 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.404+0000 7fc436600640 1 --2- 172.21.15.47:0/3904630062 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc418004800 0x7fc418004c80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.404+0000 7fc436600640 1 -- 172.21.15.47:0/3904630062 >> 172.21.15.47:0/3904630062 conn(0x7fc4300b5270 msgr2=0x7fc4300b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.404+0000 7fc436600640 1 -- 172.21.15.47:0/3904630062 shutdown_connections 2024-04-20T10:28:06.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.405+0000 7fc436600640 1 -- 172.21.15.47:0/3904630062 wait complete. 2024-04-20T10:28:06.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.407+0000 7fc436600640 1 Processor -- start 2024-04-20T10:28:06.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.407+0000 7fc436600640 1 -- start start 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc436600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc418004800 0x7fc430099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc436600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc430150470 0x7fc4301508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc436600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4301555e0 0x7fc430155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc436600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc4300af240 con 0x7fc4301555e0 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc436600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc43006c9a0 con 0x7fc418004800 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc436600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc4300af3c0 con 0x7fc430150470 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc435c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4301555e0 0x7fc430155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc435c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4301555e0 0x7fc430155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57596/0 (socket says 172.21.15.47:57596) 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.409+0000 7fc435c00640 1 -- 172.21.15.47:0/3184961612 learned_addr learned my addr 172.21.15.47:0/3184961612 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:06.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc435200640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc418004800 0x7fc430099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc42fe00640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc430150470 0x7fc4301508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc435c00640 1 -- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc430150470 msgr2=0x7fc4301508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc435c00640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc430150470 0x7fc4301508f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc435c00640 1 -- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc418004800 msgr2=0x7fc430099a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc435c00640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc418004800 0x7fc430099a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc435c00640 1 -- 172.21.15.47:0/3184961612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc430150fe0 con 0x7fc4301555e0 2024-04-20T10:28:06.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc435c00640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4301555e0 0x7fc430155a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc43006d370 tx=0x7fc4280115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc42cc00640 1 -- 172.21.15.47:0/3184961612 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428011de0 con 0x7fc4301555e0 2024-04-20T10:28:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc42cc00640 1 -- 172.21.15.47:0/3184961612 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc42801b070 con 0x7fc4301555e0 2024-04-20T10:28:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.410+0000 7fc42cc00640 1 -- 172.21.15.47:0/3184961612 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428003990 con 0x7fc4301555e0 2024-04-20T10:28:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.411+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4301512c0 con 0x7fc4301555e0 2024-04-20T10:28:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.411+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc430151880 con 0x7fc4301555e0 2024-04-20T10:28:06.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.412+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4300ae380 con 0x7fc4301555e0 2024-04-20T10:28:06.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.413+0000 7fc42cc00640 1 -- 172.21.15.47:0/3184961612 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc428003b30 con 0x7fc4301555e0 2024-04-20T10:28:06.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.414+0000 7fc42cc00640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc41006c920 0x7fc41006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.415+0000 7fc42cc00640 1 -- 172.21.15.47:0/3184961612 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fc42808c7a0 con 0x7fc4301555e0 2024-04-20T10:28:06.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.415+0000 7fc435200640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc41006c920 0x7fc41006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.416+0000 7fc435200640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc41006c920 0x7fc41006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc424001050 tx=0x7fc424009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:06.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.419+0000 7fc42cc00640 1 -- 172.21.15.47:0/3184961612 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc4300ae380 con 0x7fc4301555e0 2024-04-20T10:28:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.620+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc430075b30 con 0x7fc4301555e0 2024-04-20T10:28:06.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.620+0000 7fc42cc00640 1 -- 172.21.15.47:0/3184961612 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7fc428003e20 con 0x7fc4301555e0 2024-04-20T10:28:06.621 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc41006c920 msgr2=0x7fc41006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc41006c920 0x7fc41006ede0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc424001050 tx=0x7fc424009040 comp rx=0 tx=0).stop 2024-04-20T10:28:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4301555e0 msgr2=0x7fc430155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4301555e0 0x7fc430155a60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc43006d370 tx=0x7fc4280115a0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 shutdown_connections 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc41006c920 0x7fc41006ede0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc4301555e0 0x7fc430155a60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc430150470 0x7fc4301508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 --2- 172.21.15.47:0/3184961612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc418004800 0x7fc430099a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 >> 172.21.15.47:0/3184961612 conn(0x7fc4300b5270 msgr2=0x7fc43006e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 shutdown_connections 2024-04-20T10:28:06.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.623+0000 7fc436600640 1 -- 172.21.15.47:0/3184961612 wait complete. 2024-04-20T10:28:06.643 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:06.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.860+0000 7f9e57600640 1 Processor -- start 2024-04-20T10:28:06.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.860+0000 7f9e57600640 1 -- start start 2024-04-20T10:28:06.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.861+0000 7f9e57600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e5015ca40 0x7f9e500ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.861+0000 7f9e57600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e50141c60 con 0x7f9e50156c20 2024-04-20T10:28:06.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.861+0000 7f9e57600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9e5015cf90 con 0x7f9e500ad8e0 2024-04-20T10:28:06.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.861+0000 7f9e57600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9e5006c010 con 0x7f9e5015ca40 2024-04-20T10:28:06.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.862+0000 7f9e4fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9e5015ca40 0x7f9e500ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.862+0000 7f9e54c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9e50156c20 0x7f9e5006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60956/0 (socket says 172.21.15.47:60956) 2024-04-20T10:28:06.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.862+0000 7f9e54c00640 1 -- 172.21.15.47:0/518613378 learned_addr learned my addr 172.21.15.47:0/518613378 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:06.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.862+0000 7f9e4fe00640 1 -- 172.21.15.47:0/518613378 >> v1:172.21.15.175:6789/0 conn(0x7f9e500ad8e0 legacy=0x7f9e500d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:06.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.862+0000 7f9e4fe00640 1 -- 172.21.15.47:0/518613378 >> v1:172.21.15.47:6789/0 conn(0x7f9e50156c20 legacy=0x7f9e5006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.862+0000 7f9e4fe00640 1 -- 172.21.15.47:0/518613378 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5015d270 con 0x7f9e5015ca40 2024-04-20T10:28:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.864+0000 7f9e4fe00640 1 --2- 172.21.15.47:0/518613378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e500ad3a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e50141e30 tx=0x7f9e3802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd5c70a18a75170 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.864+0000 7f9e4f400640 1 -- 172.21.15.47:0/518613378 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38035020 con 0x7f9e5015ca40 2024-04-20T10:28:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.864+0000 7f9e4f400640 1 -- 172.21.15.47:0/518613378 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e38031830 con 0x7f9e5015ca40 2024-04-20T10:28:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.864+0000 7f9e4f400640 1 -- 172.21.15.47:0/518613378 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38031c00 con 0x7f9e5015ca40 2024-04-20T10:28:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.865+0000 7f9e57600640 1 -- 172.21.15.47:0/518613378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 msgr2=0x7f9e500ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.865+0000 7f9e57600640 1 --2- 172.21.15.47:0/518613378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e500ad3a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9e50141e30 tx=0x7f9e3802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.865+0000 7f9e57600640 1 -- 172.21.15.47:0/518613378 shutdown_connections 2024-04-20T10:28:06.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.865+0000 7f9e57600640 1 --2- 172.21.15.47:0/518613378 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e500ad3a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.865+0000 7f9e57600640 1 -- 172.21.15.47:0/518613378 >> 172.21.15.47:0/518613378 conn(0x7f9e500b3350 msgr2=0x7f9e500b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:06.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.865+0000 7f9e57600640 1 -- 172.21.15.47:0/518613378 shutdown_connections 2024-04-20T10:28:06.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.865+0000 7f9e57600640 1 -- 172.21.15.47:0/518613378 wait complete. 2024-04-20T10:28:06.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.867+0000 7f9e57600640 1 Processor -- start 2024-04-20T10:28:06.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.867+0000 7f9e57600640 1 -- start start 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e57600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e500ad8e0 0x7f9e500cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e57600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e50156c20 0x7f9e500ce120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e57600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e5015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e57600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9e500d0a70 con 0x7f9e5015ca40 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e57600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e5015d830 con 0x7f9e500ad8e0 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e57600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e500d0bf0 con 0x7f9e50156c20 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e55600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e5015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e55600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e5015b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57610/0 (socket says 172.21.15.47:57610) 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e55600640 1 -- 172.21.15.47:0/3769854661 learned_addr learned my addr 172.21.15.47:0/3769854661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e54c00640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e500ad8e0 0x7f9e500cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.869+0000 7f9e4fe00640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e50156c20 0x7f9e500ce120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e55600640 1 -- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e50156c20 msgr2=0x7f9e500ce120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e55600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e50156c20 0x7f9e500ce120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e55600640 1 -- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e500ad8e0 msgr2=0x7f9e500cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:06.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e55600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e500ad8e0 0x7f9e500cdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:06.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e55600640 1 -- 172.21.15.47:0/3769854661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e5006e740 con 0x7f9e5015ca40 2024-04-20T10:28:06.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e4fe00640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e50156c20 0x7f9e500ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:06.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e55600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e5015b410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800b410 tx=0x7f9e4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:06.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e4cc00640 1 -- 172.21.15.47:0/3769854661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48015020 con 0x7f9e5015ca40 2024-04-20T10:28:06.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e4cc00640 1 -- 172.21.15.47:0/3769854661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9e48011830 con 0x7f9e5015ca40 2024-04-20T10:28:06.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e4cc00640 1 -- 172.21.15.47:0/3769854661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e48011c60 con 0x7f9e5015ca40 2024-04-20T10:28:06.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e5006ea20 con 0x7f9e5015ca40 2024-04-20T10:28:06.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.870+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e5006efe0 con 0x7f9e5015ca40 2024-04-20T10:28:06.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.871+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e50068980 con 0x7f9e5015ca40 2024-04-20T10:28:06.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.872+0000 7f9e4cc00640 1 -- 172.21.15.47:0/3769854661 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9e48018020 con 0x7f9e5015ca40 2024-04-20T10:28:06.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.873+0000 7f9e4cc00640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9e2c06c8f0 0x7f9e2c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:06.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.873+0000 7f9e4cc00640 1 -- 172.21.15.47:0/3769854661 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f9e4808ba30 con 0x7f9e5015ca40 2024-04-20T10:28:06.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.874+0000 7f9e54c00640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9e2c06c8f0 0x7f9e2c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:06.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.875+0000 7f9e54c00640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9e2c06c8f0 0x7f9e2c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e400012e0 tx=0x7f9e40009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:06.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:06.878+0000 7f9e4cc00640 1 -- 172.21.15.47:0/3769854661 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9e4801f030 con 0x7f9e5015ca40 2024-04-20T10:28:07.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.092+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9e500cfd90 con 0x7f9e5015ca40 2024-04-20T10:28:07.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.093+0000 7f9e4cc00640 1 -- 172.21.15.47:0/3769854661 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f9e480172c0 con 0x7f9e5015ca40 2024-04-20T10:28:07.093 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:07.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9e2c06c8f0 msgr2=0x7f9e2c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9e2c06c8f0 0x7f9e2c06edb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9e400012e0 tx=0x7f9e40009040 comp rx=0 tx=0).stop 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 msgr2=0x7f9e5015b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e5015b410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800b410 tx=0x7f9e4800b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 shutdown_connections 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9e2c06c8f0 0x7f9e2c06edb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9e5015ca40 0x7f9e5015b410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e50156c20 0x7f9e500ce120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 --2- 172.21.15.47:0/3769854661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e500ad8e0 0x7f9e500cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.095+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 >> 172.21.15.47:0/3769854661 conn(0x7f9e500b3350 msgr2=0x7f9e500afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:07.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.096+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 shutdown_connections 2024-04-20T10:28:07.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.096+0000 7f9e57600640 1 -- 172.21.15.47:0/3769854661 wait complete. 2024-04-20T10:28:07.117 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:07.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.337+0000 7ff1f5c00640 1 Processor -- start 2024-04-20T10:28:07.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.337+0000 7ff1f5c00640 1 -- start start 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1f5c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1f015ca40 0x7ff1f00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1f5c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff1f00ad8e0 0x7ff1f00d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1f5c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff1f0141c60 con 0x7ff1f0156c20 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1f5c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f006c010 con 0x7ff1f015ca40 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1f5c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff1f015cf90 con 0x7ff1f00ad8e0 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1ee000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1f015ca40 0x7ff1f00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1ee000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff1f015ca40 0x7ff1f00ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57616/0 (socket says 172.21.15.47:57616) 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1ee000640 1 -- 172.21.15.47:0/3691691225 learned_addr learned my addr 172.21.15.47:0/3691691225 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.339+0000 7ff1eea00640 1 --1- 172.21.15.47:0/3691691225 >> v1:172.21.15.47:6789/0 conn(0x7ff1f0156c20 0x7ff1f006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60966/0 (socket says 172.21.15.47:60966) 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.340+0000 7ff1ef400640 1 --2- 172.21.15.47:0/3691691225 >> v2:172.21.15.175:3300/0 conn(0x7ff1f00ad8e0 0x7ff1f00d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.340+0000 7ff1ee000640 1 -- 172.21.15.47:0/3691691225 >> v2:172.21.15.175:3300/0 conn(0x7ff1f00ad8e0 msgr2=0x7ff1f00d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:07.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.340+0000 7ff1ee000640 1 --2- 172.21.15.47:0/3691691225 >> v2:172.21.15.175:3300/0 conn(0x7ff1f00ad8e0 0x7ff1f00d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.340+0000 7ff1ee000640 1 -- 172.21.15.47:0/3691691225 >> v1:172.21.15.47:6789/0 conn(0x7ff1f0156c20 legacy=0x7ff1f006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.340+0000 7ff1ee000640 1 -- 172.21.15.47:0/3691691225 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f015d270 con 0x7ff1f015ca40 2024-04-20T10:28:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.340+0000 7ff1ed600640 1 -- 172.21.15.47:0/3691691225 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2022823384 0 0) 0x7ff1f0141c60 con 0x7ff1f0156c20 2024-04-20T10:28:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.341+0000 7ff1ee000640 1 --2- 172.21.15.47:0/3691691225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f015ca40 0x7ff1f00ad3a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff1f006ff40 tx=0x7ff1dc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc7b97643a3605ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:07.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.341+0000 7ff1ed600640 1 -- 172.21.15.47:0/3691691225 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc034070 con 0x7ff1f015ca40 2024-04-20T10:28:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1ed600640 1 -- 172.21.15.47:0/3691691225 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc030860 con 0x7ff1f015ca40 2024-04-20T10:28:07.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1ed600640 1 -- 172.21.15.47:0/3691691225 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc030c30 con 0x7ff1f015ca40 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 -- 172.21.15.47:0/3691691225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f015ca40 msgr2=0x7ff1f00ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/3691691225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f015ca40 0x7ff1f00ad3a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff1f006ff40 tx=0x7ff1dc02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 -- 172.21.15.47:0/3691691225 shutdown_connections 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/3691691225 >> v2:172.21.15.175:3300/0 conn(0x7ff1f00ad8e0 0x7ff1f00d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/3691691225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f015ca40 0x7ff1f00ad3a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 -- 172.21.15.47:0/3691691225 >> 172.21.15.47:0/3691691225 conn(0x7ff1f00b3350 msgr2=0x7ff1f00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 -- 172.21.15.47:0/3691691225 shutdown_connections 2024-04-20T10:28:07.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.342+0000 7ff1f5c00640 1 -- 172.21.15.47:0/3691691225 wait complete. 2024-04-20T10:28:07.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.344+0000 7ff1f5c00640 1 Processor -- start 2024-04-20T10:28:07.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.344+0000 7ff1f5c00640 1 -- start start 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.345+0000 7ff1f5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f00ad8e0 0x7ff1f00cf5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.345+0000 7ff1f5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f0156c20 0x7ff1f00999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.345+0000 7ff1f5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1f015ca40 0x7ff1f0152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.345+0000 7ff1f5c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f00d08a0 con 0x7ff1f0156c20 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.345+0000 7ff1f5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1f015d830 con 0x7ff1f00ad8e0 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.345+0000 7ff1f5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1f00d0a20 con 0x7ff1f015ca40 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f0156c20 0x7ff1f00999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f0156c20 0x7ff1f00999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57628/0 (socket says 172.21.15.47:57628) 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 -- 172.21.15.47:0/2001943087 learned_addr learned my addr 172.21.15.47:0/2001943087 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:07.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1eea00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f00ad8e0 0x7ff1f00cf5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 -- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1f015ca40 msgr2=0x7ff1f0152fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1f015ca40 0x7ff1f0152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 -- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f00ad8e0 msgr2=0x7ff1f00cf5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f00ad8e0 0x7ff1f00cf5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 -- 172.21.15.47:0/2001943087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1f0153800 con 0x7ff1f0156c20 2024-04-20T10:28:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.346+0000 7ff1ee000640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f0156c20 0x7ff1f00999d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0090b0 tx=0x7ff1dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.347+0000 7ff1e6a00640 1 -- 172.21.15.47:0/2001943087 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc034040 con 0x7ff1f0156c20 2024-04-20T10:28:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.347+0000 7ff1e6a00640 1 -- 172.21.15.47:0/2001943087 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1dc03ad20 con 0x7ff1f0156c20 2024-04-20T10:28:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.347+0000 7ff1e6a00640 1 -- 172.21.15.47:0/2001943087 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1dc044610 con 0x7ff1f0156c20 2024-04-20T10:28:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.347+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1f0153ae0 con 0x7ff1f0156c20 2024-04-20T10:28:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.347+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1f01540a0 con 0x7ff1f0156c20 2024-04-20T10:28:07.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.348+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1f0068980 con 0x7ff1f0156c20 2024-04-20T10:28:07.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.349+0000 7ff1e6a00640 1 -- 172.21.15.47:0/2001943087 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff1dc037070 con 0x7ff1f0156c20 2024-04-20T10:28:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.350+0000 7ff1e6a00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1d006c8f0 0x7ff1d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.350+0000 7ff1e6a00640 1 -- 172.21.15.47:0/2001943087 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7ff1dc0ae160 con 0x7ff1f0156c20 2024-04-20T10:28:07.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.351+0000 7ff1eea00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1d006c8f0 0x7ff1d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.352+0000 7ff1eea00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1d006c8f0 0x7ff1d006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0007fd0 tx=0x7ff1e0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:07.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.356+0000 7ff1e6a00640 1 -- 172.21.15.47:0/2001943087 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff1dc03f070 con 0x7ff1f0156c20 2024-04-20T10:28:07.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.558+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff1f00cfd90 con 0x7ff1f0156c20 2024-04-20T10:28:07.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.559+0000 7ff1e6a00640 1 -- 172.21.15.47:0/2001943087 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7ff1dc07c240 con 0x7ff1f0156c20 2024-04-20T10:28:07.559 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1d006c8f0 msgr2=0x7ff1d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1d006c8f0 0x7ff1d006edb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff1e0007fd0 tx=0x7ff1e0009040 comp rx=0 tx=0).stop 2024-04-20T10:28:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f0156c20 msgr2=0x7ff1f00999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f0156c20 0x7ff1f00999d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff1dc0090b0 tx=0x7ff1dc0046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 shutdown_connections 2024-04-20T10:28:07.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1f015ca40 0x7ff1f0152fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1d006c8f0 0x7ff1d006edb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff1f0156c20 0x7ff1f00999d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 --2- 172.21.15.47:0/2001943087 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1f00ad8e0 0x7ff1f00cf5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 >> 172.21.15.47:0/2001943087 conn(0x7ff1f00b3350 msgr2=0x7ff1f00676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 shutdown_connections 2024-04-20T10:28:07.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.562+0000 7ff1f5c00640 1 -- 172.21.15.47:0/2001943087 wait complete. 2024-04-20T10:28:07.582 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:28:07.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.789+0000 7fb54c600640 1 Processor -- start 2024-04-20T10:28:07.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.789+0000 7fb54c600640 1 -- start start 2024-04-20T10:28:07.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb54c600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb544156c20 0x7fb54406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb54c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb54c600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb544141c60 con 0x7fb5440ad8e0 2024-04-20T10:28:07.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb54c600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb54415cf90 con 0x7fb544156c20 2024-04-20T10:28:07.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb54c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb54415d270 con 0x7fb54415ca40 2024-04-20T10:28:07.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb549c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb544156c20 0x7fb54406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb54a600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb5440ad8e0 0x7fb5440d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60978/0 (socket says 172.21.15.47:60978) 2024-04-20T10:28:07.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb54a600640 1 -- 172.21.15.47:0/3086048012 learned_addr learned my addr 172.21.15.47:0/3086048012 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:07.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.791+0000 7fb549200640 1 --2- 172.21.15.47:0/3086048012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.792+0000 7fb543e00640 1 -- 172.21.15.47:0/3086048012 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 675188798 0 0) 0x7fb544141c60 con 0x7fb5440ad8e0 2024-04-20T10:28:07.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.792+0000 7fb543e00640 1 -- 172.21.15.47:0/3086048012 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb52c003520 con 0x7fb5440ad8e0 2024-04-20T10:28:07.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.792+0000 7fb549c00640 1 -- 172.21.15.47:0/3086048012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 msgr2=0x7fb5440ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:07.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.792+0000 7fb549c00640 1 --2- 172.21.15.47:0/3086048012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.792+0000 7fb549c00640 1 -- 172.21.15.47:0/3086048012 >> v1:172.21.15.47:6789/0 conn(0x7fb5440ad8e0 legacy=0x7fb5440d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.792+0000 7fb549c00640 1 -- 172.21.15.47:0/3086048012 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54415d550 con 0x7fb544156c20 2024-04-20T10:28:07.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.793+0000 7fb549c00640 1 --2- 172.21.15.47:0/3086048012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb54406b900 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb5380097a0 tx=0x7fb53802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c132642a21d5c8cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:07.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.793+0000 7fb543e00640 1 -- 172.21.15.47:0/3086048012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538035020 con 0x7fb544156c20 2024-04-20T10:28:07.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.793+0000 7fb543e00640 1 -- 172.21.15.47:0/3086048012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb538031810 con 0x7fb544156c20 2024-04-20T10:28:07.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.793+0000 7fb543e00640 1 -- 172.21.15.47:0/3086048012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb538031be0 con 0x7fb544156c20 2024-04-20T10:28:07.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 -- 172.21.15.47:0/3086048012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 msgr2=0x7fb54406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 --2- 172.21.15.47:0/3086048012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb54406b900 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fb5380097a0 tx=0x7fb53802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 -- 172.21.15.47:0/3086048012 shutdown_connections 2024-04-20T10:28:07.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 --2- 172.21.15.47:0/3086048012 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 --2- 172.21.15.47:0/3086048012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb54406b900 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 -- 172.21.15.47:0/3086048012 >> 172.21.15.47:0/3086048012 conn(0x7fb5440b3350 msgr2=0x7fb5440b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:07.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 -- 172.21.15.47:0/3086048012 shutdown_connections 2024-04-20T10:28:07.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.794+0000 7fb54c600640 1 -- 172.21.15.47:0/3086048012 wait complete. 2024-04-20T10:28:07.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.796+0000 7fb54c600640 1 Processor -- start 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.796+0000 7fb54c600640 1 -- start start 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb54c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5440ad8e0 0x7fb5441f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb54c600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb544072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb54c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440730e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb54c600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb54406bfc0 con 0x7fb544156c20 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb54c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb54406be40 con 0x7fb5440ad8e0 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb54c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5440d0a70 con 0x7fb54415ca40 2024-04-20T10:28:07.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb549200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb544072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb549200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb544072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57646/0 (socket says 172.21.15.47:57646) 2024-04-20T10:28:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.797+0000 7fb549200640 1 -- 172.21.15.47:0/1944312637 learned_addr learned my addr 172.21.15.47:0/1944312637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb54a600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549c00640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5440ad8e0 0x7fb5441f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549200640 1 -- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 msgr2=0x7fb5440730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549200640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549200640 1 -- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5440ad8e0 msgr2=0x7fb5441f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549200640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5440ad8e0 0x7fb5441f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549200640 1 -- 172.21.15.47:0/1944312637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5440ccb30 con 0x7fb544156c20 2024-04-20T10:28:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549c00640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5440ad8e0 0x7fb5441f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb549200640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb544072ba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb53000d5f0 tx=0x7fb53000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.798+0000 7fb541600640 1 -- 172.21.15.47:0/1944312637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5300125f0 con 0x7fb544156c20 2024-04-20T10:28:07.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.799+0000 7fb541600640 1 -- 172.21.15.47:0/1944312637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb530019070 con 0x7fb544156c20 2024-04-20T10:28:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.799+0000 7fb541600640 1 -- 172.21.15.47:0/1944312637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5300129d0 con 0x7fb544156c20 2024-04-20T10:28:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.799+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5440ccdd0 con 0x7fb544156c20 2024-04-20T10:28:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.799+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5440cd390 con 0x7fb544156c20 2024-04-20T10:28:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.800+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb544068980 con 0x7fb544156c20 2024-04-20T10:28:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.801+0000 7fb541600640 1 -- 172.21.15.47:0/1944312637 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb530012b70 con 0x7fb544156c20 2024-04-20T10:28:07.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.802+0000 7fb541600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb52406c8f0 0x7fb52406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:07.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.802+0000 7fb541600640 1 -- 172.21.15.47:0/1944312637 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fb53008c2e0 con 0x7fb544156c20 2024-04-20T10:28:07.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.802+0000 7fb549c00640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb52406c8f0 0x7fb52406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:07.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.804+0000 7fb549c00640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb52406c8f0 0x7fb52406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb538036000 tx=0x7fb538008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:07.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:07.807+0000 7fb541600640 1 -- 172.21.15.47:0/1944312637 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb53005a3d0 con 0x7fb544156c20 2024-04-20T10:28:08.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.051+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5440cfd90 con 0x7fb544156c20 2024-04-20T10:28:08.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.052+0000 7fb541600640 1 -- 172.21.15.47:0/1944312637 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7fb530059cd0 con 0x7fb544156c20 2024-04-20T10:28:08.052 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:08.052 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:28:08.053 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:28:08.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.054+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb52406c8f0 msgr2=0x7fb52406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.054+0000 7fb54c600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb52406c8f0 0x7fb52406edb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb538036000 tx=0x7fb538008030 comp rx=0 tx=0).stop 2024-04-20T10:28:08.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.054+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 msgr2=0x7fb544072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.054+0000 7fb54c600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb544072ba0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb53000d5f0 tx=0x7fb53000dab0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 shutdown_connections 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb54415ca40 0x7fb5440730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb52406c8f0 0x7fb52406edb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb544156c20 0x7fb544072ba0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 --2- 172.21.15.47:0/1944312637 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5440ad8e0 0x7fb5441f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 >> 172.21.15.47:0/1944312637 conn(0x7fb5440b3350 msgr2=0x7fb544154ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:08.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 shutdown_connections 2024-04-20T10:28:08.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.055+0000 7fb54c600640 1 -- 172.21.15.47:0/1944312637 wait complete. 2024-04-20T10:28:08.074 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.281+0000 7f9756600640 1 Processor -- start 2024-04-20T10:28:08.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.282+0000 7f9756600640 1 -- start start 2024-04-20T10:28:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.283+0000 7f9756600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f97500d1a00 0x7f97500b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.283+0000 7f9756600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97500b17d0 0x7f97500d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.283+0000 7f9756600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f97500d1e90 con 0x7f97500d4270 2024-04-20T10:28:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.283+0000 7f9756600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f97500d40c0 con 0x7f97500b17d0 2024-04-20T10:28:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.283+0000 7f9756600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f97500d3820 con 0x7f97500d1a00 2024-04-20T10:28:08.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97500b17d0 0x7f97500d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f97500b17d0 0x7f97500d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57660/0 (socket says 172.21.15.47:57660) 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974fe00640 1 -- 172.21.15.47:0/4241348722 learned_addr learned my addr 172.21.15.47:0/4241348722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974ea00640 1 --2- 172.21.15.47:0/4241348722 >> v2:172.21.15.175:3300/0 conn(0x7f97500d1a00 0x7f97500b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974fe00640 1 -- 172.21.15.47:0/4241348722 >> v2:172.21.15.175:3300/0 conn(0x7f97500d1a00 msgr2=0x7f97500b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974fe00640 1 --2- 172.21.15.47:0/4241348722 >> v2:172.21.15.175:3300/0 conn(0x7f97500d1a00 0x7f97500b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974fe00640 1 -- 172.21.15.47:0/4241348722 >> v1:172.21.15.175:6789/0 conn(0x7f97500d4270 legacy=0x7f97500d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.284+0000 7f974fe00640 1 -- 172.21.15.47:0/4241348722 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97500d3b00 con 0x7f97500b17d0 2024-04-20T10:28:08.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.285+0000 7f974fe00640 1 --2- 172.21.15.47:0/4241348722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500b17d0 0x7f97500d0380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97500b8060 tx=0x7f974802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873fb1dcb06f9583 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:08.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.286+0000 7f974e000640 1 -- 172.21.15.47:0/4241348722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748035020 con 0x7f97500b17d0 2024-04-20T10:28:08.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.286+0000 7f974e000640 1 -- 172.21.15.47:0/4241348722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9748031870 con 0x7f97500b17d0 2024-04-20T10:28:08.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.286+0000 7f974e000640 1 -- 172.21.15.47:0/4241348722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748031c40 con 0x7f97500b17d0 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 -- 172.21.15.47:0/4241348722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500b17d0 msgr2=0x7f97500d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 --2- 172.21.15.47:0/4241348722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500b17d0 0x7f97500d0380 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f97500b8060 tx=0x7f974802b910 comp rx=0 tx=0).stop 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 -- 172.21.15.47:0/4241348722 shutdown_connections 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 --2- 172.21.15.47:0/4241348722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500b17d0 0x7f97500d0380 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 --2- 172.21.15.47:0/4241348722 >> v2:172.21.15.175:3300/0 conn(0x7f97500d1a00 0x7f97500b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 -- 172.21.15.47:0/4241348722 >> 172.21.15.47:0/4241348722 conn(0x7f97500b99c0 msgr2=0x7f97500b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 -- 172.21.15.47:0/4241348722 shutdown_connections 2024-04-20T10:28:08.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.287+0000 7f9756600640 1 -- 172.21.15.47:0/4241348722 wait complete. 2024-04-20T10:28:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.289+0000 7f9756600640 1 Processor -- start 2024-04-20T10:28:08.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.289+0000 7f9756600640 1 -- start start 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f9756600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97500b17d0 0x7f975006c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f9756600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500d1a00 0x7f9750157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f9756600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97500d4270 0x7f97501f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f9756600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f97500d53b0 con 0x7f97500d1a00 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f9756600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97500cfe60 con 0x7f97500d4270 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f9756600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9750161f70 con 0x7f97500b17d0 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f974ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500d1a00 0x7f9750157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f974ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500d1a00 0x7f9750157d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57674/0 (socket says 172.21.15.47:57674) 2024-04-20T10:28:08.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f974ea00640 1 -- 172.21.15.47:0/679987537 learned_addr learned my addr 172.21.15.47:0/679987537 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f974f400640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97500b17d0 0x7f975006c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.290+0000 7f974fe00640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97500d4270 0x7f97501f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f974ea00640 1 -- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97500b17d0 msgr2=0x7f975006c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f974ea00640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97500b17d0 0x7f975006c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f974ea00640 1 -- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97500d4270 msgr2=0x7f97501f49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f974ea00640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97500d4270 0x7f97501f49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f974ea00640 1 -- 172.21.15.47:0/679987537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97501f4f00 con 0x7f97500d1a00 2024-04-20T10:28:08.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f974ea00640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500d1a00 0x7f9750157d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f973c00d800 tx=0x7f973c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f9747400640 1 -- 172.21.15.47:0/679987537 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c00b6c0 con 0x7f97500d1a00 2024-04-20T10:28:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f9747400640 1 -- 172.21.15.47:0/679987537 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f973c019070 con 0x7f97500d1a00 2024-04-20T10:28:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.291+0000 7f9747400640 1 -- 172.21.15.47:0/679987537 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c00ba80 con 0x7f97500d1a00 2024-04-20T10:28:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.292+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97501f51d0 con 0x7f97500d1a00 2024-04-20T10:28:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.292+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97501f5790 con 0x7f97500d1a00 2024-04-20T10:28:08.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.293+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97500cefc0 con 0x7f97500d1a00 2024-04-20T10:28:08.294 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.294+0000 7f9747400640 1 -- 172.21.15.47:0/679987537 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f973c00bc20 con 0x7f97500d1a00 2024-04-20T10:28:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.295+0000 7f9747400640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f973006c8f0 0x7f973006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.295+0000 7f9747400640 1 -- 172.21.15.47:0/679987537 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f973c08c240 con 0x7f97500d1a00 2024-04-20T10:28:08.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.296+0000 7f974f400640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f973006c8f0 0x7f973006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.297 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.297+0000 7f974f400640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f973006c8f0 0x7f973006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f974000b450 tx=0x7f974000d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:08.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.302+0000 7f9747400640 1 -- 172.21.15.47:0/679987537 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f973c059c40 con 0x7f97500d1a00 2024-04-20T10:28:08.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.510+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9750068980 con 0x7f97500d1a00 2024-04-20T10:28:08.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.513+0000 7f9747400640 1 -- 172.21.15.47:0/679987537 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18530 (secure 0 0 0) 0x7f973c05df40 con 0x7f97500d1a00 2024-04-20T10:28:08.514 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:08.514 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":27,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:27:37.571139+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f973006c8f0 msgr2=0x7f973006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f973006c8f0 0x7f973006edb0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f974000b450 tx=0x7f974000d040 comp rx=0 tx=0).stop 2024-04-20T10:28:08.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500d1a00 msgr2=0x7f9750157d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500d1a00 0x7f9750157d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f973c00d800 tx=0x7f973c00dcc0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 shutdown_connections 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97500d4270 0x7f97501f49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f973006c8f0 0x7f973006edb0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f97500d1a00 0x7f9750157d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 --2- 172.21.15.47:0/679987537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97500b17d0 0x7f975006c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 >> 172.21.15.47:0/679987537 conn(0x7f97500b99c0 msgr2=0x7f97500b64a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 shutdown_connections 2024-04-20T10:28:08.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.516+0000 7f9756600640 1 -- 172.21.15.47:0/679987537 wait complete. 2024-04-20T10:28:08.535 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:28:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.743+0000 7fbc0f200640 1 Processor -- start 2024-04-20T10:28:08.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.744+0000 7fbc0f200640 1 -- start start 2024-04-20T10:28:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.745+0000 7fbc0f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 0x7fbc080711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.745+0000 7fbc0f200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc08160250 con 0x7fbc080ce790 2024-04-20T10:28:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.745+0000 7fbc0f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbc08160530 con 0x7fbc080ccd50 2024-04-20T10:28:08.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.745+0000 7fbc0f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc08160dd0 con 0x7fbc080cf350 2024-04-20T10:28:08.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.745+0000 7fbc0e800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbc080ce790 0x7fbc080d0110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:60990/0 (socket says 172.21.15.47:60990) 2024-04-20T10:28:08.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.745+0000 7fbc0e800640 1 -- 172.21.15.47:0/1897798797 learned_addr learned my addr 172.21.15.47:0/1897798797 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:08.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 0x7fbc080711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 435823837 0 0) 0x7fbc08160250 con 0x7fbc080ce790 2024-04-20T10:28:08.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbbf4003520 con 0x7fbc080ce790 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1051494227 0 0) 0x7fbbf4003520 con 0x7fbc080ce790 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 msgr2=0x7fbc080711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 0x7fbc080711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 >> v1:172.21.15.175:6789/0 conn(0x7fbc080ccd50 legacy=0x7fbc080cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc08160810 con 0x7fbc080ce790 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fbc04001670 con 0x7fbc080ce790 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 >> v1:172.21.15.47:6789/0 conn(0x7fbc080ce790 legacy=0x7fbc080d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40039e0 0x7fbbf4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbbf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 0x7fbbf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbc08160dd0 con 0x7fbbf4004800 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf400baa0 con 0x7fbbf40039e0 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.746+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf400bca0 con 0x7fbc080cf350 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbbf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0e800640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 0x7fbbf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0de00640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40039e0 0x7fbbf4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0d400640 1 -- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 msgr2=0x7fbbf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080cf350 0x7fbbf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0d400640 1 -- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40039e0 msgr2=0x7fbbf4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf40039e0 0x7fbbf4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.747+0000 7fbc0d400640 1 -- 172.21.15.47:0/1897798797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf4003520 con 0x7fbbf4004800 2024-04-20T10:28:08.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.748+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbbf4004c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009a90 tx=0x7fbbf4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57142b977e248036 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:08.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8039070 con 0x7fbbf4004800 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbbf802c680 con 0x7fbbf4004800 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0ca00640 1 -- 172.21.15.47:0/1897798797 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf8009690 con 0x7fbbf4004800 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0f200640 1 -- 172.21.15.47:0/1897798797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 msgr2=0x7fbbf4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0f200640 1 --2- 172.21.15.47:0/1897798797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbbf4004c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8009a90 tx=0x7fbbf4005830 comp rx=0 tx=0).stop 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0e800640 1 -- 172.21.15.47:0/1897798797 reap_dead start 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0f200640 1 -- 172.21.15.47:0/1897798797 shutdown_connections 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0f200640 1 -- 172.21.15.47:0/1897798797 >> 172.21.15.47:0/1897798797 conn(0x7fbc080b3df0 msgr2=0x7fbc080b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0f200640 1 -- 172.21.15.47:0/1897798797 shutdown_connections 2024-04-20T10:28:08.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.749+0000 7fbc0f200640 1 -- 172.21.15.47:0/1897798797 wait complete. 2024-04-20T10:28:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.751+0000 7fbc0f200640 1 Processor -- start 2024-04-20T10:28:08.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.751+0000 7fbc0f200640 1 -- start start 2024-04-20T10:28:08.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.752+0000 7fbc0f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbc080c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.752+0000 7fbc0f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080c7d50 0x7fbc080c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.752+0000 7fbc0f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc080aed90 0x7fbc080af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.752+0000 7fbc0f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0815feb0 con 0x7fbbf4004800 2024-04-20T10:28:08.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.752+0000 7fbc0f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbc080cd130 con 0x7fbc080aed90 2024-04-20T10:28:08.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.752+0000 7fbc0f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbc08160030 con 0x7fbc080c7d50 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbc080c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbc080c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57696/0 (socket says 172.21.15.47:57696) 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 -- 172.21.15.47:0/1911382753 learned_addr learned my addr 172.21.15.47:0/1911382753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080c7d50 0x7fbc080c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0e800640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc080aed90 0x7fbc080af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 -- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080c7d50 msgr2=0x7fbc080c81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080c7d50 0x7fbc080c81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 -- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc080aed90 msgr2=0x7fbc080af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc080aed90 0x7fbc080af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:08.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.753+0000 7fbc0de00640 1 -- 172.21.15.47:0/1911382753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc0806df90 con 0x7fbbf4004800 2024-04-20T10:28:08.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.754+0000 7fbc0de00640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbc080c4cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000e8c0 tx=0x7fbc0000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:08.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.754+0000 7fbbfe000640 1 -- 172.21.15.47:0/1911382753 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc00003b80 con 0x7fbbf4004800 2024-04-20T10:28:08.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.754+0000 7fbbfe000640 1 -- 172.21.15.47:0/1911382753 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbc00018070 con 0x7fbbf4004800 2024-04-20T10:28:08.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.754+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc0806e270 con 0x7fbbf4004800 2024-04-20T10:28:08.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.754+0000 7fbbfe000640 1 -- 172.21.15.47:0/1911382753 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc000050d0 con 0x7fbbf4004800 2024-04-20T10:28:08.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.754+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc0806e830 con 0x7fbbf4004800 2024-04-20T10:28:08.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.755+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc08075b30 con 0x7fbbf4004800 2024-04-20T10:28:08.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.756+0000 7fbbfe000640 1 -- 172.21.15.47:0/1911382753 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbc00005270 con 0x7fbbf4004800 2024-04-20T10:28:08.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.757+0000 7fbbfe000640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbbec06c920 0x7fbbec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:08.757 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.758+0000 7fbbfe000640 1 -- 172.21.15.47:0/1911382753 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7fbc0008bf40 con 0x7fbbf4004800 2024-04-20T10:28:08.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.758+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbbec06c920 0x7fbbec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:08.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.759+0000 7fbc0d400640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbbec06c920 0x7fbbec06ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8036000 tx=0x7fbbf8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:08.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:08.762+0000 7fbbfe000640 1 -- 172.21.15.47:0/1911382753 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbc00059f90 con 0x7fbbf4004800 2024-04-20T10:28:09.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.004+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc08070f90 con 0x7fbbf4004800 2024-04-20T10:28:09.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.005+0000 7fbbfe000640 1 -- 172.21.15.47:0/1911382753 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7fbc00059890 con 0x7fbbf4004800 2024-04-20T10:28:09.006 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:09.006 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:28:09.006 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:28:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbbec06c920 msgr2=0x7fbbec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbbec06c920 0x7fbbec06ede0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8036000 tx=0x7fbbf8030000 comp rx=0 tx=0).stop 2024-04-20T10:28:09.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 msgr2=0x7fbc080c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbc080c4cf0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000e8c0 tx=0x7fbc0000ed80 comp rx=0 tx=0).stop 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 shutdown_connections 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbc080aed90 0x7fbc080af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbc080c7d50 0x7fbc080c81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbbec06c920 0x7fbbec06ede0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 --2- 172.21.15.47:0/1911382753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbbf4004800 0x7fbc080c4cf0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 >> 172.21.15.47:0/1911382753 conn(0x7fbc080b3df0 msgr2=0x7fbc080cff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 shutdown_connections 2024-04-20T10:28:09.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.008+0000 7fbc0f200640 1 -- 172.21.15.47:0/1911382753 wait complete. 2024-04-20T10:28:09.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:27:32.532668+0000', 'modified': '2024-04-20T10:27:53.683601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4285}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4285': {'gid': 4285, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/2667922035', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6835', 'nonce': 2667922035}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2667922035}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4285, 'qdb_cluster': [4285]} 2024-04-20T10:28:09.430 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:28:09.430 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-20T10:28:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.650+0000 7f1a35200640 1 Processor -- start 2024-04-20T10:28:09.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.651+0000 7f1a35200640 1 -- start start 2024-04-20T10:28:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.652+0000 7f1a35200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a300af800 0x7f1a3015abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.652+0000 7f1a35200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a300ae9b0 con 0x7f1a30071b50 2024-04-20T10:28:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.652+0000 7f1a35200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a3006e840 con 0x7f1a300725b0 2024-04-20T10:28:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.652+0000 7f1a35200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1a300d0a70 con 0x7f1a300af800 2024-04-20T10:28:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.653+0000 7f1a25400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a300af800 0x7f1a3015abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.653+0000 7f1a25400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1a300af800 0x7f1a3015abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57702/0 (socket says 172.21.15.47:57702) 2024-04-20T10:28:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.653+0000 7f1a25400640 1 -- 172.21.15.47:0/2216092442 learned_addr learned my addr 172.21.15.47:0/2216092442 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:09.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.653+0000 7f1a25400640 1 -- 172.21.15.47:0/2216092442 >> v1:172.21.15.175:6789/0 conn(0x7f1a300725b0 legacy=0x7f1a300729c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.653+0000 7f1a25400640 1 -- 172.21.15.47:0/2216092442 >> v1:172.21.15.47:6789/0 conn(0x7f1a30071b50 legacy=0x7f1a300d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.653+0000 7f1a25400640 1 -- 172.21.15.47:0/2216092442 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3015b350 con 0x7f1a300af800 2024-04-20T10:28:09.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.654+0000 7f1a25400640 1 --2- 172.21.15.47:0/2216092442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300af800 0x7f1a3015abb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a1802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6221d07baba9664 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:09.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a2d400640 1 -- 172.21.15.47:0/2216092442 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18034070 con 0x7f1a300af800 2024-04-20T10:28:09.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a2d400640 1 -- 172.21.15.47:0/2216092442 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a18030860 con 0x7f1a300af800 2024-04-20T10:28:09.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a2d400640 1 -- 172.21.15.47:0/2216092442 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18030c30 con 0x7f1a300af800 2024-04-20T10:28:09.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a35200640 1 -- 172.21.15.47:0/2216092442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300af800 msgr2=0x7f1a3015abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a35200640 1 --2- 172.21.15.47:0/2216092442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300af800 0x7f1a3015abb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a1802b9a0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a35200640 1 -- 172.21.15.47:0/2216092442 shutdown_connections 2024-04-20T10:28:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a35200640 1 --2- 172.21.15.47:0/2216092442 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300af800 0x7f1a3015abb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a35200640 1 -- 172.21.15.47:0/2216092442 >> 172.21.15.47:0/2216092442 conn(0x7f1a300b3350 msgr2=0x7f1a300b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.655+0000 7f1a35200640 1 -- 172.21.15.47:0/2216092442 shutdown_connections 2024-04-20T10:28:09.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.656+0000 7f1a35200640 1 -- 172.21.15.47:0/2216092442 wait complete. 2024-04-20T10:28:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.657+0000 7f1a35200640 1 Processor -- start 2024-04-20T10:28:09.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.657+0000 7f1a35200640 1 -- start start 2024-04-20T10:28:09.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.658+0000 7f1a35200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30071b50 0x7f1a3014c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.658+0000 7f1a35200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300725b0 0x7f1a301542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.658+0000 7f1a35200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a300af800 0x7f1a300b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.658+0000 7f1a35200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3015bf60 con 0x7f1a300725b0 2024-04-20T10:28:09.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.658+0000 7f1a35200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3015b0f0 con 0x7f1a30071b50 2024-04-20T10:28:09.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.658+0000 7f1a35200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a3015b910 con 0x7f1a300af800 2024-04-20T10:28:09.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.658+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300725b0 0x7f1a301542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300725b0 0x7f1a301542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57712/0 (socket says 172.21.15.47:57712) 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 -- 172.21.15.47:0/681414074 learned_addr learned my addr 172.21.15.47:0/681414074 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a2e800640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a300af800 0x7f1a300b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a2de00640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30071b50 0x7f1a3014c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 -- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a300af800 msgr2=0x7f1a300b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a300af800 0x7f1a300b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 -- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30071b50 msgr2=0x7f1a3014c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30071b50 0x7f1a3014c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 -- 172.21.15.47:0/681414074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a30154d00 con 0x7f1a300725b0 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a2e800640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a300af800 0x7f1a300b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a2de00640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30071b50 0x7f1a3014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:09.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.659+0000 7f1a25400640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300725b0 0x7f1a301542b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.660+0000 7f1a26a00640 1 -- 172.21.15.47:0/681414074 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18034040 con 0x7f1a300725b0 2024-04-20T10:28:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.660+0000 7f1a26a00640 1 -- 172.21.15.47:0/681414074 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a180433f0 con 0x7f1a300725b0 2024-04-20T10:28:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.660+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a301f4de0 con 0x7f1a300725b0 2024-04-20T10:28:09.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.660+0000 7f1a26a00640 1 -- 172.21.15.47:0/681414074 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a180437e0 con 0x7f1a300725b0 2024-04-20T10:28:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.660+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a301f53a0 con 0x7f1a300725b0 2024-04-20T10:28:09.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.662+0000 7f1a26a00640 1 -- 172.21.15.47:0/681414074 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a1804a060 con 0x7f1a300725b0 2024-04-20T10:28:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.663+0000 7f1a26a00640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a0c06c8f0 0x7f1a0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.663+0000 7f1a26a00640 1 -- 172.21.15.47:0/681414074 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1a180ae1b0 con 0x7f1a300725b0 2024-04-20T10:28:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.664+0000 7f1a37df6740 1 Processor -- start 2024-04-20T10:28:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.664+0000 7f1a2de00640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a0c06c8f0 0x7f1a0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.664+0000 7f1a37df6740 1 -- start start 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a2de00640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a0c06c8f0 0x7f1a0c06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a301f2750 tx=0x7f1a20009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8ba96a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8ba96fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba91930 0x55da8bac1770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a37df6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da8b9fd9d0 con 0x55da8ba954e0 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da8b9a4270 con 0x55da8ba91930 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55da8b9f9990 con 0x55da8badf460 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8ba96fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8ba96fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57728/0 (socket says 172.21.15.47:57728) 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.665+0000 7f1a25400640 1 -- 172.21.15.47:0/4252373375 learned_addr learned my addr 172.21.15.47:0/4252373375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a2de00640 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8ba96a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a2e800640 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba91930 0x55da8bac1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a25400640 1 -- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 msgr2=0x55da8ba96a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a25400640 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8ba96a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a25400640 1 -- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba91930 msgr2=0x55da8bac1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a25400640 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba91930 0x55da8bac1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a25400640 1 -- 172.21.15.47:0/4252373375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da8b7102d0 con 0x55da8ba954e0 2024-04-20T10:28:09.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a25400640 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8ba96fd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a180ac2d0 tx=0x7f1a1807b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150fb49b9ff61eb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a1ea00640 1 -- 172.21.15.47:0/4252373375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18094170 con 0x55da8ba954e0 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a1ea00640 1 -- 172.21.15.47:0/4252373375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1807ff40 con 0x55da8ba954e0 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.666+0000 7f1a1ea00640 1 -- 172.21.15.47:0/4252373375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18084770 con 0x55da8ba954e0 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 -- 172.21.15.47:0/4252373375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 msgr2=0x55da8ba96fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8ba96fd0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1a180ac2d0 tx=0x7f1a1807b5b0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 -- 172.21.15.47:0/4252373375 shutdown_connections 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba91930 0x55da8bac1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8ba96fd0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4252373375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8ba96a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 -- 172.21.15.47:0/4252373375 >> 172.21.15.47:0/4252373375 conn(0x55da8ba93bf0 msgr2=0x55da8badc770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:09.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 -- 172.21.15.47:0/4252373375 shutdown_connections 2024-04-20T10:28:09.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.667+0000 7f1a37df6740 1 -- 172.21.15.47:0/4252373375 wait complete. 2024-04-20T10:28:09.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.668+0000 7f1a37df6740 1 -- start start 2024-04-20T10:28:09.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.668+0000 7f1a37df6740 1 Processor -- start 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.669+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba954e0 0x55da8bab6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.669+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8badf460 0x55da8bae3830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.669+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8bac5360 0x55da8bac57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.669+0000 7f1a37df6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da8b9fd9d0 con 0x55da8badf460 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.669+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da8b9b75e0 con 0x55da8ba954e0 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.669+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55da8ba959f0 con 0x55da8bac5360 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a37df6740 10 client.0 _ll_register_callbacks cb 0x55da8b70fac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8badf460 0x55da8bae3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8badf460 0x55da8bae3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57734/0 (socket says 172.21.15.47:57734) 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 -- 172.21.15.47:0/1811702865 learned_addr learned my addr 172.21.15.47:0/1811702865 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:09.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a2e800640 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8bac5360 0x55da8bac57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a2de00640 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba954e0 0x55da8bab6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 -- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8bac5360 msgr2=0x55da8bac57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8bac5360 0x55da8bac57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 -- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba954e0 msgr2=0x55da8bab6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba954e0 0x55da8bab6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.670+0000 7f1a25400640 1 -- 172.21.15.47:0/1811702865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da8b7da6b0 con 0x55da8badf460 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a25400640 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8badf460 0x55da8bae3830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a1807c1f0 tx=0x7f1a18000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8c6ebd73433441d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a13400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a13400640 1 -- 172.21.15.47:0/1811702865 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a18094170 con 0x55da8badf460 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a13400640 1 -- 172.21.15.47:0/1811702865 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1a18030720 con 0x55da8badf460 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1807ff40 con 0x55da8badf460 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a13400640 1 -- 172.21.15.47:0/1811702865 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f1a18084770 con 0x55da8badf460 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a37df6740 10 client.4821 fetch_fsmap learned FSMap version 9 2024-04-20T10:28:09.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55da8babfcb0 con 0x55da8badf460 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a13400640 1 -- 172.21.15.47:0/1811702865 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a1807ff40 con 0x55da8badf460 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a13400640 1 -- 172.21.15.47:0/1811702865 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 9) ==== 3503+0+0 (secure 0 0 0) 0x7f1a18078200 con 0x55da8badf460 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a37df6740 10 client.4821 fetch_fsmap finished waiting for FSMap version 9 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a37df6740 10 client.4821 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.671+0000 7f1a37df6740 20 client.4821 populate_metadata read hostname 'smithi047' 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.672+0000 7f1a37df6740 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bac0620 0x55da8bac0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.672+0000 7f1a37df6740 4 client.4821 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.672+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 --> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da8ba0b3f0 con 0x55da8bac0620 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.672+0000 7f1a2de00640 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bac0620 0x55da8bac0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.673+0000 7f1a2de00640 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bac0620 0x55da8bac0a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87577d80e4876c4 server_cookie=3754cab9e24083d2 in_seq=0 out_seq=0 2024-04-20T10:28:09.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.673+0000 7f1a13400640 10 client.4821 ms_handle_connect on v2:172.21.15.175:6835/2667922035 2024-04-20T10:28:09.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a13400640 1 -- 172.21.15.47:0/1811702865 <== mds.0 v2:172.21.15.175:6835/2667922035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da8ba0b3f0 con 0x55da8bac0620 2024-04-20T10:28:09.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a13400640 10 client.4821 handle_command_reply: tid=0 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 client.4821 shutdown 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8badf460 msgr2=0x55da8bae3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8badf460 0x55da8bae3830 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f1a1807c1f0 tx=0x7f1a18000bc0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 shutdown_connections 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8bac5360 0x55da8bac57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8badf460 0x55da8bae3830 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bac0620 0x55da8bac0a00 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 --2- 172.21.15.47:0/1811702865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8ba954e0 0x55da8bab6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 >> 172.21.15.47:0/1811702865 conn(0x55da8ba91930 msgr2=0x55da8ba96850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a13400640 0 client.4821 ms_handle_reset on v2:172.21.15.175:6835/2667922035 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 shutdown_connections 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 1 -- 172.21.15.47:0/1811702865 wait complete. 2024-04-20T10:28:09.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.676+0000 7f1a37df6740 20 client.4821 trim_cache size 0 max 16384 2024-04-20T10:28:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.696+0000 7f1a37df6740 1 Processor -- start 2024-04-20T10:28:09.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.696+0000 7f1a37df6740 1 -- start start 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.697+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8ba954e0 0x55da8badb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.697+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.697+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.697+0000 7f1a37df6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1a30201430 con 0x55da8bac5360 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.697+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3015cc80 con 0x55da8badf460 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.697+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a30200e40 con 0x55da8ba954e0 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57750/0 (socket says 172.21.15.47:57750) 2024-04-20T10:28:09.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 -- 172.21.15.47:0/3213404032 learned_addr learned my addr 172.21.15.47:0/3213404032 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a2de00640 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8ba954e0 0x55da8badb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a2e800640 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 -- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8ba954e0 msgr2=0x55da8badb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8ba954e0 0x55da8badb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 -- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8badf460 msgr2=0x55da8bac0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.698+0000 7f1a25400640 1 -- 172.21.15.47:0/3213404032 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da8bae37e0 con 0x55da8bac5360 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.699+0000 7f1a25400640 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a1807c1c0 tx=0x7f1a18092070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e94cc5ffbc3eb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.699+0000 7f1a1e000640 1 -- 172.21.15.47:0/3213404032 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a180842e0 con 0x55da8bac5360 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.699+0000 7f1a1e000640 1 -- 172.21.15.47:0/3213404032 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a180ab070 con 0x55da8bac5360 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.699+0000 7f1a1e000640 1 -- 172.21.15.47:0/3213404032 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1807ff40 con 0x55da8bac5360 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 -- 172.21.15.47:0/3213404032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8bac5360 msgr2=0x55da8bac0a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1a1807c1c0 tx=0x7f1a18092070 comp rx=0 tx=0).stop 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 -- 172.21.15.47:0/3213404032 shutdown_connections 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 --2- 172.21.15.47:0/3213404032 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8ba954e0 0x55da8badb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 -- 172.21.15.47:0/3213404032 >> 172.21.15.47:0/3213404032 conn(0x55da8ba91930 msgr2=0x55da8bab4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 -- 172.21.15.47:0/3213404032 shutdown_connections 2024-04-20T10:28:09.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 -- 172.21.15.47:0/3213404032 wait complete. 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.700+0000 7f1a37df6740 1 -- start start 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.701+0000 7f1a37df6740 1 Processor -- start 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8baef580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a37df6740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a37df6740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55da8b9fd9d0 con 0x55da8ba954e0 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55da8b9b75e0 con 0x55da8bac5360 2024-04-20T10:28:09.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a37df6740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55da8bae1310 con 0x55da8badf460 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a37df6740 10 client.0 _ll_register_callbacks cb 0x7f1a18093070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8baef580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8baef580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57760/0 (socket says 172.21.15.47:57760) 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a2de00640 1 -- 172.21.15.47:0/4026349310 learned_addr learned my addr 172.21.15.47:0/4026349310 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a2de00640 1 -- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 msgr2=0x55da8bac0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a2e800640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.702+0000 7f1a25400640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a2de00640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a2de00640 1 -- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8bac5360 msgr2=0x55da8bac0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a2de00640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a2de00640 1 -- 172.21.15.47:0/4026349310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da8baf0000 con 0x55da8ba954e0 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a2e800640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a25400640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:09.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a2de00640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8baef580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55da8baf09d0 tx=0x7f1a20010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134e7a533d486c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a13400640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a13400640 1 -- 172.21.15.47:0/4026349310 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a200054e0 con 0x55da8ba954e0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a13400640 1 -- 172.21.15.47:0/4026349310 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f1a2002fe00 con 0x55da8ba954e0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.703+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a180ab070 con 0x55da8ba954e0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a13400640 1 -- 172.21.15.47:0/4026349310 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) ==== 24+0+0 (secure 0 0 0) 0x7f1a20005680 con 0x55da8ba954e0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 10 client.4827 fetch_fsmap learned FSMap version 9 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) -- 0x55da8bab8e50 con 0x55da8ba954e0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a13400640 1 -- 172.21.15.47:0/4026349310 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a180ab070 con 0x55da8ba954e0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a13400640 1 -- 172.21.15.47:0/4026349310 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 9) ==== 3503+0+0 (secure 0 0 0) 0x7f1a20036050 con 0x55da8ba954e0 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 10 client.4827 fetch_fsmap finished waiting for FSMap version 9 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 10 client.4827 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 20 client.4827 populate_metadata read hostname 'smithi047' 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bada3a0 0x55da8bada780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:09.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 4 client.4827 mds_command: new command op to 4285 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:28:09.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.704+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 --> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] -- command(tid 0: {"prefix": "status"}) -- 0x55da8ba05260 con 0x55da8bada3a0 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.705+0000 7f1a25400640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bada3a0 0x55da8bada780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.705+0000 7f1a25400640 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bada3a0 0x55da8bada780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38d544158faf6dd server_cookie=8d40296e29c1a03a in_seq=0 out_seq=0 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.705+0000 7f1a13400640 10 client.4827 ms_handle_connect on v2:172.21.15.175:6835/2667922035 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a13400640 1 -- 172.21.15.47:0/4026349310 <== mds.0 v2:172.21.15.175:6835/2667922035 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x55da8ba05260 con 0x55da8bada3a0 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a13400640 10 client.4827 handle_command_reply: tid=0 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 client.4827 shutdown 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 msgr2=0x55da8baef580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8baef580 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x55da8baf09d0 tx=0x7f1a20010000 comp rx=0 tx=0).stop 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 shutdown_connections 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55da8badf460 0x55da8bac0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:6835/2667922035,v1:172.21.15.175:6837/2667922035] conn(0x55da8bada3a0 0x55da8bada780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55da8bac5360 0x55da8bac0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 --2- 172.21.15.47:0/4026349310 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55da8ba954e0 0x55da8baef580 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a13400640 0 client.4827 ms_handle_reset on v2:172.21.15.175:6835/2667922035 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 >> 172.21.15.47:0/4026349310 conn(0x55da8ba91930 msgr2=0x55da8bae1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 shutdown_connections 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.706+0000 7f1a37df6740 1 -- 172.21.15.47:0/4026349310 wait complete. 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a37df6740 20 client.4827 trim_cache size 0 max 16384 2024-04-20T10:28:09.709 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "775946e6-93c9-4843-aaf4-db758801eec1", 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4285, 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 36.660490101999997, 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 9, 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 27, 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 27, 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 42.485526976999999 2024-04-20T10:28:09.710 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:28:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a0c06c8f0 msgr2=0x7f1a0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a0c06c8f0 0x7f1a0c06edb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1a301f2750 tx=0x7f1a20009040 comp rx=0 tx=0).stop 2024-04-20T10:28:09.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300725b0 msgr2=0x7f1a301542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300725b0 0x7f1a301542b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1a18009870 tx=0x7f1a180046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 shutdown_connections 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a300af800 0x7f1a300b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a0c06c8f0 0x7f1a0c06edb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a300725b0 0x7f1a301542b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 --2- 172.21.15.47:0/681414074 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a30071b50 0x7f1a3014c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 >> 172.21.15.47:0/681414074 conn(0x7f1a300b3350 msgr2=0x7f1a3006c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.707+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 shutdown_connections 2024-04-20T10:28:09.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:09.708+0000 7f1a35200640 1 -- 172.21.15.47:0/681414074 wait complete. 2024-04-20T10:28:09.725 INFO:teuthology.run_tasks:Running task sequential... 2024-04-20T10:28:09.738 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-20T10:28:09.738 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-20T10:28:09.738 INFO:teuthology.task.print:Enabling mgr modules 2024-04-20T10:28:09.738 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-04-20T10:28:09.748 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-04-20T10:28:09.748 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-20T10:28:09.748 INFO:tasks.ceph_fuse:client.0 config is {} 2024-04-20T10:28:09.748 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:28:09.748 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:09.749 INFO:tasks.ceph_fuse:client.2 config is {} 2024-04-20T10:28:09.749 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:28:09.749 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:09.749 INFO:tasks.ceph_fuse:client.1 config is {} 2024-04-20T10:28:09.749 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:28:09.749 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:09.749 INFO:tasks.ceph_fuse:client.3 config is {} 2024-04-20T10:28:09.749 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-20T10:28:09.749 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:09.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:09.749 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T10:28:09.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:09.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-04-20T10:28:09.902 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-04-20T10:28:09.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:28:09.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:09.906 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:28:09.967 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-20T10:28:09.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:09.969 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-20T10:28:10.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:10.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-20T10:28:10.167 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-04-20T10:28:10.167 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:28:10.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.354+0000 7f0c4ee00640 1 Processor -- start 2024-04-20T10:28:10.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.354+0000 7f0c4ee00640 1 -- start start 2024-04-20T10:28:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4ee00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0c48163b60 0x7f0c48071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4ee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4806c310 con 0x7f0c48163070 2024-04-20T10:28:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4ee00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0c4806c5f0 con 0x7f0c481651d0 2024-04-20T10:28:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4ee00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0c48073cf0 con 0x7f0c48163b60 2024-04-20T10:28:10.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4e400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0c48163b60 0x7f0c48071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4e400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0c48163b60 0x7f0c48071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57762/0 (socket says 172.21.15.47:57762) 2024-04-20T10:28:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4e400640 1 -- 172.21.15.47:0/3158203257 learned_addr learned my addr 172.21.15.47:0/3158203257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.356+0000 7f0c4da00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0c48163070 0x7f0c480735e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:32774/0 (socket says 172.21.15.47:32774) 2024-04-20T10:28:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.357+0000 7f0c4e400640 1 -- 172.21.15.47:0/3158203257 >> v1:172.21.15.175:6789/0 conn(0x7f0c481651d0 legacy=0x7f0c48163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.357+0000 7f0c4e400640 1 -- 172.21.15.47:0/3158203257 >> v1:172.21.15.47:6789/0 conn(0x7f0c48163070 legacy=0x7f0c480735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:10.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.357+0000 7f0c4e400640 1 -- 172.21.15.47:0/3158203257 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4806c8d0 con 0x7f0c48163b60 2024-04-20T10:28:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.358+0000 7f0c4e400640 1 --2- 172.21.15.47:0/3158203257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163b60 0x7f0c48071d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009e50 tx=0x7f0c4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd4f204436a1741 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.358+0000 7f0c47e00640 1 -- 172.21.15.47:0/3158203257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4002ca80 con 0x7f0c48163b60 2024-04-20T10:28:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.358+0000 7f0c47e00640 1 -- 172.21.15.47:0/3158203257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c4002cc20 con 0x7f0c48163b60 2024-04-20T10:28:10.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.358+0000 7f0c47e00640 1 -- 172.21.15.47:0/3158203257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c400343f0 con 0x7f0c48163b60 2024-04-20T10:28:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.359+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3158203257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163b60 msgr2=0x7f0c48071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.359+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3158203257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163b60 0x7f0c48071d90 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0c40009e50 tx=0x7f0c4002bee0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.359+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3158203257 shutdown_connections 2024-04-20T10:28:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.359+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3158203257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163b60 0x7f0c48071d90 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.359+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3158203257 >> 172.21.15.47:0/3158203257 conn(0x7f0c480d0040 msgr2=0x7f0c480d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:10.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.359+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3158203257 shutdown_connections 2024-04-20T10:28:10.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.359+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3158203257 wait complete. 2024-04-20T10:28:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.361+0000 7f0c4ee00640 1 Processor -- start 2024-04-20T10:28:10.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.361+0000 7f0c4ee00640 1 -- start start 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.362+0000 7f0c4ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163070 0x7f0c4809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.362+0000 7f0c4ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c48163b60 0x7f0c481642f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.362+0000 7f0c4ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c481651d0 0x7f0c480b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.362+0000 7f0c4ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0c481662a0 con 0x7f0c48163070 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.362+0000 7f0c4ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c4806ce90 con 0x7f0c48163b60 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.362+0000 7f0c4ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0c4816d630 con 0x7f0c481651d0 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.362+0000 7f0c4da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163070 0x7f0c4809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:10.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163070 0x7f0c4809a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57770/0 (socket says 172.21.15.47:57770) 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 -- 172.21.15.47:0/3107915334 learned_addr learned my addr 172.21.15.47:0/3107915334 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4e400640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c481651d0 0x7f0c480b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4d000640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c48163b60 0x7f0c481642f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 -- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c481651d0 msgr2=0x7f0c480b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c481651d0 0x7f0c480b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 -- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c48163b60 msgr2=0x7f0c481642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c48163b60 0x7f0c481642f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 -- 172.21.15.47:0/3107915334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4809d3e0 con 0x7f0c48163070 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4d000640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c48163b60 0x7f0c481642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.363+0000 7f0c4da00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163070 0x7f0c4809a9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c3c0 tx=0x7f0c3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.364+0000 7f0c45600640 1 -- 172.21.15.47:0/3107915334 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0074e0 con 0x7f0c48163070 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.364+0000 7f0c45600640 1 -- 172.21.15.47:0/3107915334 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0c3c007680 con 0x7f0c48163070 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.364+0000 7f0c45600640 1 -- 172.21.15.47:0/3107915334 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c004a00 con 0x7f0c48163070 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.364+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4809d6c0 con 0x7f0c48163070 2024-04-20T10:28:10.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.364+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c4809b8a0 con 0x7f0c48163070 2024-04-20T10:28:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.365+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c4809aff0 con 0x7f0c48163070 2024-04-20T10:28:10.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.366+0000 7f0c45600640 1 -- 172.21.15.47:0/3107915334 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0c3c01a020 con 0x7f0c48163070 2024-04-20T10:28:10.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.367+0000 7f0c45600640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0c2806c8f0 0x7f0c2806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:10.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.367+0000 7f0c45600640 1 -- 172.21.15.47:0/3107915334 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6532+0+0 (secure 0 0 0) 0x7f0c3c08b650 con 0x7f0c48163070 2024-04-20T10:28:10.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.368+0000 7f0c4d000640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0c2806c8f0 0x7f0c2806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:10.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.369+0000 7f0c4d000640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0c2806c8f0 0x7f0c2806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c48158400 tx=0x7f0c34009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:10.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.371+0000 7f0c45600640 1 -- 172.21.15.47:0/3107915334 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0c4809aff0 con 0x7f0c48163070 2024-04-20T10:28:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.618+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0c4809baa0 con 0x7f0c48163070 2024-04-20T10:28:10.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.619+0000 7f0c45600640 1 -- 172.21.15.47:0/3107915334 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f0c4809baa0 con 0x7f0c48163070 2024-04-20T10:28:10.619 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0c2806c8f0 msgr2=0x7f0c2806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0c2806c8f0 0x7f0c2806edb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0c48158400 tx=0x7f0c34009040 comp rx=0 tx=0).stop 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163070 msgr2=0x7f0c4809a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163070 0x7f0c4809a9f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00c3c0 tx=0x7f0c3c00c880 comp rx=0 tx=0).stop 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 shutdown_connections 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c481651d0 0x7f0c480b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c48163b60 0x7f0c481642f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0c2806c8f0 0x7f0c2806edb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 --2- 172.21.15.47:0/3107915334 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0c48163070 0x7f0c4809a9f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:10.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.621+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 >> 172.21.15.47:0/3107915334 conn(0x7f0c480d0040 msgr2=0x7f0c48160ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:10.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.622+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 shutdown_connections 2024-04-20T10:28:10.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10.622+0000 7f0c4ee00640 1 -- 172.21.15.47:0/3107915334 wait complete. 2024-04-20T10:28:10.641 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:28:10.641 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:28:10.641 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-20T10:28:10.642 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:28:10.642 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:10.642 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:28:10.642 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:28:10.642 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:28:10.642 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:28:10.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:10.642 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:10.696 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42792sec preferred_lft 42792sec 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:10.697 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-20T10:28:10.698 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:28:10.699 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:10.699 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:10.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:28:10.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr flush dev ceph-brx 2024-04-20T10:28:10.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set ceph-brx up 2024-04-20T10:28:10.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:28:10.699 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:10.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:10.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:10.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:10.959 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:28:11.031 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-20T10:28:11.035 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:11.035 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-20T10:28:11.090 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-20T10:28:11.091 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:28:11.091 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:28:11.091 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:28:11.091 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:28:11.091 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:11.091 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:11.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:28:11.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:28:11.091 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:28:11.092 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:11.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:11.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:11.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:11.311 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:28:11.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:11.362 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-20T10:28:11.413 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:11.413 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:11.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:11.413 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-20T10:28:11.413 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:11.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:11.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:11.601 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-20T10:28:11.601 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:11.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:11.883 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:11.883 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:11.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-20T10:28:11.883 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:28:11.883 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:11.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:12.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:12.089 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T10:28:12.090 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:12.090 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:12.138 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:28:12.138 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:12.138 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:12.230 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:28:12.311 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:28:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:28:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:28:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:28:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:28:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:28:12.365 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:28:12.366 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:12.367 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:12.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:12.368 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:28:12.414 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:28:12.414 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T10:28:12.461 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:28:12.533 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:12.575 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:28:12.576 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:28:12.577 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:28:12.578 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:12.578 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:12.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:12.579 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:28:12.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-04-20T10:28:12.637+0000 7fad7763b480 -1 init, newargv = 0x55aa80b75230 newargc=15 2024-04-20T10:28:12.637 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[31194]: starting ceph client 2024-04-20T10:28:12.649 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[31194]: starting fuse 2024-04-20T10:28:12.661 INFO:teuthology.orchestra.run.smithi047.stdout:42 2024-04-20T10:28:12.661 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-04-20T10:28:12.662 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-04-20T10:28:12.663 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:28:12.862 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.31194.asok 2024-04-20T10:28:12.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:12.880 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.31194.asok 2024-04-20T10:28:12.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:12.880 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.31194.asok status 2024-04-20T10:28:13.110 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "31194", 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-04-20T10:28:13.111 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4842, 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4842 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4013942443 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:28:13.112 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "172.21.15.47:0", 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 4013942443 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4842 172.21.15.47:0/4013942443", 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "172.21.15.47:0/4013942443", 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 9, 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 27, 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-04-20T10:28:13.113 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:28:13.125 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:28:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.321+0000 7f3a11c00640 1 Processor -- start 2024-04-20T10:28:13.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.322+0000 7f3a11c00640 1 -- start start 2024-04-20T10:28:13.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.323+0000 7f3a11c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c15e630 0x7f3a0c0c8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.323+0000 7f3a11c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f3a0c0d1350 0x7f3a0c0cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.323+0000 7f3a11c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a0c0cda10 0x7f3a0c0cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.323+0000 7f3a11c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f3a0c0b6100 con 0x7f3a0c0cda10 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.323+0000 7f3a11c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3a0c0d18a0 con 0x7f3a0c0d1350 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.323+0000 7f3a11c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0c0d1b80 con 0x7f3a0c15e630 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.323+0000 7f3a11200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a0c0cda10 0x7f3a0c0cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a11200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f3a0c0cda10 0x7f3a0c0cfe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57774/0 (socket says 172.21.15.47:57774) 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a11200640 1 -- 172.21.15.47:0/4267210303 learned_addr learned my addr 172.21.15.47:0/4267210303 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:13.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a03400640 1 --2- 172.21.15.47:0/4267210303 >> v2:172.21.15.175:3300/0 conn(0x7f3a0c0d1350 0x7f3a0c0cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a0be00640 1 --2- 172.21.15.47:0/4267210303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c15e630 0x7f3a0c0c8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a11200640 1 -- 172.21.15.47:0/4267210303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c15e630 msgr2=0x7f3a0c0c8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a11200640 1 --2- 172.21.15.47:0/4267210303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c15e630 0x7f3a0c0c8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a11200640 1 -- 172.21.15.47:0/4267210303 >> v2:172.21.15.175:3300/0 conn(0x7f3a0c0d1350 msgr2=0x7f3a0c0cd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a11200640 1 --2- 172.21.15.47:0/4267210303 >> v2:172.21.15.175:3300/0 conn(0x7f3a0c0d1350 0x7f3a0c0cd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.324+0000 7f3a11200640 1 -- 172.21.15.47:0/4267210303 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c0d1e60 con 0x7f3a0c0cda10 2024-04-20T10:28:13.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.326+0000 7f3a11200640 1 --2- 172.21.15.47:0/4267210303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c0cda10 0x7f3a0c0cfe30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0cfd70 tx=0x7f3a0402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ee4dd430b6fba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.328+0000 7f3a0b400640 1 -- 172.21.15.47:0/4267210303 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0402c9f0 con 0x7f3a0c0cda10 2024-04-20T10:28:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.328+0000 7f3a0b400640 1 -- 172.21.15.47:0/4267210303 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0402cb90 con 0x7f3a0c0cda10 2024-04-20T10:28:13.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.328+0000 7f3a0b400640 1 -- 172.21.15.47:0/4267210303 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a040343f0 con 0x7f3a0c0cda10 2024-04-20T10:28:13.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.329+0000 7f3a11c00640 1 -- 172.21.15.47:0/4267210303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c0cda10 msgr2=0x7f3a0c0cfe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:13.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.329+0000 7f3a11c00640 1 --2- 172.21.15.47:0/4267210303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c0cda10 0x7f3a0c0cfe30 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0cfd70 tx=0x7f3a0402be50 comp rx=0 tx=0).stop 2024-04-20T10:28:13.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.331+0000 7f3a11c00640 1 -- 172.21.15.47:0/4267210303 shutdown_connections 2024-04-20T10:28:13.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.331+0000 7f3a11c00640 1 --2- 172.21.15.47:0/4267210303 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c0cda10 0x7f3a0c0cfe30 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.331+0000 7f3a11c00640 1 --2- 172.21.15.47:0/4267210303 >> v2:172.21.15.175:3300/0 conn(0x7f3a0c0d1350 0x7f3a0c0cd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.331+0000 7f3a11c00640 1 --2- 172.21.15.47:0/4267210303 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c15e630 0x7f3a0c0c8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.331+0000 7f3a11c00640 1 -- 172.21.15.47:0/4267210303 >> 172.21.15.47:0/4267210303 conn(0x7f3a0c0b3f40 msgr2=0x7f3a0c0b4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.335+0000 7f3a11c00640 1 -- 172.21.15.47:0/4267210303 shutdown_connections 2024-04-20T10:28:13.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.335+0000 7f3a11c00640 1 -- 172.21.15.47:0/4267210303 wait complete. 2024-04-20T10:28:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.337+0000 7f3a11c00640 1 Processor -- start 2024-04-20T10:28:13.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.337+0000 7f3a11c00640 1 -- start start 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.338+0000 7f3a11c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c0cda10 0x7f3a0c155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.338+0000 7f3a11c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a0c0d1350 0x7f3a0c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.338+0000 7f3a11c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c15e630 0x7f3a0c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.338+0000 7f3a11c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c0b5d20 con 0x7f3a0c15e630 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.338+0000 7f3a11c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0c0b5390 con 0x7f3a0c0d1350 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.338+0000 7f3a11c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3a0c0d0370 con 0x7f3a0c0cda10 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.338+0000 7f3a11200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c15e630 0x7f3a0c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c15e630 0x7f3a0c151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:57786/0 (socket says 172.21.15.47:57786) 2024-04-20T10:28:13.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 -- 172.21.15.47:0/3924392763 learned_addr learned my addr 172.21.15.47:0/3924392763 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a0be00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c0cda10 0x7f3a0c155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 -- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c0cda10 msgr2=0x7f3a0c155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c0cda10 0x7f3a0c155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 -- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a0c0d1350 msgr2=0x7f3a0c0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a0c0d1350 0x7f3a0c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 -- 172.21.15.47:0/3924392763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a0c15b850 con 0x7f3a0c15e630 2024-04-20T10:28:13.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a0be00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c0cda10 0x7f3a0c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.339+0000 7f3a11200640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c15e630 0x7f3a0c151960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a04036040 tx=0x7f3a04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.340+0000 7f3a08c00640 1 -- 172.21.15.47:0/3924392763 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a04043020 con 0x7f3a0c15e630 2024-04-20T10:28:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.340+0000 7f3a08c00640 1 -- 172.21.15.47:0/3924392763 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3a0403e450 con 0x7f3a0c15e630 2024-04-20T10:28:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.340+0000 7f3a08c00640 1 -- 172.21.15.47:0/3924392763 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a0403e840 con 0x7f3a0c15e630 2024-04-20T10:28:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.340+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a0c15baf0 con 0x7f3a0c15e630 2024-04-20T10:28:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.340+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a0c0ca870 con 0x7f3a0c15e630 2024-04-20T10:28:13.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.341+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a0c15bcc0 con 0x7f3a0c15e630 2024-04-20T10:28:13.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.342+0000 7f3a08c00640 1 -- 172.21.15.47:0/3924392763 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3a04046020 con 0x7f3a0c15e630 2024-04-20T10:28:13.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.343+0000 7f3a08c00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f39e806c8f0 0x7f39e806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:13.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.343+0000 7f3a08c00640 1 -- 172.21.15.47:0/3924392763 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6648+0+0 (secure 0 0 0) 0x7f3a040ae830 con 0x7f3a0c15e630 2024-04-20T10:28:13.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.345+0000 7f3a0be00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f39e806c8f0 0x7f39e806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:13.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.348+0000 7f3a0be00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f39e806c8f0 0x7f39e806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006fd0 tx=0x7f39fc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:13.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.348+0000 7f3a08c00640 1 -- 172.21.15.47:0/3924392763 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3a0404d030 con 0x7f3a0c15e630 2024-04-20T10:28:13.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.589+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3a0c075b30 con 0x7f3a0c15e630 2024-04-20T10:28:13.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.589+0000 7f3a08c00640 1 -- 172.21.15.47:0/3924392763 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7f3a0407c810 con 0x7f3a0c15e630 2024-04-20T10:28:13.590 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:28:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f39e806c8f0 msgr2=0x7f39e806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f39e806c8f0 0x7f39e806edb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006fd0 tx=0x7f39fc008040 comp rx=0 tx=0).stop 2024-04-20T10:28:13.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c15e630 msgr2=0x7f3a0c151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c15e630 0x7f3a0c151960 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3a04036040 tx=0x7f3a04004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 shutdown_connections 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f39e806c8f0 0x7f39e806edb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3a0c15e630 0x7f3a0c151960 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3a0c0d1350 0x7f3a0c0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 --2- 172.21.15.47:0/3924392763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3a0c0cda10 0x7f3a0c155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 >> 172.21.15.47:0/3924392763 conn(0x7f3a0c0b3f40 msgr2=0x7f3a0c0b4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 shutdown_connections 2024-04-20T10:28:13.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13.592+0000 7f3a11c00640 1 -- 172.21.15.47:0/3924392763 wait complete. 2024-04-20T10:28:13.610 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:28:13.610 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:28:13.610 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-20T10:28:13.611 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-20T10:28:13.611 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:13.611 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:28:13.611 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:28:13.611 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:28:13.611 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-20T10:28:13.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:13.611 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-20T10:28:13.660 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42789sec preferred_lft 42789sec 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:28:13.661 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::50dc:baff:fe1b:7c56/64 scope link 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::8014:63ff:fed5:b599/64 scope link tentative 2024-04-20T10:28:13.662 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:13.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:13.663 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:28:13.733 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-20T10:28:13.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:13.737 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-20T10:28:13.779 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-20T10:28:13.779 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:28:13.779 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:28:13.780 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:28:13.780 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:28:13.780 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:13.780 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:13.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:28:13.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:28:13.780 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:28:13.780 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:13.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:14.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:14.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:14.043 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:28:14.092 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:28:14.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:14.093 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list-id 2024-04-20T10:28:14.147 INFO:teuthology.orchestra.run.smithi047.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-20T10:28:14.148 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:14.148 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:14.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:28:14.148 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-20T10:28:14.148 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:14.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:14.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:14.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:14.350 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T10:28:14.418 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:14.418 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether a2:09:e2:8a:83:98 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::a009:e2ff:fe8a:8398/64 scope link 2024-04-20T10:28:14.419 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:14.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-20T10:28:14.502 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether a2:09:e2:8a:83:98 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::a009:e2ff:fe8a:8398/64 scope link 2024-04-20T10:28:14.503 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:14.507 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-20T10:28:14.508 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:14.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:14.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:14.799 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:14.799 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:28:14.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.1 up 2024-04-20T10:28:14.799 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T10:28:14.799 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:28:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:14.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:15.008 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2024-04-20T10:28:15.008 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:15.008 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:15.057 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-20T10:28:15.057 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:15.057 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:15.106 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:28:15.186 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:15.242 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:28:15.243 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:15.244 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:15.245 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:15.245 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:28:15.245 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:28:15.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:15.246 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:28:15.300 INFO:teuthology.orchestra.run.smithi047.stdout:42 2024-04-20T10:28:15.301 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [42] 2024-04-20T10:28:15.301 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2024-04-20T10:28:15.308 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:28:15.380 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:15.422 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:15.423 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:28:15.424 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:28:15.425 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:28:15.425 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:15.425 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:28:15.425 INFO:teuthology.orchestra.run.smithi047.stdout:/home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:15.425 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:28:15.425 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:28:15.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:15.426 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:28:15.482 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi047.stderr:2024-04-20T10:28:15.482+0000 7f7ae420d480 -1 init, newargv = 0x55ff37689d50 newargc=15 2024-04-20T10:28:15.482 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi047.stderr:ceph-fuse[31700]: starting ceph client 2024-04-20T10:28:15.495 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi047.stderr:ceph-fuse[31700]: starting fuse 2024-04-20T10:28:15.510 INFO:teuthology.orchestra.run.smithi047.stdout:42 2024-04-20T10:28:15.510 INFO:teuthology.orchestra.run.smithi047.stdout:43 2024-04-20T10:28:15.511 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42, 43] 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-04-20T10:28:15.511 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:28:15.512 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.2")) 2024-04-20T10:28:15.513 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:28:15.710 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.2.31700.asok 2024-04-20T10:28:15.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:15.728 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.31700.asok 2024-04-20T10:28:15.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:15.728 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.31700.asok status 2024-04-20T10:28:15.963 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:28:15.963 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-04-20T10:28:15.963 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "2", 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "31700", 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-04-20T10:28:15.964 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 4851, 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 4851 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3358582104 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:28:15.965 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3358582104 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.4851 192.168.0.2:0/3358582104", 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.2:0/3358582104", 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 9, 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 29, 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-04-20T10:28:15.966 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:28:15.978 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:28:16.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.192+0000 7fea35800640 1 Processor -- start 2024-04-20T10:28:16.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.192+0000 7fea35800640 1 -- start start 2024-04-20T10:28:16.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea35800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea30161d50 0x7fea300d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:16.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea35800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3006cde0 0x7fea300b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:16.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea35800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fea3006c190 con 0x7fea300b3330 2024-04-20T10:28:16.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea35800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fea30162130 con 0x7fea3006cde0 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea35800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fea3006c470 con 0x7fea30161d50 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea34e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3006cde0 0x7fea300b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea34e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3006cde0 0x7fea300b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50916/0 (socket says 172.21.15.47:50916) 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea34e00640 1 -- 172.21.15.47:0/4285757133 learned_addr learned my addr 172.21.15.47:0/4285757133 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.194+0000 7fea27e00640 1 --1- 172.21.15.47:0/4285757133 >> v1:172.21.15.47:6789/0 conn(0x7fea300b3330 0x7fea3006c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44856/0 (socket says 172.21.15.47:44856) 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.195+0000 7fea2fe00640 1 --2- 172.21.15.47:0/4285757133 >> v2:172.21.15.175:3300/0 conn(0x7fea30161d50 0x7fea300d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.195+0000 7fea34e00640 1 -- 172.21.15.47:0/4285757133 >> v2:172.21.15.175:3300/0 conn(0x7fea30161d50 msgr2=0x7fea300d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.195+0000 7fea34e00640 1 --2- 172.21.15.47:0/4285757133 >> v2:172.21.15.175:3300/0 conn(0x7fea30161d50 0x7fea300d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.195+0000 7fea34e00640 1 -- 172.21.15.47:0/4285757133 >> v1:172.21.15.47:6789/0 conn(0x7fea300b3330 legacy=0x7fea3006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:16.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.195+0000 7fea34e00640 1 -- 172.21.15.47:0/4285757133 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300d2e00 con 0x7fea3006cde0 2024-04-20T10:28:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.195+0000 7fea2f400640 1 -- 172.21.15.47:0/4285757133 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 67617853 0 0) 0x7fea3006c190 con 0x7fea300b3330 2024-04-20T10:28:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.196+0000 7fea34e00640 1 --2- 172.21.15.47:0/4285757133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3006cde0 0x7fea300b3880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea300b6f50 tx=0x7fea2802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ba57dd7ad3e53a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.196+0000 7fea2f400640 1 -- 172.21.15.47:0/4285757133 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2802cb20 con 0x7fea3006cde0 2024-04-20T10:28:16.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.196+0000 7fea2f400640 1 -- 172.21.15.47:0/4285757133 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea2802ccc0 con 0x7fea3006cde0 2024-04-20T10:28:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.196+0000 7fea2f400640 1 -- 172.21.15.47:0/4285757133 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28034610 con 0x7fea3006cde0 2024-04-20T10:28:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 -- 172.21.15.47:0/4285757133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3006cde0 msgr2=0x7fea300b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 --2- 172.21.15.47:0/4285757133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3006cde0 0x7fea300b3880 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fea300b6f50 tx=0x7fea2802bf80 comp rx=0 tx=0).stop 2024-04-20T10:28:16.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 -- 172.21.15.47:0/4285757133 shutdown_connections 2024-04-20T10:28:16.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 --2- 172.21.15.47:0/4285757133 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3006cde0 0x7fea300b3880 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 --2- 172.21.15.47:0/4285757133 >> v2:172.21.15.175:3300/0 conn(0x7fea30161d50 0x7fea300d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 -- 172.21.15.47:0/4285757133 >> 172.21.15.47:0/4285757133 conn(0x7fea300d4450 msgr2=0x7fea300d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:16.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 -- 172.21.15.47:0/4285757133 shutdown_connections 2024-04-20T10:28:16.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.197+0000 7fea35800640 1 -- 172.21.15.47:0/4285757133 wait complete. 2024-04-20T10:28:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.199+0000 7fea35800640 1 Processor -- start 2024-04-20T10:28:16.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.199+0000 7fea35800640 1 -- start start 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.200+0000 7fea35800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3006cde0 0x7fea3015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.200+0000 7fea35800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea300b3330 0x7fea300cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.200+0000 7fea35800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea30161d50 0x7fea3015d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.200+0000 7fea35800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fea300b7660 con 0x7fea300b3330 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.200+0000 7fea35800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea300d33c0 con 0x7fea3006cde0 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.200+0000 7fea35800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea300b77e0 con 0x7fea30161d50 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.200+0000 7fea27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea300b3330 0x7fea300cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea300b3330 0x7fea300cdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50918/0 (socket says 172.21.15.47:50918) 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 -- 172.21.15.47:0/87809785 learned_addr learned my addr 172.21.15.47:0/87809785 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:16.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea2fe00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3006cde0 0x7fea3015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea34e00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea30161d50 0x7fea3015d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 -- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea30161d50 msgr2=0x7fea3015d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea30161d50 0x7fea3015d160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 -- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3006cde0 msgr2=0x7fea3015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3006cde0 0x7fea3015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 -- 172.21.15.47:0/87809785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea300c82a0 con 0x7fea300b3330 2024-04-20T10:28:16.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea34e00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea30161d50 0x7fea3015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.201+0000 7fea27e00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea300b3330 0x7fea300cdb80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c4d0 tx=0x7fea1800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.202+0000 7fea2cc00640 1 -- 172.21.15.47:0/87809785 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18018070 con 0x7fea300b3330 2024-04-20T10:28:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.202+0000 7fea2cc00640 1 -- 172.21.15.47:0/87809785 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fea18007500 con 0x7fea300b3330 2024-04-20T10:28:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.202+0000 7fea2cc00640 1 -- 172.21.15.47:0/87809785 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea18004a20 con 0x7fea300b3330 2024-04-20T10:28:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.202+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea300ccdc0 con 0x7fea300b3330 2024-04-20T10:28:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.202+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea300cd380 con 0x7fea300b3330 2024-04-20T10:28:16.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.203+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea300cd870 con 0x7fea300b3330 2024-04-20T10:28:16.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.204+0000 7fea2cc00640 1 -- 172.21.15.47:0/87809785 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea18004bc0 con 0x7fea300b3330 2024-04-20T10:28:16.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.205+0000 7fea2cc00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea0c06c8f0 0x7fea0c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:16.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.205+0000 7fea2cc00640 1 -- 172.21.15.47:0/87809785 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fea1808b1b0 con 0x7fea300b3330 2024-04-20T10:28:16.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.206+0000 7fea2fe00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea0c06c8f0 0x7fea0c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:16.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.207+0000 7fea2fe00640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea0c06c8f0 0x7fea0c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea20006fd0 tx=0x7fea20008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:16.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.210+0000 7fea2cc00640 1 -- 172.21.15.47:0/87809785 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea18059220 con 0x7fea300b3330 2024-04-20T10:28:16.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.455+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fea30075b30 con 0x7fea300b3330 2024-04-20T10:28:16.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.456+0000 7fea2cc00640 1 -- 172.21.15.47:0/87809785 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fea18058b20 con 0x7fea300b3330 2024-04-20T10:28:16.456 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:28:16.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea0c06c8f0 msgr2=0x7fea0c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea0c06c8f0 0x7fea0c06edb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fea20006fd0 tx=0x7fea20008040 comp rx=0 tx=0).stop 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea300b3330 msgr2=0x7fea300cdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea300b3330 0x7fea300cdb80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fea1800c4d0 tx=0x7fea1800c990 comp rx=0 tx=0).stop 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 shutdown_connections 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea30161d50 0x7fea3015d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea0c06c8f0 0x7fea0c06edb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea300b3330 0x7fea300cdb80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 --2- 172.21.15.47:0/87809785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3006cde0 0x7fea3015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.458+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 >> 172.21.15.47:0/87809785 conn(0x7fea300d4450 msgr2=0x7fea30160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.459+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 shutdown_connections 2024-04-20T10:28:16.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:16.459+0000 7fea35800640 1 -- 172.21.15.47:0/87809785 wait complete. 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:28:16.478 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:28:16.479 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-20T10:28:16.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:16.479 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T10:28:16.535 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42759sec preferred_lft 42759sec 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T10:28:16.536 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:16.537 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-20T10:28:16.537 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:16.537 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:16.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add name ceph-brx type bridge 2024-04-20T10:28:16.537 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr flush dev ceph-brx 2024-04-20T10:28:16.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set ceph-brx up 2024-04-20T10:28:16.538 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-20T10:28:16.538 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:16.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:16 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:17.004 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:17.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:17.014 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:28:17.083 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T10:28:17.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:17.087 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T10:28:17.137 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T10:28:17.137 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:28:17.137 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:28:17.137 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:28:17.137 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:28:17.138 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:17.138 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:17.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:28:17.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:28:17.138 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:28:17.138 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:17.232 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:17.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:17.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:17.876 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T10:28:17.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:17.927 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-04-20T10:28:17.975 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:17.975 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:17.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:17.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-20T10:28:17.975 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:18.077 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:18.152 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:18.161 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-20T10:28:18.162 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:18.268 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:18.456 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:18.466 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:18.466 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:18.466 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-04-20T10:28:18.466 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.0 master ceph-brx 2024-04-20T10:28:18.466 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:18.561 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:18.658 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:18.669 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-04-20T10:28:18.669 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:18.669 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:18.713 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-20T10:28:18.713 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:18.713 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:18.805 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T10:28:18.878 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:18.929 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T10:28:18.930 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T10:28:18.931 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:28:18.932 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:28:18.932 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:28:18.932 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:28:18.932 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T10:28:18.932 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:18.932 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:18.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:18.933 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T10:28:18.979 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:28:18.979 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-04-20T10:28:19.026 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T10:28:19.098 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T10:28:19.145 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T10:28:19.146 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:28:19.147 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:28:19.147 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:28:19.147 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:28:19.147 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T10:28:19.147 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:19.147 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:19.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:19.148 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T10:28:19.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:2024-04-20T10:28:19.219+0000 7f14a8a39480 -1 init, newargv = 0x5620ecb73fc0 newargc=15 2024-04-20T10:28:19.221 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[28198]: starting ceph client 2024-04-20T10:28:19.230 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi175.stderr:ceph-fuse[28198]: starting fuse 2024-04-20T10:28:19.234 INFO:teuthology.orchestra.run.smithi175.stdout:41 2024-04-20T10:28:19.234 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T10:28:19.235 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.1")) 2024-04-20T10:28:19.236 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T10:28:19.419 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.1.28198.asok 2024-04-20T10:28:19.424 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:19.436 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.28198.asok 2024-04-20T10:28:19.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:19.437 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.28198.asok status 2024-04-20T10:28:19.663 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "1", 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "28198", 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 4553, 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T10:28:19.664 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 4553 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 207317676 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 207317676 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.4553 192.168.0.1:0/207317676", 2024-04-20T10:28:19.665 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.1:0/207317676", 2024-04-20T10:28:19.666 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T10:28:19.666 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 9, 2024-04-20T10:28:19.666 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 29, 2024-04-20T10:28:19.666 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:28:19.666 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T10:28:19.666 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T10:28:19.666 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T10:28:19.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:28:19.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.899+0000 7fd5de000640 1 Processor -- start 2024-04-20T10:28:19.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.899+0000 7fd5de000640 1 -- start start 2024-04-20T10:28:19.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.901+0000 7fd5de000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd5d8072190 0x7fd5d80735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.901+0000 7fd5de000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d815cc90 con 0x7fd5d815a1f0 2024-04-20T10:28:19.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.901+0000 7fd5de000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd5d815cf70 con 0x7fd5d8159e10 2024-04-20T10:28:19.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.901+0000 7fd5de000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd5d8072900 con 0x7fd5d8072190 2024-04-20T10:28:19.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.901+0000 7fd5d7e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd5d815a1f0 0x7fd5d8072ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44866/0 (socket says 172.21.15.47:44866) 2024-04-20T10:28:19.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.901+0000 7fd5d7e00640 1 -- 172.21.15.47:0/2655768888 learned_addr learned my addr 172.21.15.47:0/2655768888 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5dd600640 1 --2- 172.21.15.47:0/2655768888 >> v2:172.21.15.175:3300/0 conn(0x7fd5d8072190 0x7fd5d80735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3162411416 0 0) 0x7fd5d815cc90 con 0x7fd5d815a1f0 2024-04-20T10:28:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd5bc003520 con 0x7fd5d815a1f0 2024-04-20T10:28:19.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2507241070 0 0) 0x7fd5bc003520 con 0x7fd5d815a1f0 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 >> v2:172.21.15.175:3300/0 conn(0x7fd5d8072190 msgr2=0x7fd5d80735f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 --2- 172.21.15.47:0/2655768888 >> v2:172.21.15.175:3300/0 conn(0x7fd5d8072190 0x7fd5d80735f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 >> v1:172.21.15.175:6789/0 conn(0x7fd5d8159e10 legacy=0x7fd5d8071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d815d250 con 0x7fd5d815a1f0 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fd5c4001670 con 0x7fd5d815a1f0 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 >> v1:172.21.15.47:6789/0 conn(0x7fd5d815a1f0 legacy=0x7fd5d8072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc003a00 0x7fd5bc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc004800 0x7fd5bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0051c0 0x7fd5bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc00c090 con 0x7fd5bc003a00 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc00c240 con 0x7fd5bc0051c0 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.902+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc00c440 con 0x7fd5bc004800 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc003a00 0x7fd5bc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc004800 0x7fd5bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dd600640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0051c0 0x7fd5bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dcc00640 1 -- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc004800 msgr2=0x7fd5bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc004800 0x7fd5bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dcc00640 1 -- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0051c0 msgr2=0x7fd5bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0051c0 0x7fd5bc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dcc00640 1 -- 172.21.15.47:0/2655768888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc003520 con 0x7fd5bc003a00 2024-04-20T10:28:19.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.903+0000 7fd5dd600640 1 -- 172.21.15.47:0/2655768888 reap_dead start 2024-04-20T10:28:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.904+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc003a00 0x7fd5bc003e20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8001e50 tx=0x7fd5bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a8ad3dbb68b2e79 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c8001a70 con 0x7fd5bc003a00 2024-04-20T10:28:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5c8003b70 con 0x7fd5bc003a00 2024-04-20T10:28:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5d7400640 1 -- 172.21.15.47:0/2655768888 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c80050d0 con 0x7fd5bc003a00 2024-04-20T10:28:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5de000640 1 -- 172.21.15.47:0/2655768888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc003a00 msgr2=0x7fd5bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5de000640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc003a00 0x7fd5bc003e20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8001e50 tx=0x7fd5bc005830 comp rx=0 tx=0).stop 2024-04-20T10:28:19.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5de000640 1 -- 172.21.15.47:0/2655768888 shutdown_connections 2024-04-20T10:28:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5de000640 1 --2- 172.21.15.47:0/2655768888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5bc003a00 0x7fd5bc003e20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5de000640 1 -- 172.21.15.47:0/2655768888 >> 172.21.15.47:0/2655768888 conn(0x7fd5d80d0040 msgr2=0x7fd5d80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.905+0000 7fd5de000640 1 -- 172.21.15.47:0/2655768888 shutdown_connections 2024-04-20T10:28:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.906+0000 7fd5de000640 1 -- 172.21.15.47:0/2655768888 wait complete. 2024-04-20T10:28:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.907+0000 7fd5de000640 1 Processor -- start 2024-04-20T10:28:19.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.908+0000 7fd5de000640 1 -- start start 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5de000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc003a00 0x7fd5d8158720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5de000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5d815b200 0x7fd5d815b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5de000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8157400 0x7fd5d8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5de000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d815d810 con 0x7fd5d8157400 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5de000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d815ca40 con 0x7fd5d815b200 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5de000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5d806da90 con 0x7fd5bc003a00 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8157400 0x7fd5d8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8157400 0x7fd5d8157840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50930/0 (socket says 172.21.15.47:50930) 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 -- 172.21.15.47:0/2203251966 learned_addr learned my addr 172.21.15.47:0/2203251966 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:19.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc003a00 0x7fd5d8158720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5d7e00640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5d815b200 0x7fd5d815b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 -- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc003a00 msgr2=0x7fd5d8158720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc003a00 0x7fd5d8158720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 -- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5d815b200 msgr2=0x7fd5d815b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5d815b200 0x7fd5d815b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.909+0000 7fd5dd600640 1 -- 172.21.15.47:0/2203251966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d815bfc0 con 0x7fd5d8157400 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.910+0000 7fd5dd600640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8157400 0x7fd5d8157840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5d806ec80 tx=0x7fd5d0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.910+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2203251966 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0003610 con 0x7fd5d8157400 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.910+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2203251966 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd5d001b070 con 0x7fd5d8157400 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.910+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2203251966 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d0003a10 con 0x7fd5d8157400 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.910+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d815c2a0 con 0x7fd5d8157400 2024-04-20T10:28:19.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.910+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d80b2e60 con 0x7fd5d8157400 2024-04-20T10:28:19.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.911+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d8075b60 con 0x7fd5d8157400 2024-04-20T10:28:19.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.912+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2203251966 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd5d0003bb0 con 0x7fd5d8157400 2024-04-20T10:28:19.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.913+0000 7fd5d4c00640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5b806c920 0x7fd5b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:19.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.913+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2203251966 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fd5d008c320 con 0x7fd5d8157400 2024-04-20T10:28:19.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.914+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5b806c920 0x7fd5b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:19.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.915+0000 7fd5dcc00640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5b806c920 0x7fd5b806ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8034000 tx=0x7fd5c8009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:19.918+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2203251966 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd5d001d310 con 0x7fd5d8157400 2024-04-20T10:28:20.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.165+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd5d80b1740 con 0x7fd5d8157400 2024-04-20T10:28:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.166+0000 7fd5d4c00640 1 -- 172.21.15.47:0/2203251966 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) ==== 53+0+88 (secure 0 0 0) 0x7fd5d005a300 con 0x7fd5d8157400 2024-04-20T10:28:20.166 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5b806c920 msgr2=0x7fd5b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5b806c920 0x7fd5b806ede0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd5c8034000 tx=0x7fd5c8009030 comp rx=0 tx=0).stop 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8157400 msgr2=0x7fd5d8157840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8157400 0x7fd5d8157840 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd5d806ec80 tx=0x7fd5d0011920 comp rx=0 tx=0).stop 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 shutdown_connections 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd5b806c920 0x7fd5b806ede0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd5d8157400 0x7fd5d8157840 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5d815b200 0x7fd5d815b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 --2- 172.21.15.47:0/2203251966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc003a00 0x7fd5d8158720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 >> 172.21.15.47:0/2203251966 conn(0x7fd5d80d0040 msgr2=0x7fd5d8156c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.169+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 shutdown_connections 2024-04-20T10:28:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:20.170+0000 7fd5de000640 1 -- 172.21.15.47:0/2203251966 wait complete. 2024-04-20T10:28:20.190 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:28:20.190 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:28:20.190 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-04-20T10:28:20.190 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-20T10:28:20.190 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:20.190 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:28:20.191 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:28:20.191 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:28:20.191 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-20T10:28:20.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:20.191 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-04-20T10:28:20.248 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:20.248 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-20T10:28:20.249 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42755sec preferred_lft 42755sec 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether ee:1f:5e:26:b8:6a brd ff:ff:ff:ff:ff:ff 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::fce6:b8ff:fe12:df9c/64 scope link 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether ee:1f:5e:26:b8:6a brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec1f:5eff:fe26:b86a/64 scope link 2024-04-20T10:28:20.250 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:20.251 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:28:20.330 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-04-20T10:28:20.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:20.334 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-04-20T10:28:20.385 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-04-20T10:28:20.386 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:28:20.386 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-20T10:28:20.386 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-20T10:28:20.386 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:28:20.386 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:20.386 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:20.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-20T10:28:20.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-20T10:28:20.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-20T10:28:20.387 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:20.487 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:20.582 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:20.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:20.591 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T10:28:20.638 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T10:28:20.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:20.639 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-04-20T10:28:20.733 INFO:teuthology.orchestra.run.smithi175.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-20T10:28:20.734 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:20.734 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:20.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:28:20.734 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-20T10:28:20.734 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:20.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:20 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:20.904 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:20 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:20.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:20.914 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T10:28:20.984 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:20.984 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:20.984 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:20.984 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.984 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:20.984 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.984 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:28:20.985 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 66:54:b4:a6:81:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:28:20.985 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:28:20.985 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.985 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::6454:b4ff:fea6:81b5/64 scope link 2024-04-20T10:28:20.985 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:20.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:20.988 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 66:54:b4:a6:81:b5 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-20T10:28:21.071 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-20T10:28:21.072 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:21.072 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::6454:b4ff:fea6:81b5/64 scope link 2024-04-20T10:28:21.072 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:28:21.075 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-20T10:28:21.075 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:21.075 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:21.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-20T10:28:21.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-20T10:28:21.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-20T10:28:21.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-20T10:28:21.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-20T10:28:21.076 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:21.176 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:21.339 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:21.349 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:28:21.349 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-04-20T10:28:21.349 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.1 up 2024-04-20T10:28:21.350 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.1 master ceph-brx 2024-04-20T10:28:21.350 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-04-20T10:28:21.452 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:21 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:28:21.551 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:21 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:21.560 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2024-04-20T10:28:21.560 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:21.560 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:21.605 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-20T10:28:21.606 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:28:21.606 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:21.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T10:28:21.783 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T10:28:21.840 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T10:28:21.841 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:28:21.842 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:28:21.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:21.843 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T10:28:21.895 INFO:teuthology.orchestra.run.smithi175.stdout:41 2024-04-20T10:28:21.895 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2024-04-20T10:28:21.895 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2024-04-20T10:28:21.900 DEBUG:teuthology.orchestra.run.smithi175:> sudo modprobe fuse 2024-04-20T10:28:21.973 DEBUG:teuthology.orchestra.run.smithi175:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/proc 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/sys 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/dev 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/security 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/shm 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/pts 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/run 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/cgroup 2024-04-20T10:28:22.015 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/pstore 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/bpf 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/ 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/selinux 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/mqueue 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/dev/hugepages 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/debug 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/tracing 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/fs/fuse/connections 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/sys/kernel/config 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:28:22.016 INFO:teuthology.orchestra.run.smithi175.stdout:/run/user/1000 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-4 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-5 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-6 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/var/lib/ceph/osd/ceph-7 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:28:22.017 INFO:teuthology.orchestra.run.smithi175.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:28:22.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.018 DEBUG:teuthology.orchestra.run.smithi175:> ls /sys/fs/fuse/connections 2024-04-20T10:28:22.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:2024-04-20T10:28:22.069+0000 7f2db46d3480 -1 init, newargv = 0x55aa780f2c50 newargc=15 2024-04-20T10:28:22.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:ceph-fuse[28654]: starting ceph client 2024-04-20T10:28:22.081 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi175.stderr:ceph-fuse[28654]: starting fuse 2024-04-20T10:28:22.103 INFO:teuthology.orchestra.run.smithi175.stdout:41 2024-04-20T10:28:22.103 INFO:teuthology.orchestra.run.smithi175.stdout:43 2024-04-20T10:28:22.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 43] 2024-04-20T10:28:22.103 DEBUG:teuthology.orchestra.run.smithi175:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> import glob 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> import re 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> import os 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> import subprocess 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> def _find_admin_socket(client_name): 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> files = glob.glob(asok_path) 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> # Given a non-glob path, it better be there 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> if "*" not in asok_path: 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> assert(len(files) == 1) 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> return files[0] 2024-04-20T10:28:22.104 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> for f in files: 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> contents = proc_f.read() 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> if mountpoint in contents: 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> return f 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> print(_find_admin_socket("client.3")) 2024-04-20T10:28:22.105 DEBUG:teuthology.orchestra.run.smithi175:> ' 2024-04-20T10:28:22.294 INFO:teuthology.orchestra.run.smithi175.stdout:/var/run/ceph/ceph-client.3.28654.asok 2024-04-20T10:28:22.298 INFO:teuthology.orchestra.run.smithi175.stderr:2024-04-20T10:28:22 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:28:22.311 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.28654.asok 2024-04-20T10:28:22.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.311 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.28654.asok status 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout:{ 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "metadata": { 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "entity_id": "3", 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "hostname": "smithi175", 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "pid": "28654", 2024-04-20T10:28:22.536 INFO:teuthology.orchestra.run.smithi175.stdout: "root": "/" 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_count": 0, 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "dentry_pinned_count": 0, 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "id": 4556, 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "inst": { 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "name": { 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "client", 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "num": 4556 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2844259737 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: } 2024-04-20T10:28:22.537 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": { 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "type": "v1", 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "addr": "192.168.0.2:0", 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "nonce": 2844259737 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: }, 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "inst_str": "client.4556 192.168.0.2:0/2844259737", 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "addr_str": "192.168.0.2:0/2844259737", 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "inode_count": 1, 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "mds_epoch": 9, 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch": 29, 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "blocklisted": false, 2024-04-20T10:28:22.538 INFO:teuthology.orchestra.run.smithi175.stdout: "fs_name": "cephfs" 2024-04-20T10:28:22.539 INFO:teuthology.orchestra.run.smithi175.stdout:} 2024-04-20T10:28:22.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.551 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-04-20T10:28:22.552 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:22.614 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-04-20T10:28:22.614 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:22.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.615 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:22.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.706 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-04-20T10:28:22.706 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:22.765 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-04-20T10:28:22.765 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:22.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.765 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:22.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.853 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T10:28:22.853 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:22.911 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T10:28:22.911 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:22.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.911 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:22.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:22.988 DEBUG:teuthology.orchestra.run.smithi175:> stat --file-system '--printf=%T 2024-04-20T10:28:22.988 DEBUG:teuthology.orchestra.run.smithi175:> ' -- /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:23.041 INFO:teuthology.orchestra.run.smithi175.stdout:fuseblk 2024-04-20T10:28:23.041 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:23.041 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:23.041 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:23.113 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-20T10:28:23.131 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.340+0000 7f0b58c00640 1 Processor -- start 2024-04-20T10:28:23.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.340+0000 7f0b58c00640 1 -- start start 2024-04-20T10:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.342+0000 7f0b58c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b54161d50 0x7f0b540d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.342+0000 7f0b58c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 0x7f0b540b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.342+0000 7f0b58c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b5406c190 con 0x7f0b540b3330 2024-04-20T10:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.342+0000 7f0b58c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0b54162130 con 0x7f0b54161d50 2024-04-20T10:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.342+0000 7f0b58c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b5406c470 con 0x7f0b5406cde0 2024-04-20T10:28:23.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.342+0000 7f0b52a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0b540b3330 0x7f0b5406c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44876/0 (socket says 172.21.15.47:44876) 2024-04-20T10:28:23.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.342+0000 7f0b52a00640 1 -- 172.21.15.47:0/2654499563 learned_addr learned my addr 172.21.15.47:0/2654499563 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b53e00640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 0x7f0b540b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b53400640 1 --2- 172.21.15.47:0/2654499563 >> v2:172.21.15.175:3300/0 conn(0x7f0b54161d50 0x7f0b540d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3164696927 0 0) 0x7f0b5406c190 con 0x7f0b540b3330 2024-04-20T10:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0b38003520 con 0x7f0b540b3330 2024-04-20T10:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3737136446 0 0) 0x7f0b38003520 con 0x7f0b540b3330 2024-04-20T10:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 msgr2=0x7f0b540b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:23.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 0x7f0b540b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 >> v2:172.21.15.175:3300/0 conn(0x7f0b54161d50 msgr2=0x7f0b540d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 --2- 172.21.15.47:0/2654499563 >> v2:172.21.15.175:3300/0 conn(0x7f0b54161d50 0x7f0b540d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540d2e00 con 0x7f0b540b3330 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f0b40001690 con 0x7f0b540b3330 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 >> v1:172.21.15.47:6789/0 conn(0x7f0b540b3330 legacy=0x7f0b5406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b38003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38004800 0x7f0b38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 0x7f0b380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5406c470 con 0x7f0b380039b0 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3800baa0 con 0x7f0b38004800 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.343+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3800bca0 con 0x7f0b5406cde0 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b53400640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b38003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b53e00640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 0x7f0b380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b53400640 1 -- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 msgr2=0x7f0b380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b53400640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b5406cde0 0x7f0b380051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b52a00640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38004800 0x7f0b38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b53400640 1 -- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38004800 msgr2=0x7f0b38004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b53400640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38004800 0x7f0b38004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b53400640 1 -- 172.21.15.47:0/2654499563 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b38003520 con 0x7f0b380039b0 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.344+0000 7f0b52a00640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b38004800 0x7f0b38004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:23.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.345+0000 7f0b53400640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b38003dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400ba00 tx=0x7f0b38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33961dcabadeadc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44036020 con 0x7f0b380039b0 2024-04-20T10:28:23.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b44032630 con 0x7f0b380039b0 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b52000640 1 -- 172.21.15.47:0/2654499563 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400b600 con 0x7f0b380039b0 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b58c00640 1 -- 172.21.15.47:0/2654499563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 msgr2=0x7f0b38003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b58c00640 1 --2- 172.21.15.47:0/2654499563 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b38003dd0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0b4400ba00 tx=0x7f0b38005830 comp rx=0 tx=0).stop 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b53e00640 1 -- 172.21.15.47:0/2654499563 reap_dead start 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b58c00640 1 -- 172.21.15.47:0/2654499563 shutdown_connections 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b58c00640 1 -- 172.21.15.47:0/2654499563 >> 172.21.15.47:0/2654499563 conn(0x7f0b540d4450 msgr2=0x7f0b540d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b58c00640 1 -- 172.21.15.47:0/2654499563 shutdown_connections 2024-04-20T10:28:23.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.346+0000 7f0b58c00640 1 -- 172.21.15.47:0/2654499563 wait complete. 2024-04-20T10:28:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.349+0000 7f0b58c00640 1 Processor -- start 2024-04-20T10:28:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.349+0000 7f0b58c00640 1 -- start start 2024-04-20T10:28:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.350+0000 7f0b58c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b541f40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.350+0000 7f0b58c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540cb520 0x7f0b540cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.350+0000 7f0b58c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b7bb0 0x7f0b540b8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.350+0000 7f0b58c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540b3880 con 0x7f0b380039b0 2024-04-20T10:28:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.350+0000 7f0b58c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b540d33c0 con 0x7f0b540cb520 2024-04-20T10:28:23.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.350+0000 7f0b58c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b540b3a00 con 0x7f0b540b7bb0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b541f40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b541f40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50938/0 (socket says 172.21.15.47:50938) 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 -- 172.21.15.47:0/548749426 learned_addr learned my addr 172.21.15.47:0/548749426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53e00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b7bb0 0x7f0b540b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 -- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b7bb0 msgr2=0x7f0b540b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b7bb0 0x7f0b540b8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 -- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540cb520 msgr2=0x7f0b540cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b52a00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540cb520 0x7f0b540cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540cb520 0x7f0b540cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.351+0000 7f0b53400640 1 -- 172.21.15.47:0/548749426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b540b8920 con 0x7f0b380039b0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.352+0000 7f0b53400640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b541f40f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b44035040 tx=0x7f0b44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.352+0000 7f0b4b400640 1 -- 172.21.15.47:0/548749426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44041070 con 0x7f0b380039b0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.352+0000 7f0b4b400640 1 -- 172.21.15.47:0/548749426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b44030a40 con 0x7f0b380039b0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.352+0000 7f0b4b400640 1 -- 172.21.15.47:0/548749426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b44036040 con 0x7f0b380039b0 2024-04-20T10:28:23.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.352+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b540b8c00 con 0x7f0b380039b0 2024-04-20T10:28:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.352+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b540b91c0 con 0x7f0b380039b0 2024-04-20T10:28:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.353+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5406c6b0 con 0x7f0b380039b0 2024-04-20T10:28:23.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.355+0000 7f0b4b400640 1 -- 172.21.15.47:0/548749426 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b44039020 con 0x7f0b380039b0 2024-04-20T10:28:23.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.355+0000 7f0b4b400640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b3406c920 0x7f0b3406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.356+0000 7f0b4b400640 1 -- 172.21.15.47:0/548749426 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f0b440ae0d0 con 0x7f0b380039b0 2024-04-20T10:28:23.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.356+0000 7f0b52a00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b3406c920 0x7f0b3406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.357+0000 7f0b52a00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b3406c920 0x7f0b3406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b540ce720 tx=0x7f0b40009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:23.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.360+0000 7f0b4b400640 1 -- 172.21.15.47:0/548749426 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0b5406c6b0 con 0x7f0b380039b0 2024-04-20T10:28:23.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.567+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b540b5ba0 con 0x7f0b380039b0 2024-04-20T10:28:23.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.569+0000 7f0b4b400640 1 -- 172.21.15.47:0/548749426 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f0b4407c0b0 con 0x7f0b380039b0 2024-04-20T10:28:23.570 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:23.570 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.571+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b3406c920 msgr2=0x7f0b3406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.571+0000 7f0b58c00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b3406c920 0x7f0b3406ede0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0b540ce720 tx=0x7f0b40009040 comp rx=0 tx=0).stop 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.571+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 msgr2=0x7f0b541f40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.571+0000 7f0b58c00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b541f40f0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0b44035040 tx=0x7f0b44004730 comp rx=0 tx=0).stop 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 shutdown_connections 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b540b7bb0 0x7f0b540b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b540cb520 0x7f0b540cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b3406c920 0x7f0b3406ede0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 --2- 172.21.15.47:0/548749426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b380039b0 0x7f0b541f40f0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 >> 172.21.15.47:0/548749426 conn(0x7f0b540d4450 msgr2=0x7f0b5415f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 shutdown_connections 2024-04-20T10:28:23.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.572+0000 7f0b58c00640 1 -- 172.21.15.47:0/548749426 wait complete. 2024-04-20T10:28:23.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:23.592 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.781+0000 7f0d69e00640 1 Processor -- start 2024-04-20T10:28:23.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.781+0000 7f0d69e00640 1 -- start start 2024-04-20T10:28:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.783+0000 7f0d69e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0d640d0380 0x7f0d640d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.783+0000 7f0d69e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d640d1e90 con 0x7f0d640d1a00 2024-04-20T10:28:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.783+0000 7f0d69e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d640d0d10 con 0x7f0d640d4270 2024-04-20T10:28:23.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.783+0000 7f0d69e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0d640d40c0 con 0x7f0d640d0380 2024-04-20T10:28:23.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.783+0000 7f0d62000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0d640d1a00 0x7f0d640b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44880/0 (socket says 172.21.15.47:44880) 2024-04-20T10:28:23.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.783+0000 7f0d62000640 1 -- 172.21.15.47:0/402888019 learned_addr learned my addr 172.21.15.47:0/402888019 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d63400640 1 --2- 172.21.15.47:0/402888019 >> v2:172.21.15.175:3300/0 conn(0x7f0d640d0380 0x7f0d640d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3710150113 0 0) 0x7f0d4c001490 con 0x7f0d640d1a00 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d44003520 con 0x7f0d640d1a00 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 766207960 0 0) 0x7f0d44003520 con 0x7f0d640d1a00 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 >> v2:172.21.15.175:3300/0 conn(0x7f0d640d0380 msgr2=0x7f0d640d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 --2- 172.21.15.47:0/402888019 >> v2:172.21.15.175:3300/0 conn(0x7f0d640d0380 0x7f0d640d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 >> v1:172.21.15.175:6789/0 conn(0x7f0d640d4270 legacy=0x7f0d640d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d640d3820 con 0x7f0d640d1a00 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.784+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f0d4c001bc0 con 0x7f0d640d1a00 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 >> v1:172.21.15.47:6789/0 conn(0x7f0d640d1a00 legacy=0x7f0d640b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d61600640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d44003a00 0x7f0d44003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d61600640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44004800 0x7f0d44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d61600640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d440051c0 0x7f0d4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0d640d40c0 con 0x7f0d44004800 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4400c0c0 con 0x7f0d44003a00 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d4400c2c0 con 0x7f0d440051c0 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d62000640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44004800 0x7f0d44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d62a00640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d44003a00 0x7f0d44003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.785+0000 7f0d63400640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d440051c0 0x7f0d4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.786+0000 7f0d62000640 1 -- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d440051c0 msgr2=0x7f0d4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.786+0000 7f0d62000640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d440051c0 0x7f0d4400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.786+0000 7f0d62000640 1 -- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d44003a00 msgr2=0x7f0d44003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.786+0000 7f0d62000640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d44003a00 0x7f0d44003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.786+0000 7f0d62000640 1 -- 172.21.15.47:0/402888019 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d44003520 con 0x7f0d44004800 2024-04-20T10:28:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.786+0000 7f0d62a00640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d44003a00 0x7f0d44003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:23.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.786+0000 7f0d63400640 1 -- 172.21.15.47:0/402888019 reap_dead start 2024-04-20T10:28:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.787+0000 7f0d62000640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44004800 0x7f0d44004c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00a950 tx=0x7f0d44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=798758d36395ac8a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.787+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00a5a0 con 0x7f0d44004800 2024-04-20T10:28:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.787+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4c034650 con 0x7f0d44004800 2024-04-20T10:28:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.787+0000 7f0d61600640 1 -- 172.21.15.47:0/402888019 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c034a80 con 0x7f0d44004800 2024-04-20T10:28:23.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.788+0000 7f0d69e00640 1 -- 172.21.15.47:0/402888019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44004800 msgr2=0x7f0d44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.788+0000 7f0d69e00640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44004800 0x7f0d44004c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c00a950 tx=0x7f0d44005830 comp rx=0 tx=0).stop 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.788+0000 7f0d69e00640 1 -- 172.21.15.47:0/402888019 shutdown_connections 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.788+0000 7f0d69e00640 1 --2- 172.21.15.47:0/402888019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d44004800 0x7f0d44004c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.788+0000 7f0d69e00640 1 -- 172.21.15.47:0/402888019 >> 172.21.15.47:0/402888019 conn(0x7f0d640b99c0 msgr2=0x7f0d640b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.788+0000 7f0d69e00640 1 -- 172.21.15.47:0/402888019 shutdown_connections 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.788+0000 7f0d69e00640 1 -- 172.21.15.47:0/402888019 wait complete. 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.790+0000 7f0d69e00640 1 Processor -- start 2024-04-20T10:28:23.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.790+0000 7f0d69e00640 1 -- start start 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.791+0000 7f0d69e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d44004800 0x7f0d6415a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.791+0000 7f0d69e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d640b1870 0x7f0d640b1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.791+0000 7f0d69e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d6406ba00 0x7f0d6406be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.791+0000 7f0d69e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0d640cfe60 con 0x7f0d6406ba00 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.791+0000 7f0d69e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d640d3de0 con 0x7f0d640b1870 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.791+0000 7f0d69e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d640d53b0 con 0x7f0d44004800 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d6406ba00 0x7f0d6406be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d6406ba00 0x7f0d6406be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:50954/0 (socket says 172.21.15.47:50954) 2024-04-20T10:28:23.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 -- 172.21.15.47:0/1346444455 learned_addr learned my addr 172.21.15.47:0/1346444455 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:23.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d62a00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d44004800 0x7f0d6415a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d62000640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d640b1870 0x7f0d640b1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 -- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d44004800 msgr2=0x7f0d6415a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d44004800 0x7f0d6415a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 -- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d640b1870 msgr2=0x7f0d640b1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d640b1870 0x7f0d640b1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d63400640 1 -- 172.21.15.47:0/1346444455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6406c710 con 0x7f0d6406ba00 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.792+0000 7f0d62000640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d640b1870 0x7f0d640b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.793+0000 7f0d63400640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d6406ba00 0x7f0d6406be80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c011490 tx=0x7f0d5c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.793+0000 7f0d5b400640 1 -- 172.21.15.47:0/1346444455 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c004440 con 0x7f0d6406ba00 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.793+0000 7f0d5b400640 1 -- 172.21.15.47:0/1346444455 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d5c01b070 con 0x7f0d6406ba00 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.793+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6406c9f0 con 0x7f0d6406ba00 2024-04-20T10:28:23.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.793+0000 7f0d5b400640 1 -- 172.21.15.47:0/1346444455 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c003850 con 0x7f0d6406ba00 2024-04-20T10:28:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.793+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d6406c3c0 con 0x7f0d6406ba00 2024-04-20T10:28:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.795+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d640cec30 con 0x7f0d6406ba00 2024-04-20T10:28:23.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.796+0000 7f0d5b400640 1 -- 172.21.15.47:0/1346444455 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0d5c0039f0 con 0x7f0d6406ba00 2024-04-20T10:28:23.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.797+0000 7f0d5b400640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0d4006c8f0 0x7f0d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:23.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.797+0000 7f0d5b400640 1 -- 172.21.15.47:0/1346444455 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f0d5c08c3f0 con 0x7f0d6406ba00 2024-04-20T10:28:23.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.797+0000 7f0d62a00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0d4006c8f0 0x7f0d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:23.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.799+0000 7f0d62a00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0d4006c8f0 0x7f0d4006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d640caf90 tx=0x7f0d54009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:23.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:23.801+0000 7f0d5b400640 1 -- 172.21.15.47:0/1346444455 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0d640cec30 con 0x7f0d6406ba00 2024-04-20T10:28:24.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.014+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0d640b2230 con 0x7f0d6406ba00 2024-04-20T10:28:24.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.015+0000 7f0d5b400640 1 -- 172.21.15.47:0/1346444455 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f0d5c05a470 con 0x7f0d6406ba00 2024-04-20T10:28:24.015 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.018+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0d4006c8f0 msgr2=0x7f0d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.018+0000 7f0d69e00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0d4006c8f0 0x7f0d4006edb0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0d640caf90 tx=0x7f0d54009040 comp rx=0 tx=0).stop 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.018+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d6406ba00 msgr2=0x7f0d6406be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.018+0000 7f0d69e00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d6406ba00 0x7f0d6406be80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c011490 tx=0x7f0d5c009f90 comp rx=0 tx=0).stop 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 shutdown_connections 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0d4006c8f0 0x7f0d4006edb0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0d6406ba00 0x7f0d6406be80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d640b1870 0x7f0d640b1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 --2- 172.21.15.47:0/1346444455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d44004800 0x7f0d6415a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 >> 172.21.15.47:0/1346444455 conn(0x7f0d640b99c0 msgr2=0x7f0d640b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:24.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 shutdown_connections 2024-04-20T10:28:24.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.019+0000 7f0d69e00640 1 -- 172.21.15.47:0/1346444455 wait complete. 2024-04-20T10:28:24.040 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.246+0000 7fda71a00640 1 Processor -- start 2024-04-20T10:28:24.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.246+0000 7fda71a00640 1 -- start start 2024-04-20T10:28:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.248+0000 7fda71a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fda6c06f090 0x7fda6c0ad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.248+0000 7fda71a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda6c06df70 con 0x7fda6c0d0a70 2024-04-20T10:28:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.248+0000 7fda71a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fda6c06e310 con 0x7fda6c162c00 2024-04-20T10:28:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.248+0000 7fda71a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fda6c162fe0 con 0x7fda6c06f090 2024-04-20T10:28:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.248+0000 7fda6a000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fda6c0d0a70 0x7fda6c0ad050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40262/0 (socket says 172.21.15.47:40262) 2024-04-20T10:28:24.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.248+0000 7fda6a000640 1 -- 172.21.15.47:0/356866794 learned_addr learned my addr 172.21.15.47:0/356866794 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.248+0000 7fda6b400640 1 --2- 172.21.15.47:0/356866794 >> v2:172.21.15.47:3300/0 conn(0x7fda6c06f090 0x7fda6c0ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.249+0000 7fda69600640 1 -- 172.21.15.47:0/356866794 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1523873971 0 0) 0x7fda6c06df70 con 0x7fda6c0d0a70 2024-04-20T10:28:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.249+0000 7fda6b400640 1 -- 172.21.15.47:0/356866794 >> v1:172.21.15.175:6789/0 conn(0x7fda6c162c00 legacy=0x7fda6c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.249+0000 7fda6b400640 1 -- 172.21.15.47:0/356866794 >> v1:172.21.15.47:6789/0 conn(0x7fda6c0d0a70 legacy=0x7fda6c0ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.249+0000 7fda6b400640 1 -- 172.21.15.47:0/356866794 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c06e5b0 con 0x7fda6c06f090 2024-04-20T10:28:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.250+0000 7fda6b400640 1 --2- 172.21.15.47:0/356866794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c06f090 0x7fda6c0ad760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda6c06e4e0 tx=0x7fda6402be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa4d00aaa1d455b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.250+0000 7fda69600640 1 -- 172.21.15.47:0/356866794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64039070 con 0x7fda6c06f090 2024-04-20T10:28:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.250+0000 7fda69600640 1 -- 172.21.15.47:0/356866794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda6402cbc0 con 0x7fda6c06f090 2024-04-20T10:28:24.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.250+0000 7fda69600640 1 -- 172.21.15.47:0/356866794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda64034440 con 0x7fda6c06f090 2024-04-20T10:28:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.251+0000 7fda71a00640 1 -- 172.21.15.47:0/356866794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c06f090 msgr2=0x7fda6c0ad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.251+0000 7fda71a00640 1 --2- 172.21.15.47:0/356866794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c06f090 0x7fda6c0ad760 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fda6c06e4e0 tx=0x7fda6402be10 comp rx=0 tx=0).stop 2024-04-20T10:28:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.251+0000 7fda71a00640 1 -- 172.21.15.47:0/356866794 shutdown_connections 2024-04-20T10:28:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.251+0000 7fda71a00640 1 --2- 172.21.15.47:0/356866794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c06f090 0x7fda6c0ad760 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.251+0000 7fda71a00640 1 -- 172.21.15.47:0/356866794 >> 172.21.15.47:0/356866794 conn(0x7fda6c0b3350 msgr2=0x7fda6c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.251+0000 7fda71a00640 1 -- 172.21.15.47:0/356866794 shutdown_connections 2024-04-20T10:28:24.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.251+0000 7fda71a00640 1 -- 172.21.15.47:0/356866794 wait complete. 2024-04-20T10:28:24.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.253+0000 7fda71a00640 1 Processor -- start 2024-04-20T10:28:24.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.253+0000 7fda71a00640 1 -- start start 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda71a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda6c06f090 0x7fda6c1fd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda71a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c0d0a70 0x7fda6c14e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda71a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda6c162c00 0x7fda6c1651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda71a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c06dbb0 con 0x7fda6c0d0a70 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda71a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda6c06ec00 con 0x7fda6c162c00 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda71a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda6c06dd30 con 0x7fda6c06f090 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda6a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c0d0a70 0x7fda6c14e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda6a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c0d0a70 0x7fda6c14e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40076/0 (socket says 172.21.15.47:40076) 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda6a000640 1 -- 172.21.15.47:0/751335609 learned_addr learned my addr 172.21.15.47:0/751335609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda6a000640 1 -- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda6c06f090 msgr2=0x7fda6c1fd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.254+0000 7fda6a000640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda6c06f090 0x7fda6c1fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda6a000640 1 -- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda6c162c00 msgr2=0x7fda6c1651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda6a000640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda6c162c00 0x7fda6c1651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda6a000640 1 -- 172.21.15.47:0/751335609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda6c1fc580 con 0x7fda6c0d0a70 2024-04-20T10:28:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda6a000640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c0d0a70 0x7fda6c14e6a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c4d0 tx=0x7fda5800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:24.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda62a00640 1 -- 172.21.15.47:0/751335609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58018070 con 0x7fda6c0d0a70 2024-04-20T10:28:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda62a00640 1 -- 172.21.15.47:0/751335609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fda58007500 con 0x7fda6c0d0a70 2024-04-20T10:28:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda62a00640 1 -- 172.21.15.47:0/751335609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda58004a20 con 0x7fda6c0d0a70 2024-04-20T10:28:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda6c151e50 con 0x7fda6c0d0a70 2024-04-20T10:28:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.255+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda6c1fc820 con 0x7fda6c0d0a70 2024-04-20T10:28:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.256+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda6c068980 con 0x7fda6c0d0a70 2024-04-20T10:28:24.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.257+0000 7fda62a00640 1 -- 172.21.15.47:0/751335609 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fda58004bc0 con 0x7fda6c0d0a70 2024-04-20T10:28:24.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.258+0000 7fda62a00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fda4c06c8f0 0x7fda4c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.259+0000 7fda62a00640 1 -- 172.21.15.47:0/751335609 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fda5808b240 con 0x7fda6c0d0a70 2024-04-20T10:28:24.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.259+0000 7fda6aa00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fda4c06c8f0 0x7fda4c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.260+0000 7fda6aa00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fda4c06c8f0 0x7fda4c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda5c006fd0 tx=0x7fda5c006bc0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:24.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.263+0000 7fda62a00640 1 -- 172.21.15.47:0/751335609 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fda58059220 con 0x7fda6c0d0a70 2024-04-20T10:28:24.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.467+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fda6c160bf0 con 0x7fda6c0d0a70 2024-04-20T10:28:24.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.468+0000 7fda62a00640 1 -- 172.21.15.47:0/751335609 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fda58058b20 con 0x7fda6c0d0a70 2024-04-20T10:28:24.469 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.471+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fda4c06c8f0 msgr2=0x7fda4c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.471+0000 7fda71a00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fda4c06c8f0 0x7fda4c06edb0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fda5c006fd0 tx=0x7fda5c006bc0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.471+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c0d0a70 msgr2=0x7fda6c14e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.471+0000 7fda71a00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c0d0a70 0x7fda6c14e6a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fda5800c4d0 tx=0x7fda5800c990 comp rx=0 tx=0).stop 2024-04-20T10:28:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 shutdown_connections 2024-04-20T10:28:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda6c162c00 0x7fda6c1651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fda4c06c8f0 0x7fda4c06edb0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fda6c0d0a70 0x7fda6c14e6a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 --2- 172.21.15.47:0/751335609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda6c06f090 0x7fda6c1fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 >> 172.21.15.47:0/751335609 conn(0x7fda6c0b3350 msgr2=0x7fda6c0722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 shutdown_connections 2024-04-20T10:28:24.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.472+0000 7fda71a00640 1 -- 172.21.15.47:0/751335609 wait complete. 2024-04-20T10:28:24.492 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:24.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.707+0000 7f5da5800640 1 Processor -- start 2024-04-20T10:28:24.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.707+0000 7f5da5800640 1 -- start start 2024-04-20T10:28:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5da5800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5da015f7b0 0x7f5da0070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5da5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015fc60 0x7f5da00cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5da5800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5da006ed60 con 0x7f5da0144eb0 2024-04-20T10:28:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5da5800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5da0145290 con 0x7f5da015f7b0 2024-04-20T10:28:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5da5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5da006f970 con 0x7f5da015fc60 2024-04-20T10:28:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5d9e000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5da015f7b0 0x7f5da0070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5d9e000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5da015f7b0 0x7f5da0070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40092/0 (socket says 172.21.15.47:40092) 2024-04-20T10:28:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.709+0000 7f5d9e000640 1 -- 172.21.15.47:0/3181495253 learned_addr learned my addr 172.21.15.47:0/3181495253 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.710+0000 7f5d9f400640 1 --2- 172.21.15.47:0/3181495253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015fc60 0x7f5da00cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.710+0000 7f5d9e000640 1 -- 172.21.15.47:0/3181495253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015fc60 msgr2=0x7f5da00cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.710+0000 7f5d9e000640 1 --2- 172.21.15.47:0/3181495253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015fc60 0x7f5da00cb9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.710+0000 7f5d9e000640 1 -- 172.21.15.47:0/3181495253 >> v1:172.21.15.175:6789/0 conn(0x7f5da0144eb0 legacy=0x7f5da015f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.710+0000 7f5d9e000640 1 -- 172.21.15.47:0/3181495253 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da006f040 con 0x7f5da015f7b0 2024-04-20T10:28:24.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.710+0000 7f5d9f400640 1 --2- 172.21.15.47:0/3181495253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015fc60 0x7f5da00cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.711+0000 7f5d9e000640 1 --2- 172.21.15.47:0/3181495253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da015f7b0 0x7f5da0070eb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800bb10 tx=0x7f5d8802e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8f74463ffacf86 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.711+0000 7f5d9d600640 1 -- 172.21.15.47:0/3181495253 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8803b070 con 0x7f5da015f7b0 2024-04-20T10:28:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.711+0000 7f5d9d600640 1 -- 172.21.15.47:0/3181495253 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d88036440 con 0x7f5da015f7b0 2024-04-20T10:28:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.711+0000 7f5d9d600640 1 -- 172.21.15.47:0/3181495253 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88036810 con 0x7f5da015f7b0 2024-04-20T10:28:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.712+0000 7f5da5800640 1 -- 172.21.15.47:0/3181495253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da015f7b0 msgr2=0x7f5da0070eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.712+0000 7f5da5800640 1 --2- 172.21.15.47:0/3181495253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da015f7b0 0x7f5da0070eb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800bb10 tx=0x7f5d8802e150 comp rx=0 tx=0).stop 2024-04-20T10:28:24.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.712+0000 7f5da5800640 1 -- 172.21.15.47:0/3181495253 shutdown_connections 2024-04-20T10:28:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.712+0000 7f5da5800640 1 --2- 172.21.15.47:0/3181495253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015fc60 0x7f5da00cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.712+0000 7f5da5800640 1 --2- 172.21.15.47:0/3181495253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da015f7b0 0x7f5da0070eb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.712+0000 7f5da5800640 1 -- 172.21.15.47:0/3181495253 >> 172.21.15.47:0/3181495253 conn(0x7f5da0070390 msgr2=0x7f5da00707a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.712+0000 7f5da5800640 1 -- 172.21.15.47:0/3181495253 shutdown_connections 2024-04-20T10:28:24.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.713+0000 7f5da5800640 1 -- 172.21.15.47:0/3181495253 wait complete. 2024-04-20T10:28:24.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.714+0000 7f5da5800640 1 Processor -- start 2024-04-20T10:28:24.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.714+0000 7f5da5800640 1 -- start start 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.715+0000 7f5da5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da0144eb0 0x7f5da00b6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.715+0000 7f5da5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015f7b0 0x7f5da00b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.715+0000 7f5da5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da015fc60 0x7f5da00b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.715+0000 7f5da5800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5da006e8f0 con 0x7f5da0144eb0 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.715+0000 7f5da5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5da006f600 con 0x7f5da015fc60 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.715+0000 7f5da5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5da006ea70 con 0x7f5da015f7b0 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da0144eb0 0x7f5da00b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da0144eb0 0x7f5da00b6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40096/0 (socket says 172.21.15.47:40096) 2024-04-20T10:28:24.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 -- 172.21.15.47:0/892916181 learned_addr learned my addr 172.21.15.47:0/892916181 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9e000640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015f7b0 0x7f5da00b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9f400640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da015fc60 0x7f5da00b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 -- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015f7b0 msgr2=0x7f5da00b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015f7b0 0x7f5da00b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 -- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da015fc60 msgr2=0x7f5da00b4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da015fc60 0x7f5da00b4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.716+0000 7f5d9ea00640 1 -- 172.21.15.47:0/892916181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5da00c61e0 con 0x7f5da0144eb0 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.717+0000 7f5d9ea00640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da0144eb0 0x7f5da00b6920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c430 tx=0x7f5d9000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.717+0000 7f5d96a00640 1 -- 172.21.15.47:0/892916181 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011790 con 0x7f5da0144eb0 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.717+0000 7f5d96a00640 1 -- 172.21.15.47:0/892916181 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5d90017070 con 0x7f5da0144eb0 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.717+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5da00c6480 con 0x7f5da0144eb0 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.717+0000 7f5d96a00640 1 -- 172.21.15.47:0/892916181 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d90011bb0 con 0x7f5da0144eb0 2024-04-20T10:28:24.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.717+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5da00b4660 con 0x7f5da0144eb0 2024-04-20T10:28:24.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.719+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5da0069090 con 0x7f5da0144eb0 2024-04-20T10:28:24.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.719+0000 7f5d96a00640 1 -- 172.21.15.47:0/892916181 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5d9001c460 con 0x7f5da0144eb0 2024-04-20T10:28:24.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.721+0000 7f5d96a00640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5d7c06c8f0 0x7f5d7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:24.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.721+0000 7f5d96a00640 1 -- 172.21.15.47:0/892916181 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f5d9008be20 con 0x7f5da0144eb0 2024-04-20T10:28:24.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.722+0000 7f5d9e000640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5d7c06c8f0 0x7f5d7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:24.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.723+0000 7f5d9e000640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5d7c06c8f0 0x7f5d7c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d88038000 tx=0x7f5d88008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:24.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.726+0000 7f5d96a00640 1 -- 172.21.15.47:0/892916181 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5d90059ea0 con 0x7f5da0144eb0 2024-04-20T10:28:24.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.932+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5da00d0c70 con 0x7f5da0144eb0 2024-04-20T10:28:24.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.933+0000 7f5d96a00640 1 -- 172.21.15.47:0/892916181 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f5d900597a0 con 0x7f5da0144eb0 2024-04-20T10:28:24.933 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5d7c06c8f0 msgr2=0x7f5d7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5d7c06c8f0 0x7f5d7c06edb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f5d88038000 tx=0x7f5d88008040 comp rx=0 tx=0).stop 2024-04-20T10:28:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da0144eb0 msgr2=0x7f5da00b6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da0144eb0 0x7f5da00b6920 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d9000c430 tx=0x7f5d9000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 shutdown_connections 2024-04-20T10:28:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5da015fc60 0x7f5da00b4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5da015f7b0 0x7f5da00b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5d7c06c8f0 0x7f5d7c06edb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 --2- 172.21.15.47:0/892916181 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5da0144eb0 0x7f5da00b6920 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 >> 172.21.15.47:0/892916181 conn(0x7f5da0070390 msgr2=0x7f5da00b10e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 shutdown_connections 2024-04-20T10:28:24.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:24.936+0000 7f5da5800640 1 -- 172.21.15.47:0/892916181 wait complete. 2024-04-20T10:28:24.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.179+0000 7fb464000640 1 Processor -- start 2024-04-20T10:28:25.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.180+0000 7fb464000640 1 -- start start 2024-04-20T10:28:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.181+0000 7fb464000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb45c0d51b0 0x7fb45c0b6120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.181+0000 7fb464000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 0x7fb45c0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.181+0000 7fb464000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb45c15ed40 con 0x7fb45c15e960 2024-04-20T10:28:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.181+0000 7fb464000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb45c0b6e90 con 0x7fb45c0d51b0 2024-04-20T10:28:25.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.181+0000 7fb464000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c0d0e50 con 0x7fb45c0d1c10 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.181+0000 7fb461600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb45c15e960 0x7fb45c0d4aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40268/0 (socket says 172.21.15.47:40268) 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb461600640 1 -- 172.21.15.47:0/2301839052 learned_addr learned my addr 172.21.15.47:0/2301839052 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb460c00640 1 --2- 172.21.15.47:0/2301839052 >> v2:172.21.15.175:3300/0 conn(0x7fb45c0d51b0 0x7fb45c0b6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 96586131 0 0) 0x7fb45c15ed40 con 0x7fb45c15e960 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb440003520 con 0x7fb45c15e960 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb462000640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 0x7fb45c0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2425320473 0 0) 0x7fb440003520 con 0x7fb45c15e960 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 msgr2=0x7fb45c0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.182+0000 7fb45be00640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 0x7fb45c0b6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 >> v2:172.21.15.175:3300/0 conn(0x7fb45c0d51b0 msgr2=0x7fb45c0b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 --2- 172.21.15.47:0/2301839052 >> v2:172.21.15.175:3300/0 conn(0x7fb45c0d51b0 0x7fb45c0b6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c0d1130 con 0x7fb45c15e960 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fb44c001670 con 0x7fb45c15e960 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 >> v1:172.21.15.47:6789/0 conn(0x7fb45c15e960 legacy=0x7fb45c0d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4400039b0 0x7fb440003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 0x7fb440004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb4400051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0b6e90 con 0x7fb440004d40 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0d0e50 con 0x7fb4400039b0 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb44000b9c0 con 0x7fb45c0d1c10 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb462000640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb4400051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.183+0000 7fb460c00640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 0x7fb440004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.184+0000 7fb461600640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4400039b0 0x7fb440003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.184+0000 7fb462000640 1 -- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 msgr2=0x7fb440004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.184+0000 7fb462000640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c0d1c10 0x7fb440004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.184+0000 7fb462000640 1 -- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4400039b0 msgr2=0x7fb440003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:25.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.184+0000 7fb462000640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4400039b0 0x7fb440003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.184+0000 7fb462000640 1 -- 172.21.15.47:0/2301839052 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb440003520 con 0x7fb440004d40 2024-04-20T10:28:25.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.184+0000 7fb461600640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4400039b0 0x7fb440003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.185+0000 7fb462000640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb4400051c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb454007360 tx=0x7fb440005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a001e04f37d61b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.185+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454039070 con 0x7fb440004d40 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.185+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb45402cc80 con 0x7fb440004d40 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.185+0000 7fb45be00640 1 -- 172.21.15.47:0/2301839052 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454009be0 con 0x7fb440004d40 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.186+0000 7fb464000640 1 -- 172.21.15.47:0/2301839052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 msgr2=0x7fb4400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.186+0000 7fb464000640 1 --2- 172.21.15.47:0/2301839052 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb4400051c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb454007360 tx=0x7fb440005830 comp rx=0 tx=0).stop 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.186+0000 7fb462000640 1 -- 172.21.15.47:0/2301839052 reap_dead start 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.186+0000 7fb464000640 1 -- 172.21.15.47:0/2301839052 shutdown_connections 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.186+0000 7fb464000640 1 -- 172.21.15.47:0/2301839052 >> 172.21.15.47:0/2301839052 conn(0x7fb45c0b96f0 msgr2=0x7fb45c0b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.186+0000 7fb464000640 1 -- 172.21.15.47:0/2301839052 shutdown_connections 2024-04-20T10:28:25.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.186+0000 7fb464000640 1 -- 172.21.15.47:0/2301839052 wait complete. 2024-04-20T10:28:25.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.188+0000 7fb464000640 1 Processor -- start 2024-04-20T10:28:25.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.188+0000 7fb464000640 1 -- start start 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb464000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb45c1f4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb464000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c14a970 0x7fb45c0ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb464000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45c0cd960 0x7fb45c0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb464000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0b6660 con 0x7fb440004d40 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb464000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c0d16f0 con 0x7fb45c0cd960 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb464000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb45c0b67e0 con 0x7fb45c14a970 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb461600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb45c1f4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb461600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb45c1f4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40114/0 (socket says 172.21.15.47:40114) 2024-04-20T10:28:25.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb461600640 1 -- 172.21.15.47:0/3232281909 learned_addr learned my addr 172.21.15.47:0/3232281909 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb461600640 1 -- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c14a970 msgr2=0x7fb45c0ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb461600640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c14a970 0x7fb45c0ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb461600640 1 -- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45c0cd960 msgr2=0x7fb45c0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb462000640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45c0cd960 0x7fb45c0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.189+0000 7fb461600640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45c0cd960 0x7fb45c0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.190+0000 7fb461600640 1 -- 172.21.15.47:0/3232281909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c1f52e0 con 0x7fb440004d40 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.190+0000 7fb461600640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb45c1f4da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d8b0 tx=0x7fb44c00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.190+0000 7fb459600640 1 -- 172.21.15.47:0/3232281909 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c019070 con 0x7fb440004d40 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.190+0000 7fb459600640 1 -- 172.21.15.47:0/3232281909 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb44c0074e0 con 0x7fb440004d40 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.190+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c157cb0 con 0x7fb440004d40 2024-04-20T10:28:25.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.190+0000 7fb459600640 1 -- 172.21.15.47:0/3232281909 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0049e0 con 0x7fb440004d40 2024-04-20T10:28:25.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.190+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c14b410 con 0x7fb440004d40 2024-04-20T10:28:25.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.191+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c068980 con 0x7fb440004d40 2024-04-20T10:28:25.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.192+0000 7fb459600640 1 -- 172.21.15.47:0/3232281909 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb44c01c070 con 0x7fb440004d40 2024-04-20T10:28:25.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.194+0000 7fb459600640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb43c06c920 0x7fb43c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.194+0000 7fb459600640 1 -- 172.21.15.47:0/3232281909 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fb44c08bb90 con 0x7fb440004d40 2024-04-20T10:28:25.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.195+0000 7fb460c00640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb43c06c920 0x7fb43c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.196+0000 7fb460c00640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb43c06c920 0x7fb43c06ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb45c06c370 tx=0x7fb448008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:25.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.199+0000 7fb459600640 1 -- 172.21.15.47:0/3232281909 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb45c1f52e0 con 0x7fb440004d40 2024-04-20T10:28:25.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.405+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb45c0d0af0 con 0x7fb440004d40 2024-04-20T10:28:25.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.406+0000 7fb459600640 1 -- 172.21.15.47:0/3232281909 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fb44c007690 con 0x7fb440004d40 2024-04-20T10:28:25.406 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb43c06c920 msgr2=0x7fb43c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb43c06c920 0x7fb43c06ede0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb45c06c370 tx=0x7fb448008040 comp rx=0 tx=0).stop 2024-04-20T10:28:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 msgr2=0x7fb45c1f4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb45c1f4da0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d8b0 tx=0x7fb44c00dd70 comp rx=0 tx=0).stop 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 shutdown_connections 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb45c0cd960 0x7fb45c0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45c14a970 0x7fb45c0ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb43c06c920 0x7fb43c06ede0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 --2- 172.21.15.47:0/3232281909 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb440004d40 0x7fb45c1f4da0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.409+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 >> 172.21.15.47:0/3232281909 conn(0x7fb45c0b96f0 msgr2=0x7fb45c0cf6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.410+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 shutdown_connections 2024-04-20T10:28:25.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.410+0000 7fb464000640 1 -- 172.21.15.47:0/3232281909 wait complete. 2024-04-20T10:28:25.429 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.642+0000 7f1499a00640 1 Processor -- start 2024-04-20T10:28:25.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.642+0000 7f1499a00640 1 -- start start 2024-04-20T10:28:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.644+0000 7f1499a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 0x7f149406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.644+0000 7f1499a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149406f060 con 0x7f149415e8a0 2024-04-20T10:28:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.644+0000 7f1499a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f149406e7c0 con 0x7f149406dbb0 2024-04-20T10:28:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.644+0000 7f1499a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f149406eaa0 con 0x7f14940d0a70 2024-04-20T10:28:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.644+0000 7f1492a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f149415e8a0 0x7f14940d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40278/0 (socket says 172.21.15.47:40278) 2024-04-20T10:28:25.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.644+0000 7f1492a00640 1 -- 172.21.15.47:0/2668248275 learned_addr learned my addr 172.21.15.47:0/2668248275 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1492000640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 0x7f149406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 425805360 0 0) 0x7f149406f060 con 0x7f149415e8a0 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1478003520 con 0x7f149415e8a0 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3848644358 0 0) 0x7f1478003520 con 0x7f149415e8a0 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 msgr2=0x7f149406e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 0x7f149406e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 >> v1:172.21.15.175:6789/0 conn(0x7f149406dbb0 legacy=0x7f14940ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149406ed80 con 0x7f149415e8a0 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f1484001670 con 0x7f149415e8a0 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 >> v1:172.21.15.47:6789/0 conn(0x7f149415e8a0 legacy=0x7f14940d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14780039e0 0x7f1478003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1478004800 0x7f1478004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 0x7f14780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f149406eaa0 con 0x7f1478004800 2024-04-20T10:28:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f147800ba70 con 0x7f14780039e0 2024-04-20T10:28:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.645+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f147800bc70 con 0x7f14940d0a70 2024-04-20T10:28:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1492000640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1478004800 0x7f1478004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1492a00640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14780039e0 0x7f1478003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1493400640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 0x7f14780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1492000640 1 -- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 msgr2=0x7f14780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:25.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1492000640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 0x7f14780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1492000640 1 -- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14780039e0 msgr2=0x7f1478003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:25.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1492000640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14780039e0 0x7f1478003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1492000640 1 -- 172.21.15.47:0/2668248275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1478003520 con 0x7f1478004800 2024-04-20T10:28:25.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.646+0000 7f1493400640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14940d0a70 0x7f14780051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:25.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.648+0000 7f1492000640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1478004800 0x7f1478004c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1480009a90 tx=0x7f1478005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fa9488f27165e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:25.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480039070 con 0x7f1478004800 2024-04-20T10:28:25.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148002c680 con 0x7f1478004800 2024-04-20T10:28:25.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1491600640 1 -- 172.21.15.47:0/2668248275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480009690 con 0x7f1478004800 2024-04-20T10:28:25.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1499a00640 1 -- 172.21.15.47:0/2668248275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1478004800 msgr2=0x7f1478004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1499a00640 1 --2- 172.21.15.47:0/2668248275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1478004800 0x7f1478004c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1480009a90 tx=0x7f1478005830 comp rx=0 tx=0).stop 2024-04-20T10:28:25.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1493400640 1 -- 172.21.15.47:0/2668248275 reap_dead start 2024-04-20T10:28:25.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1499a00640 1 -- 172.21.15.47:0/2668248275 shutdown_connections 2024-04-20T10:28:25.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.649+0000 7f1499a00640 1 -- 172.21.15.47:0/2668248275 >> 172.21.15.47:0/2668248275 conn(0x7f14940b3350 msgr2=0x7f14940b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:25.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.650+0000 7f1499a00640 1 -- 172.21.15.47:0/2668248275 shutdown_connections 2024-04-20T10:28:25.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.650+0000 7f1499a00640 1 -- 172.21.15.47:0/2668248275 wait complete. 2024-04-20T10:28:25.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.651+0000 7f1499a00640 1 Processor -- start 2024-04-20T10:28:25.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.652+0000 7f1499a00640 1 -- start start 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.652+0000 7f1499a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1478004800 0x7f1494147850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.652+0000 7f1499a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1494159d20 0x7f1494157f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.652+0000 7f1499a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14941611d0 0x7f1494158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.652+0000 7f1499a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f149406fef0 con 0x7f1494159d20 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.652+0000 7f1499a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f149406fd70 con 0x7f1478004800 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.652+0000 7f1499a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f149406e1e0 con 0x7f14941611d0 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1494159d20 0x7f1494157f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1494159d20 0x7f1494157f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40130/0 (socket says 172.21.15.47:40130) 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 -- 172.21.15.47:0/2038481921 learned_addr learned my addr 172.21.15.47:0/2038481921 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1493400640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14941611d0 0x7f1494158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 -- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14941611d0 msgr2=0x7f1494158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14941611d0 0x7f1494158480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 -- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1478004800 msgr2=0x7f1494147850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1478004800 0x7f1494147850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.653+0000 7f1492000640 1 -- 172.21.15.47:0/2038481921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1494150450 con 0x7f1494159d20 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.654+0000 7f1492000640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1494159d20 0x7f1494157f40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.654+0000 7f148aa00640 1 -- 172.21.15.47:0/2038481921 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480042020 con 0x7f1494159d20 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.654+0000 7f148aa00640 1 -- 172.21.15.47:0/2038481921 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f148003a040 con 0x7f1494159d20 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.654+0000 7f148aa00640 1 -- 172.21.15.47:0/2038481921 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1480039070 con 0x7f1494159d20 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.654+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1494152e30 con 0x7f1494159d20 2024-04-20T10:28:25.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.654+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14941591d0 con 0x7f1494159d20 2024-04-20T10:28:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.655+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1494068980 con 0x7f1494159d20 2024-04-20T10:28:25.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.656+0000 7f148aa00640 1 -- 172.21.15.47:0/2038481921 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1480045020 con 0x7f1494159d20 2024-04-20T10:28:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.657+0000 7f148aa00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f147406c920 0x7f147406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:25.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.658+0000 7f148aa00640 1 -- 172.21.15.47:0/2038481921 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f14800ad410 con 0x7f1494159d20 2024-04-20T10:28:25.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.658+0000 7f1492a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f147406c920 0x7f147406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:25.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.660+0000 7f1492a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f147406c920 0x7f147406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1484001200 tx=0x7f1484001110 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:25.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.662+0000 7f148aa00640 1 -- 172.21.15.47:0/2038481921 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f148004c070 con 0x7f1494159d20 2024-04-20T10:28:25.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.869+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1494158d20 con 0x7f1494159d20 2024-04-20T10:28:25.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.871+0000 7f148aa00640 1 -- 172.21.15.47:0/2038481921 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f148007b3f0 con 0x7f1494159d20 2024-04-20T10:28:25.871 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:25.872 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f147406c920 msgr2=0x7f147406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f147406c920 0x7f147406ede0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1484001200 tx=0x7f1484001110 comp rx=0 tx=0).stop 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1494159d20 msgr2=0x7f1494157f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1494159d20 0x7f1494157f40 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1480000c00 tx=0x7f1480004730 comp rx=0 tx=0).stop 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 shutdown_connections 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f14941611d0 0x7f1494158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f147406c920 0x7f147406ede0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1494159d20 0x7f1494157f40 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 --2- 172.21.15.47:0/2038481921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1478004800 0x7f1494147850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 >> 172.21.15.47:0/2038481921 conn(0x7f14940b3350 msgr2=0x7f1494153830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 shutdown_connections 2024-04-20T10:28:25.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:25.874+0000 7f1499a00640 1 -- 172.21.15.47:0/2038481921 wait complete. 2024-04-20T10:28:25.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:25.894 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:26.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.104+0000 7f5515200640 1 Processor -- start 2024-04-20T10:28:26.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.105+0000 7f5515200640 1 -- start start 2024-04-20T10:28:26.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.107+0000 7f5515200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f551015b360 0x7f5510071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.107+0000 7f5515200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551006f3b0 con 0x7f551015af80 2024-04-20T10:28:26.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.107+0000 7f5515200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f551006f690 con 0x7f55100cb570 2024-04-20T10:28:26.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.107+0000 7f5515200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f551009e150 con 0x7f551015b360 2024-04-20T10:28:26.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.107+0000 7f550f400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f551015af80 0x7f55100cae60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40288/0 (socket says 172.21.15.47:40288) 2024-04-20T10:28:26.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.107+0000 7f550f400640 1 -- 172.21.15.47:0/3730679983 learned_addr learned my addr 172.21.15.47:0/3730679983 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.107+0000 7f550fe00640 1 --2- 172.21.15.47:0/3730679983 >> v2:172.21.15.175:3300/0 conn(0x7f551015b360 0x7f5510071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 761808055 0 0) 0x7f551006f3b0 con 0x7f551015af80 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f54f4005730 con 0x7f551015af80 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1636644544 0 0) 0x7f54f4005730 con 0x7f551015af80 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 >> v2:172.21.15.175:3300/0 conn(0x7f551015b360 msgr2=0x7f5510071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 --2- 172.21.15.47:0/3730679983 >> v2:172.21.15.175:3300/0 conn(0x7f551015b360 0x7f5510071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 >> v1:172.21.15.175:6789/0 conn(0x7f55100cb570 legacy=0x7f55100ad9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55100ad1a0 con 0x7f551015af80 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f5500001670 con 0x7f551015af80 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 >> v1:172.21.15.47:6789/0 conn(0x7f551015af80 legacy=0x7f55100cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4005bc0 0x7f54f4005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4006a10 0x7f54f4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40073d0 0x7f54f400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f551009e150 con 0x7f54f40073d0 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54f400e370 con 0x7f54f4006a10 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.108+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54f400e570 con 0x7f54f4005bc0 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550fe00640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40073d0 0x7f54f400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550f400640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4005bc0 0x7f54f4005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550fe00640 1 -- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4005bc0 msgr2=0x7f54f4005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550fe00640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54f4005bc0 0x7f54f4005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550ea00640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4006a10 0x7f54f4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550fe00640 1 -- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4006a10 msgr2=0x7f54f4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:26.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550fe00640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4006a10 0x7f54f4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550fe00640 1 -- 172.21.15.47:0/3730679983 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f4005730 con 0x7f54f40073d0 2024-04-20T10:28:26.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.109+0000 7f550ea00640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f4006a10 0x7f54f4006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.110+0000 7f550fe00640 1 -- 172.21.15.47:0/3730679983 reap_dead start 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.110+0000 7f550fe00640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40073d0 0x7f54f400dbd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5508009fc0 tx=0x7f54f4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cec83ccc3d7bf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.110+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508009c10 con 0x7f54f40073d0 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.110+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f550802cc60 con 0x7f54f40073d0 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.110+0000 7f550e000640 1 -- 172.21.15.47:0/3730679983 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5508003bd0 con 0x7f54f40073d0 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.111+0000 7f5515200640 1 -- 172.21.15.47:0/3730679983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40073d0 msgr2=0x7f54f400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.111+0000 7f5515200640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40073d0 0x7f54f400dbd0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5508009fc0 tx=0x7f54f4007a40 comp rx=0 tx=0).stop 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.111+0000 7f5515200640 1 -- 172.21.15.47:0/3730679983 shutdown_connections 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.111+0000 7f5515200640 1 --2- 172.21.15.47:0/3730679983 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54f40073d0 0x7f54f400dbd0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.111+0000 7f5515200640 1 -- 172.21.15.47:0/3730679983 >> 172.21.15.47:0/3730679983 conn(0x7f55100cffa0 msgr2=0x7f55100d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.111+0000 7f5515200640 1 -- 172.21.15.47:0/3730679983 shutdown_connections 2024-04-20T10:28:26.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.111+0000 7f5515200640 1 -- 172.21.15.47:0/3730679983 wait complete. 2024-04-20T10:28:26.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.113+0000 7f5515200640 1 Processor -- start 2024-04-20T10:28:26.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.113+0000 7f5515200640 1 -- start start 2024-04-20T10:28:26.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f5515200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40073d0 0x7f551009b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f5515200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f551009ccc0 0x7f551009d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f5515200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55100b38e0 0x7f55100b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f5515200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f55100ca910 con 0x7f551009ccc0 2024-04-20T10:28:26.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f5515200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55100acf50 con 0x7f54f40073d0 2024-04-20T10:28:26.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f5515200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55100caa90 con 0x7f55100b38e0 2024-04-20T10:28:26.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f550ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f551009ccc0 0x7f551009d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f550ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f551009ccc0 0x7f551009d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40148/0 (socket says 172.21.15.47:40148) 2024-04-20T10:28:26.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.114+0000 7f550ea00640 1 -- 172.21.15.47:0/3208366814 learned_addr learned my addr 172.21.15.47:0/3208366814 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550f400640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40073d0 0x7f551009b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550fe00640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55100b38e0 0x7f55100b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550ea00640 1 -- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55100b38e0 msgr2=0x7f55100b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550ea00640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55100b38e0 0x7f55100b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550ea00640 1 -- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40073d0 msgr2=0x7f551009b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550ea00640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40073d0 0x7f551009b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550ea00640 1 -- 172.21.15.47:0/3208366814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55100b19d0 con 0x7f551009ccc0 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f550ea00640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f551009ccc0 0x7f551009d140 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00d770 tx=0x7f54fc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f5507400640 1 -- 172.21.15.47:0/3208366814 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc005270 con 0x7f551009ccc0 2024-04-20T10:28:26.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f5507400640 1 -- 172.21.15.47:0/3208366814 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f54fc017070 con 0x7f551009ccc0 2024-04-20T10:28:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55100b16f0 con 0x7f551009ccc0 2024-04-20T10:28:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.115+0000 7f5507400640 1 -- 172.21.15.47:0/3208366814 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54fc00b6b0 con 0x7f551009ccc0 2024-04-20T10:28:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.116+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55100b30e0 con 0x7f551009ccc0 2024-04-20T10:28:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.117+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5510075b30 con 0x7f551009ccc0 2024-04-20T10:28:26.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.118+0000 7f5507400640 1 -- 172.21.15.47:0/3208366814 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f54fc00b850 con 0x7f551009ccc0 2024-04-20T10:28:26.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.119+0000 7f5507400640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f54f006c920 0x7f54f006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.119+0000 7f5507400640 1 -- 172.21.15.47:0/3208366814 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f54fc08bd30 con 0x7f551009ccc0 2024-04-20T10:28:26.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.119+0000 7f550f400640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f54f006c920 0x7f54f006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.121+0000 7f550f400640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f54f006c920 0x7f54f006ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5500001eb0 tx=0x7f5500000f10 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:26.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.123+0000 7f5507400640 1 -- 172.21.15.47:0/3208366814 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f54fc019310 con 0x7f551009ccc0 2024-04-20T10:28:26.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.318+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5510073e20 con 0x7f551009ccc0 2024-04-20T10:28:26.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.320+0000 7f5507400640 1 -- 172.21.15.47:0/3208366814 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f54fc001230 con 0x7f551009ccc0 2024-04-20T10:28:26.320 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f54f006c920 msgr2=0x7f54f006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f54f006c920 0x7f54f006ede0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f5500001eb0 tx=0x7f5500000f10 comp rx=0 tx=0).stop 2024-04-20T10:28:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f551009ccc0 msgr2=0x7f551009d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f551009ccc0 0x7f551009d140 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00d770 tx=0x7f54fc00dc30 comp rx=0 tx=0).stop 2024-04-20T10:28:26.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 shutdown_connections 2024-04-20T10:28:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55100b38e0 0x7f55100b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f54f006c920 0x7f54f006ede0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f551009ccc0 0x7f551009d140 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 --2- 172.21.15.47:0/3208366814 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54f40073d0 0x7f551009b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 >> 172.21.15.47:0/3208366814 conn(0x7f55100cffa0 msgr2=0x7f5510155c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 shutdown_connections 2024-04-20T10:28:26.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.322+0000 7f5515200640 1 -- 172.21.15.47:0/3208366814 wait complete. 2024-04-20T10:28:26.341 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.557+0000 7f31ad400640 1 Processor -- start 2024-04-20T10:28:26.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.557+0000 7f31ad400640 1 -- start start 2024-04-20T10:28:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.558+0000 7f31ad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 0x7f31a806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.558+0000 7f31ad400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a815b5d0 con 0x7f31a815b1f0 2024-04-20T10:28:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.558+0000 7f31ad400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a806e730 con 0x7f31a80d0010 2024-04-20T10:28:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.558+0000 7f31ad400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31a80ae310 con 0x7f31a806ca10 2024-04-20T10:28:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.559+0000 7f31a7e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f31a815b1f0 0x7f31a815f730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40298/0 (socket says 172.21.15.47:40298) 2024-04-20T10:28:26.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.559+0000 7f31a7e00640 1 -- 172.21.15.47:0/1141806756 learned_addr learned my addr 172.21.15.47:0/1141806756 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.559+0000 7f31aca00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 0x7f31a806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.559+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 873465575 0 0) 0x7f31a815b5d0 con 0x7f31a815b1f0 2024-04-20T10:28:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.559+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f318c003520 con 0x7f31a815b1f0 2024-04-20T10:28:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4156810158 0 0) 0x7f318c003520 con 0x7f31a815b1f0 2024-04-20T10:28:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 msgr2=0x7f31a806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 0x7f31a806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 >> v1:172.21.15.175:6789/0 conn(0x7f31a80d0010 legacy=0x7f31a80adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a80d05d0 con 0x7f31a815b1f0 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f31980019f0 con 0x7f31a815b1f0 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 >> v1:172.21.15.47:6789/0 conn(0x7f31a815b1f0 legacy=0x7f31a815f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c0039e0 0x7f318c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 0x7f318c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f318c004d40 0x7f318c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31a80ae310 con 0x7f318c004d40 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f318c00dd00 con 0x7f318c0039e0 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f318c00df00 con 0x7f31a806ca10 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.560+0000 7f31aca00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f318c004d40 0x7f318c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31a7400640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 0x7f318c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31a7e00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c0039e0 0x7f318c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31aca00640 1 -- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 msgr2=0x7f318c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31aca00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 0x7f318c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31aca00640 1 -- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c0039e0 msgr2=0x7f318c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31aca00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c0039e0 0x7f318c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31aca00640 1 -- 172.21.15.47:0/1141806756 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f318c003520 con 0x7f318c004d40 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31a7e00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c0039e0 0x7f318c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:26.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.561+0000 7f31a7400640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a806ca10 0x7f318c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.562+0000 7f31aca00640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f318c004d40 0x7f318c0051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009ac0 tx=0x7f318c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6442b9762032ad32 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.562+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0039070 con 0x7f318c004d40 2024-04-20T10:28:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.562+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a002c6b0 con 0x7f318c004d40 2024-04-20T10:28:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.562+0000 7f31a6a00640 1 -- 172.21.15.47:0/1141806756 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0009690 con 0x7f318c004d40 2024-04-20T10:28:26.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.563+0000 7f31ad400640 1 -- 172.21.15.47:0/1141806756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f318c004d40 msgr2=0x7f318c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.563+0000 7f31ad400640 1 --2- 172.21.15.47:0/1141806756 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f318c004d40 0x7f318c0051c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009ac0 tx=0x7f318c005830 comp rx=0 tx=0).stop 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.563+0000 7f31aca00640 1 -- 172.21.15.47:0/1141806756 reap_dead start 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.563+0000 7f31ad400640 1 -- 172.21.15.47:0/1141806756 shutdown_connections 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.563+0000 7f31ad400640 1 -- 172.21.15.47:0/1141806756 >> 172.21.15.47:0/1141806756 conn(0x7f31a80b5270 msgr2=0x7f31a80b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.563+0000 7f31ad400640 1 -- 172.21.15.47:0/1141806756 shutdown_connections 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.563+0000 7f31ad400640 1 -- 172.21.15.47:0/1141806756 wait complete. 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.565+0000 7f31ad400640 1 Processor -- start 2024-04-20T10:28:26.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.565+0000 7f31ad400640 1 -- start start 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31ad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c004d40 0x7f31a81f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31ad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a81f83d0 0x7f31a80b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31ad400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a81f3b20 0x7f31a81e46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31ad400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31a80d0e70 con 0x7f31a81f3b20 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31ad400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31a80d0b90 con 0x7f318c004d40 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31ad400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31a806cea0 con 0x7f31a81f83d0 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31aca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a81f3b20 0x7f31a81e46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31aca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a81f3b20 0x7f31a81e46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40162/0 (socket says 172.21.15.47:40162) 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.566+0000 7f31aca00640 1 -- 172.21.15.47:0/3140927702 learned_addr learned my addr 172.21.15.47:0/3140927702 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31a7e00640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c004d40 0x7f31a81f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31a7400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a81f83d0 0x7f31a80b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31aca00640 1 -- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a81f83d0 msgr2=0x7f31a80b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31aca00640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a81f83d0 0x7f31a80b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31aca00640 1 -- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c004d40 msgr2=0x7f31a81f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31aca00640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c004d40 0x7f31a81f4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31aca00640 1 -- 172.21.15.47:0/3140927702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a80b1bb0 con 0x7f31a81f3b20 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31a7400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a81f83d0 0x7f31a80b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31aca00640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a81f3b20 0x7f31a81e46f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009ac0 tx=0x7f31a0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f319fe00640 1 -- 172.21.15.47:0/3140927702 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0044070 con 0x7f31a81f3b20 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f319fe00640 1 -- 172.21.15.47:0/3140927702 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f31a003a040 con 0x7f31a81f3b20 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a80b1e50 con 0x7f31a81f3b20 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.567+0000 7f319fe00640 1 -- 172.21.15.47:0/3140927702 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0039070 con 0x7f31a81f3b20 2024-04-20T10:28:26.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.568+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a81f78f0 con 0x7f31a81f3b20 2024-04-20T10:28:26.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.569+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a8075b30 con 0x7f31a81f3b20 2024-04-20T10:28:26.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.570+0000 7f319fe00640 1 -- 172.21.15.47:0/3140927702 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f31a0003610 con 0x7f31a81f3b20 2024-04-20T10:28:26.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.571+0000 7f319fe00640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318806c8f0 0x7f318806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:26.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.571+0000 7f319fe00640 1 -- 172.21.15.47:0/3140927702 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f31a00acb00 con 0x7f31a81f3b20 2024-04-20T10:28:26.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.571+0000 7f31a7e00640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318806c8f0 0x7f318806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:26.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.573+0000 7f31a7e00640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318806c8f0 0x7f318806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3198002230 tx=0x7f3198009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:26.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.575+0000 7f319fe00640 1 -- 172.21.15.47:0/3140927702 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f31a007a530 con 0x7f31a81f3b20 2024-04-20T10:28:26.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.781+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f31a8072860 con 0x7f31a81f3b20 2024-04-20T10:28:26.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.782+0000 7f319fe00640 1 -- 172.21.15.47:0/3140927702 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f31a007e830 con 0x7f31a81f3b20 2024-04-20T10:28:26.783 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:26.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318806c8f0 msgr2=0x7f318806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318806c8f0 0x7f318806edb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3198002230 tx=0x7f3198009040 comp rx=0 tx=0).stop 2024-04-20T10:28:26.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a81f3b20 msgr2=0x7f31a81e46f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:26.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a81f3b20 0x7f31a81e46f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f31a0009ac0 tx=0x7f31a0004730 comp rx=0 tx=0).stop 2024-04-20T10:28:26.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 shutdown_connections 2024-04-20T10:28:26.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318806c8f0 0x7f318806edb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a81f3b20 0x7f31a81e46f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a81f83d0 0x7f31a80b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 --2- 172.21.15.47:0/3140927702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f318c004d40 0x7f31a81f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:26.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 >> 172.21.15.47:0/3140927702 conn(0x7f31a80b5270 msgr2=0x7f31a814fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:26.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 shutdown_connections 2024-04-20T10:28:26.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:26.785+0000 7f31ad400640 1 -- 172.21.15.47:0/3140927702 wait complete. 2024-04-20T10:28:26.805 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.019+0000 7ff21e800640 1 Processor -- start 2024-04-20T10:28:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.019+0000 7ff21e800640 1 -- start start 2024-04-20T10:28:27.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.021+0000 7ff21e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 0x7ff2180cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.021+0000 7ff21e800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21806b9a0 con 0x7ff2180ad660 2024-04-20T10:28:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.021+0000 7ff21e800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff21806bc10 con 0x7ff2180ca740 2024-04-20T10:28:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.021+0000 7ff21e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff21806c540 con 0x7ff2181629b0 2024-04-20T10:28:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.022+0000 7ff21de00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff2180ad660 0x7ff2180cdea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40312/0 (socket says 172.21.15.47:40312) 2024-04-20T10:28:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.022+0000 7ff21de00640 1 -- 172.21.15.47:0/1009803629 learned_addr learned my addr 172.21.15.47:0/1009803629 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.022+0000 7ff21d400640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 0x7ff2180cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.022+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 712307004 0 0) 0x7ff21806b9a0 con 0x7ff2180ad660 2024-04-20T10:28:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.022+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff200003520 con 0x7ff2180ad660 2024-04-20T10:28:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4259048412 0 0) 0x7ff200003520 con 0x7ff2180ad660 2024-04-20T10:28:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 msgr2=0x7ff2180cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 0x7ff2180cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 >> v1:172.21.15.175:6789/0 conn(0x7ff2180ca740 legacy=0x7ff2180acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff21806bef0 con 0x7ff2180ad660 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7ff2100019f0 con 0x7ff2180ad660 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 >> v1:172.21.15.47:6789/0 conn(0x7ff2180ad660 legacy=0x7ff2180cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 0x7ff2000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200003f60 0x7ff200004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff200004d40 0x7ff2000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff21806c540 con 0x7ff200004d40 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff20000baa0 con 0x7ff200003f60 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff20000bca0 con 0x7ff2181629b0 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff21de00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff200004d40 0x7ff2000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.023+0000 7ff21ca00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200003f60 0x7ff200004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.024+0000 7ff21d400640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 0x7ff2000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.024+0000 7ff21de00640 1 -- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 msgr2=0x7ff2000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.024+0000 7ff21de00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2181629b0 0x7ff2000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.024+0000 7ff21de00640 1 -- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200003f60 msgr2=0x7ff200004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.024+0000 7ff21de00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200003f60 0x7ff200004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.024+0000 7ff21de00640 1 -- 172.21.15.47:0/1009803629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff200003520 con 0x7ff200004d40 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.025+0000 7ff21de00640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff200004d40 0x7ff2000051c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff21000a860 tx=0x7ff200005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36713ed42fe3a212 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.025+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000a480 con 0x7ff200004d40 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.025+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff210034630 con 0x7ff200004d40 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.025+0000 7ff217e00640 1 -- 172.21.15.47:0/1009803629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210034aa0 con 0x7ff200004d40 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.026+0000 7ff21e800640 1 -- 172.21.15.47:0/1009803629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff200004d40 msgr2=0x7ff2000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.026+0000 7ff21e800640 1 --2- 172.21.15.47:0/1009803629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff200004d40 0x7ff2000051c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff21000a860 tx=0x7ff200005830 comp rx=0 tx=0).stop 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.026+0000 7ff21de00640 1 -- 172.21.15.47:0/1009803629 reap_dead start 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.026+0000 7ff21e800640 1 -- 172.21.15.47:0/1009803629 shutdown_connections 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.026+0000 7ff21e800640 1 -- 172.21.15.47:0/1009803629 >> 172.21.15.47:0/1009803629 conn(0x7ff2180cffa0 msgr2=0x7ff2180d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.026+0000 7ff21e800640 1 -- 172.21.15.47:0/1009803629 shutdown_connections 2024-04-20T10:28:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.026+0000 7ff21e800640 1 -- 172.21.15.47:0/1009803629 wait complete. 2024-04-20T10:28:27.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.028+0000 7ff21e800640 1 Processor -- start 2024-04-20T10:28:27.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.028+0000 7ff21e800640 1 -- start start 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200004d40 0x7ff2180b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff21815cf40 0x7ff2181fc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff21815d3c0 0x7ff2181fc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21e800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2180cf5f0 con 0x7ff21815d3c0 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff218162d90 con 0x7ff200004d40 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2180cf770 con 0x7ff21815cf40 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff21815d3c0 0x7ff2181fc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff21815d3c0 0x7ff2181fc6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40186/0 (socket says 172.21.15.47:40186) 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.029+0000 7ff21de00640 1 -- 172.21.15.47:0/3626355741 learned_addr learned my addr 172.21.15.47:0/3626355741 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21d400640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200004d40 0x7ff2180b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21ca00640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff21815cf40 0x7ff2181fc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21de00640 1 -- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff21815cf40 msgr2=0x7ff2181fc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21de00640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff21815cf40 0x7ff2181fc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21de00640 1 -- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200004d40 msgr2=0x7ff2180b5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21de00640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200004d40 0x7ff2180b5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21de00640 1 -- 172.21.15.47:0/3626355741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2181f9400 con 0x7ff21815d3c0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff21de00640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff21815d3c0 0x7ff2181fc6e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff21000a860 tx=0x7ff21806bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff215600640 1 -- 172.21.15.47:0/3626355741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff210039040 con 0x7ff21815d3c0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.030+0000 7ff215600640 1 -- 172.21.15.47:0/3626355741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff210043070 con 0x7ff21815d3c0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.031+0000 7ff215600640 1 -- 172.21.15.47:0/3626355741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff21000a480 con 0x7ff21815d3c0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.031+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2181f96e0 con 0x7ff21815d3c0 2024-04-20T10:28:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.031+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2181f9f80 con 0x7ff21815d3c0 2024-04-20T10:28:27.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.032+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2181f9d70 con 0x7ff21815d3c0 2024-04-20T10:28:27.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.033+0000 7ff215600640 1 -- 172.21.15.47:0/3626355741 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2100349e0 con 0x7ff21815d3c0 2024-04-20T10:28:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.034+0000 7ff215600640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1f806c8f0 0x7ff1f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.034+0000 7ff215600640 1 -- 172.21.15.47:0/3626355741 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff2100adce0 con 0x7ff21815d3c0 2024-04-20T10:28:27.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.034+0000 7ff21d400640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1f806c8f0 0x7ff1f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.036+0000 7ff21d400640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1f806c8f0 0x7ff1f806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff2180ca000 tx=0x7ff20c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.038+0000 7ff215600640 1 -- 172.21.15.47:0/3626355741 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff21007bcc0 con 0x7ff21815d3c0 2024-04-20T10:28:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.244+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7ff218075b30 con 0x7ff21815d3c0 2024-04-20T10:28:27.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.245+0000 7ff215600640 1 -- 172.21.15.47:0/3626355741 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7ff21007b5c0 con 0x7ff21815d3c0 2024-04-20T10:28:27.245 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1f806c8f0 msgr2=0x7ff1f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1f806c8f0 0x7ff1f806edb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff2180ca000 tx=0x7ff20c008040 comp rx=0 tx=0).stop 2024-04-20T10:28:27.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff21815d3c0 msgr2=0x7ff2181fc6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff21815d3c0 0x7ff2181fc6e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7ff21000a860 tx=0x7ff21806bde0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 shutdown_connections 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff1f806c8f0 0x7ff1f806edb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff21815d3c0 0x7ff2181fc6e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff21815cf40 0x7ff2181fc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 --2- 172.21.15.47:0/3626355741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff200004d40 0x7ff2180b5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 >> 172.21.15.47:0/3626355741 conn(0x7ff2180cffa0 msgr2=0x7ff2180afb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 shutdown_connections 2024-04-20T10:28:27.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.248+0000 7ff21e800640 1 -- 172.21.15.47:0/3626355741 wait complete. 2024-04-20T10:28:27.268 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.481+0000 7f3166c00640 1 Processor -- start 2024-04-20T10:28:27.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.482+0000 7f3166c00640 1 -- start start 2024-04-20T10:28:27.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.483+0000 7f3166c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 0x7f316015f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.483+0000 7f3166c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f316015b5d0 con 0x7f316006ca10 2024-04-20T10:28:27.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.483+0000 7f3166c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f316015fc70 con 0x7f31600d0010 2024-04-20T10:28:27.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.483+0000 7f3166c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f316006e730 con 0x7f316015b1f0 2024-04-20T10:28:27.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.484+0000 7f3166200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f316006ca10 0x7f316006ec10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40322/0 (socket says 172.21.15.47:40322) 2024-04-20T10:28:27.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.484+0000 7f3166200640 1 -- 172.21.15.47:0/2158093604 learned_addr learned my addr 172.21.15.47:0/2158093604 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:27.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.484+0000 7f3165800640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 0x7f316015f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.484+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3092809710 0 0) 0x7f316015b5d0 con 0x7f316006ca10 2024-04-20T10:28:27.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.484+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3148005730 con 0x7f316006ca10 2024-04-20T10:28:27.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3972018702 0 0) 0x7f3148005730 con 0x7f316006ca10 2024-04-20T10:28:27.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 msgr2=0x7f316015f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 0x7f316015f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 >> v1:172.21.15.175:6789/0 conn(0x7f31600d0010 legacy=0x7f31600adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31600ae310 con 0x7f316006ca10 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f3165800640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 0x7f316015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f31580019f0 con 0x7f316006ca10 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 >> v1:172.21.15.47:6789/0 conn(0x7f316006ca10 legacy=0x7f316006ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 0x7f3148005bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3148006180 0x7f3148006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f31480073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f316006e730 con 0x7f3148006f50 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f314800dce0 con 0x7f3148006180 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f314800dee0 con 0x7f316015b1f0 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f3166200640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f31480073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.485+0000 7f3165800640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 0x7f3148005bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.486+0000 7f3164e00640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3148006180 0x7f3148006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.486+0000 7f3166200640 1 -- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 msgr2=0x7f3148005bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.486+0000 7f3166200640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f316015b1f0 0x7f3148005bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.486+0000 7f3166200640 1 -- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3148006180 msgr2=0x7f3148006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.486+0000 7f3166200640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3148006180 0x7f3148006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.486+0000 7f3166200640 1 -- 172.21.15.47:0/2158093604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3148005730 con 0x7f3148006f50 2024-04-20T10:28:27.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.486+0000 7f3164e00640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3148006180 0x7f3148006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.487+0000 7f3166200640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f31480073d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f315800a780 tx=0x7f3148007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e94dc1803a847f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.487+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315800a3d0 con 0x7f3148006f50 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.487+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3158034650 con 0x7f3148006f50 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.487+0000 7f315fe00640 1 -- 172.21.15.47:0/2158093604 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3158034ac0 con 0x7f3148006f50 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.488+0000 7f3166c00640 1 -- 172.21.15.47:0/2158093604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 msgr2=0x7f31480073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.488+0000 7f3166c00640 1 --2- 172.21.15.47:0/2158093604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f31480073d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f315800a780 tx=0x7f3148007a40 comp rx=0 tx=0).stop 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.488+0000 7f3166200640 1 -- 172.21.15.47:0/2158093604 reap_dead start 2024-04-20T10:28:27.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.488+0000 7f3166c00640 1 -- 172.21.15.47:0/2158093604 shutdown_connections 2024-04-20T10:28:27.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.488+0000 7f3166c00640 1 -- 172.21.15.47:0/2158093604 >> 172.21.15.47:0/2158093604 conn(0x7f31600b5270 msgr2=0x7f31600b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:27.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.488+0000 7f3166c00640 1 -- 172.21.15.47:0/2158093604 shutdown_connections 2024-04-20T10:28:27.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.488+0000 7f3166c00640 1 -- 172.21.15.47:0/2158093604 wait complete. 2024-04-20T10:28:27.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.490+0000 7f3166c00640 1 Processor -- start 2024-04-20T10:28:27.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.490+0000 7f3166c00640 1 -- start start 2024-04-20T10:28:27.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.491+0000 7f3166c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f3160099940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.491+0000 7f3166c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316014e740 0x7f3160157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.491+0000 7f3166c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3160157fb0 0x7f3160159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.491+0000 7f3166c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31600d0b90 con 0x7f3148006f50 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.491+0000 7f3166c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31600d08b0 con 0x7f316014e740 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.491+0000 7f3166c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31600d0e70 con 0x7f3160157fb0 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.491+0000 7f3165800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f3160099940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f3160099940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40212/0 (socket says 172.21.15.47:40212) 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 -- 172.21.15.47:0/3477876177 learned_addr learned my addr 172.21.15.47:0/3477876177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3166200640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3160157fb0 0x7f3160159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3164e00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316014e740 0x7f3160157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 -- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3160157fb0 msgr2=0x7f3160159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3160157fb0 0x7f3160159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 -- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316014e740 msgr2=0x7f3160157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316014e740 0x7f3160157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 -- 172.21.15.47:0/3477876177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3160099e80 con 0x7f3148006f50 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3166200640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3160157fb0 0x7f3160159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.492+0000 7f3165800640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f3160099940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3154011a00 tx=0x7f3154011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.493+0000 7f315d600640 1 -- 172.21.15.47:0/3477876177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315400faf0 con 0x7f3148006f50 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.493+0000 7f315d600640 1 -- 172.21.15.47:0/3477876177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f315400fc90 con 0x7f3148006f50 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.493+0000 7f315d600640 1 -- 172.21.15.47:0/3477876177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3154004030 con 0x7f3148006f50 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.493+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3160158430 con 0x7f3148006f50 2024-04-20T10:28:27.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.493+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f316014bbb0 con 0x7f3148006f50 2024-04-20T10:28:27.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.494+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3160075b30 con 0x7f3148006f50 2024-04-20T10:28:27.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.495+0000 7f315d600640 1 -- 172.21.15.47:0/3477876177 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3154012040 con 0x7f3148006f50 2024-04-20T10:28:27.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.496+0000 7f315d600640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f314006c920 0x7f314006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.496+0000 7f315d600640 1 -- 172.21.15.47:0/3477876177 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f315408c3d0 con 0x7f3148006f50 2024-04-20T10:28:27.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.497+0000 7f3164e00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f314006c920 0x7f314006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.498+0000 7f3164e00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f314006c920 0x7f314006ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f316014da60 tx=0x7f314c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.501+0000 7f315d600640 1 -- 172.21.15.47:0/3477876177 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3160099e80 con 0x7f3148006f50 2024-04-20T10:28:27.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.708+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f31600b1790 con 0x7f3148006f50 2024-04-20T10:28:27.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.709+0000 7f315d600640 1 -- 172.21.15.47:0/3477876177 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f3154019070 con 0x7f3148006f50 2024-04-20T10:28:27.709 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.712+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f314006c920 msgr2=0x7f314006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.712+0000 7f3166c00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f314006c920 0x7f314006ede0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f316014da60 tx=0x7f314c009040 comp rx=0 tx=0).stop 2024-04-20T10:28:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.712+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 msgr2=0x7f3160099940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.712+0000 7f3166c00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f3160099940 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3154011a00 tx=0x7f3154011ec0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 shutdown_connections 2024-04-20T10:28:27.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3160157fb0 0x7f3160159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f316014e740 0x7f3160157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f314006c920 0x7f314006ede0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 --2- 172.21.15.47:0/3477876177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3148006f50 0x7f3160099940 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 >> 172.21.15.47:0/3477876177 conn(0x7f31600b5270 msgr2=0x7f31600738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:27.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 shutdown_connections 2024-04-20T10:28:27.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.713+0000 7f3166c00640 1 -- 172.21.15.47:0/3477876177 wait complete. 2024-04-20T10:28:27.733 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.947+0000 7f31a8400640 1 Processor -- start 2024-04-20T10:28:27.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.947+0000 7f31a8400640 1 -- start start 2024-04-20T10:28:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.949+0000 7f31a8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a006d510 0x7f31a00b3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.949+0000 7f31a8400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f31a0163060 0x7f31a00ce9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.949+0000 7f31a8400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f31a00b1c60 con 0x7f31a006d8f0 2024-04-20T10:28:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.949+0000 7f31a8400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f31a00b2d00 con 0x7f31a0163060 2024-04-20T10:28:27.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.949+0000 7f31a8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31a00b1f40 con 0x7f31a006d510 2024-04-20T10:28:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a006d510 0x7f31a00b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a006d510 0x7f31a00b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:51634/0 (socket says 172.21.15.47:51634) 2024-04-20T10:28:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5a00640 1 -- 172.21.15.47:0/2332256998 learned_addr learned my addr 172.21.15.47:0/2332256998 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5000640 1 --2- 172.21.15.47:0/2332256998 >> v2:172.21.15.175:3300/0 conn(0x7f31a0163060 0x7f31a00ce9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5a00640 1 -- 172.21.15.47:0/2332256998 >> v2:172.21.15.175:3300/0 conn(0x7f31a0163060 msgr2=0x7f31a00ce9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5a00640 1 --2- 172.21.15.47:0/2332256998 >> v2:172.21.15.175:3300/0 conn(0x7f31a0163060 0x7f31a00ce9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5a00640 1 -- 172.21.15.47:0/2332256998 >> v1:172.21.15.175:6789/0 conn(0x7f31a006d8f0 legacy=0x7f31a00cef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5a00640 1 -- 172.21.15.47:0/2332256998 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a00b2220 con 0x7f31a006d510 2024-04-20T10:28:27.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.950+0000 7f31a5000640 1 --2- 172.21.15.47:0/2332256998 >> v2:172.21.15.175:3300/0 conn(0x7f31a0163060 0x7f31a00ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:28:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.952+0000 7f31a5a00640 1 --2- 172.21.15.47:0/2332256998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a006d510 0x7f31a00b3810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3190009e50 tx=0x7f319002bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d3b3b4c4278afd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.952+0000 7f319fe00640 1 -- 172.21.15.47:0/2332256998 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f319002ca70 con 0x7f31a006d510 2024-04-20T10:28:27.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f319fe00640 1 -- 172.21.15.47:0/2332256998 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f319002cc10 con 0x7f31a006d510 2024-04-20T10:28:27.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f319fe00640 1 -- 172.21.15.47:0/2332256998 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31900343f0 con 0x7f31a006d510 2024-04-20T10:28:27.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f31a8400640 1 -- 172.21.15.47:0/2332256998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a006d510 msgr2=0x7f31a00b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f31a8400640 1 --2- 172.21.15.47:0/2332256998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a006d510 0x7f31a00b3810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3190009e50 tx=0x7f319002bed0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f31a8400640 1 -- 172.21.15.47:0/2332256998 shutdown_connections 2024-04-20T10:28:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f31a8400640 1 --2- 172.21.15.47:0/2332256998 >> v2:172.21.15.175:3300/0 conn(0x7f31a0163060 0x7f31a00ce9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f31a8400640 1 --2- 172.21.15.47:0/2332256998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a006d510 0x7f31a00b3810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.953+0000 7f31a8400640 1 -- 172.21.15.47:0/2332256998 >> 172.21.15.47:0/2332256998 conn(0x7f31a0071800 msgr2=0x7f31a0071c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.954+0000 7f31a8400640 1 -- 172.21.15.47:0/2332256998 shutdown_connections 2024-04-20T10:28:27.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.954+0000 7f31a8400640 1 -- 172.21.15.47:0/2332256998 wait complete. 2024-04-20T10:28:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.956+0000 7f31a8400640 1 Processor -- start 2024-04-20T10:28:27.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.956+0000 7f31a8400640 1 -- start start 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.957+0000 7f31a8400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006d510 0x7f31a00c47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.957+0000 7f31a8400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a006d8f0 0x7f31a00c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.957+0000 7f31a8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0163060 0x7f31a00c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.957+0000 7f31a8400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f31a00cf630 con 0x7f31a006d8f0 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.957+0000 7f31a8400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31a00b27e0 con 0x7f31a006d510 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.957+0000 7f31a8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31a0142f30 con 0x7f31a0163060 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a006d8f0 0x7f31a00c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a006d8f0 0x7f31a00c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40228/0 (socket says 172.21.15.47:40228) 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 -- 172.21.15.47:0/4009284885 learned_addr learned my addr 172.21.15.47:0/4009284885 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5a00640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006d510 0x7f31a00c47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a6400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0163060 0x7f31a00c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 -- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0163060 msgr2=0x7f31a00c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0163060 0x7f31a00c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 -- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006d510 msgr2=0x7f31a00c47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006d510 0x7f31a00c47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a5000640 1 -- 172.21.15.47:0/4009284885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31a00c9e50 con 0x7f31a006d8f0 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.958+0000 7f31a6400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0163060 0x7f31a00c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.959+0000 7f31a5000640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a006d8f0 0x7f31a00c9910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f318c00e960 tx=0x7f318c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.959+0000 7f319d600640 1 -- 172.21.15.47:0/4009284885 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c017070 con 0x7f31a006d8f0 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.959+0000 7f319d600640 1 -- 172.21.15.47:0/4009284885 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f318c00ca00 con 0x7f31a006d8f0 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.959+0000 7f319d600640 1 -- 172.21.15.47:0/4009284885 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f318c00cdd0 con 0x7f31a006d8f0 2024-04-20T10:28:27.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.959+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31a00d29d0 con 0x7f31a006d8f0 2024-04-20T10:28:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.959+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31a00d2f90 con 0x7f31a006d8f0 2024-04-20T10:28:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.960+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31a0068980 con 0x7f31a006d8f0 2024-04-20T10:28:27.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.961+0000 7f319d600640 1 -- 172.21.15.47:0/4009284885 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f318c01a070 con 0x7f31a006d8f0 2024-04-20T10:28:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.962+0000 7f319d600640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318006c8f0 0x7f318006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:27.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.962+0000 7f319d600640 1 -- 172.21.15.47:0/4009284885 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f318c08bd40 con 0x7f31a006d8f0 2024-04-20T10:28:27.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.963+0000 7f31a5a00640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318006c8f0 0x7f318006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:27.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.964+0000 7f31a5a00640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318006c8f0 0x7f318006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3190035000 tx=0x7f3190030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:27.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:27.968+0000 7f319d600640 1 -- 172.21.15.47:0/4009284885 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f318c059760 con 0x7f31a006d8f0 2024-04-20T10:28:28.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.175+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31a00b35f0 con 0x7f31a006d8f0 2024-04-20T10:28:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.177+0000 7f319d600640 1 -- 172.21.15.47:0/4009284885 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f318c05da60 con 0x7f31a006d8f0 2024-04-20T10:28:28.177 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:28.178 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:28.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318006c8f0 msgr2=0x7f318006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318006c8f0 0x7f318006edb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3190035000 tx=0x7f3190030000 comp rx=0 tx=0).stop 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a006d8f0 msgr2=0x7f31a00c9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a006d8f0 0x7f31a00c9910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f318c00e960 tx=0x7f318c00ee20 comp rx=0 tx=0).stop 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 shutdown_connections 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31a0163060 0x7f31a00c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f318006c8f0 0x7f318006edb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f31a006d8f0 0x7f31a00c9910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 --2- 172.21.15.47:0/4009284885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31a006d510 0x7f31a00c47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 >> 172.21.15.47:0/4009284885 conn(0x7f31a0071800 msgr2=0x7f31a015ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 shutdown_connections 2024-04-20T10:28:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.180+0000 7f31a8400640 1 -- 172.21.15.47:0/4009284885 wait complete. 2024-04-20T10:28:28.201 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:28.202 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:28.233 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:28.252 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:28.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.406+0000 7fddc3c00640 1 Processor -- start 2024-04-20T10:28:28.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.406+0000 7fddc3c00640 1 -- start start 2024-04-20T10:28:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.408+0000 7fddc3c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc15ca40 0x7fddbc0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.408+0000 7fddc3c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fddbc0ad8e0 0x7fddbc0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.408+0000 7fddc3c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fddbc141c60 con 0x7fddbc156c20 2024-04-20T10:28:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.408+0000 7fddc3c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fddbc06c010 con 0x7fddbc0ad8e0 2024-04-20T10:28:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.408+0000 7fddc3c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc15cf90 con 0x7fddbc15ca40 2024-04-20T10:28:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.408+0000 7fddc1c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fddbc0ad8e0 0x7fddbc0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddc1200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fddbc156c20 0x7fddbc06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40324/0 (socket says 172.21.15.47:40324) 2024-04-20T10:28:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddc1200640 1 -- 172.21.15.47:0/2273350405 learned_addr learned my addr 172.21.15.47:0/2273350405 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddbbe00640 1 --2- 172.21.15.47:0/2273350405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc15ca40 0x7fddbc0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddc1c00640 1 -- 172.21.15.47:0/2273350405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc15ca40 msgr2=0x7fddbc0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddc1c00640 1 --2- 172.21.15.47:0/2273350405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc15ca40 0x7fddbc0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddc1c00640 1 -- 172.21.15.47:0/2273350405 >> v1:172.21.15.47:6789/0 conn(0x7fddbc156c20 legacy=0x7fddbc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddc1c00640 1 -- 172.21.15.47:0/2273350405 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc15d270 con 0x7fddbc0ad8e0 2024-04-20T10:28:28.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.409+0000 7fddbb400640 1 -- 172.21.15.47:0/2273350405 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2785229619 0 0) 0x7fddbc141c60 con 0x7fddbc156c20 2024-04-20T10:28:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.410+0000 7fddc1c00640 1 --2- 172.21.15.47:0/2273350405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc0ad8e0 0x7fddbc0d0360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddb40097a0 tx=0x7fddb402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97a9021952f00efb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddbb400640 1 -- 172.21.15.47:0/2273350405 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4035020 con 0x7fddbc0ad8e0 2024-04-20T10:28:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddbb400640 1 -- 172.21.15.47:0/2273350405 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb4031810 con 0x7fddbc0ad8e0 2024-04-20T10:28:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddbb400640 1 -- 172.21.15.47:0/2273350405 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4031be0 con 0x7fddbc0ad8e0 2024-04-20T10:28:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddc3c00640 1 -- 172.21.15.47:0/2273350405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc0ad8e0 msgr2=0x7fddbc0d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddc3c00640 1 --2- 172.21.15.47:0/2273350405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc0ad8e0 0x7fddbc0d0360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fddb40097a0 tx=0x7fddb402b8b0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddc3c00640 1 -- 172.21.15.47:0/2273350405 shutdown_connections 2024-04-20T10:28:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddc3c00640 1 --2- 172.21.15.47:0/2273350405 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc0ad8e0 0x7fddbc0d0360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddc3c00640 1 --2- 172.21.15.47:0/2273350405 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc15ca40 0x7fddbc0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddc3c00640 1 -- 172.21.15.47:0/2273350405 >> 172.21.15.47:0/2273350405 conn(0x7fddbc0b3350 msgr2=0x7fddbc0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.411+0000 7fddc3c00640 1 -- 172.21.15.47:0/2273350405 shutdown_connections 2024-04-20T10:28:28.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.412+0000 7fddc3c00640 1 -- 172.21.15.47:0/2273350405 wait complete. 2024-04-20T10:28:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.413+0000 7fddc3c00640 1 Processor -- start 2024-04-20T10:28:28.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.414+0000 7fddc3c00640 1 -- start start 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.414+0000 7fddc3c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc0ad8e0 0x7fddbc1f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.414+0000 7fddc3c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddbc156c20 0x7fddbc15b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.414+0000 7fddc3c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc15ca40 0x7fddbc15b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.414+0000 7fddc3c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc0d08a0 con 0x7fddbc15ca40 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc3c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fddbc15d830 con 0x7fddbc156c20 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc3c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fddbc0d0a20 con 0x7fddbc0ad8e0 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc15ca40 0x7fddbc15b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc15ca40 0x7fddbc15b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40242/0 (socket says 172.21.15.47:40242) 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 -- 172.21.15.47:0/4172554331 learned_addr learned my addr 172.21.15.47:0/4172554331 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:28.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddbbe00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddbc156c20 0x7fddbc15b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1200640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc0ad8e0 0x7fddbc1f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 -- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc0ad8e0 msgr2=0x7fddbc1f41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc0ad8e0 0x7fddbc1f41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 -- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddbc156c20 msgr2=0x7fddbc15b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddbc156c20 0x7fddbc15b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.415+0000 7fddc1c00640 1 -- 172.21.15.47:0/4172554331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddbc155620 con 0x7fddbc15ca40 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.416+0000 7fddc1c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc15ca40 0x7fddbc15b950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098c0 tx=0x7fddb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.416+0000 7fddb8c00640 1 -- 172.21.15.47:0/4172554331 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4041070 con 0x7fddbc15ca40 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.416+0000 7fddb8c00640 1 -- 172.21.15.47:0/4172554331 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fddb4031e40 con 0x7fddbc15ca40 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.416+0000 7fddb8c00640 1 -- 172.21.15.47:0/4172554331 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb4035040 con 0x7fddbc15ca40 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.416+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddbc1558c0 con 0x7fddbc15ca40 2024-04-20T10:28:28.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.416+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddbc099790 con 0x7fddbc15ca40 2024-04-20T10:28:28.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.418+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddbc068980 con 0x7fddbc15ca40 2024-04-20T10:28:28.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.418+0000 7fddb8c00640 1 -- 172.21.15.47:0/4172554331 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fddb4038020 con 0x7fddbc15ca40 2024-04-20T10:28:28.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.419+0000 7fddb8c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd9806c8f0 0x7fdd9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.419+0000 7fddb8c00640 1 -- 172.21.15.47:0/4172554331 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fddb40ad9d0 con 0x7fddbc15ca40 2024-04-20T10:28:28.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.420+0000 7fddc1200640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd9806c8f0 0x7fdd9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.421+0000 7fddc1200640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd9806c8f0 0x7fdd9806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fddac0012e0 tx=0x7fddac007ed0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:28.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.425+0000 7fddb8c00640 1 -- 172.21.15.47:0/4172554331 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fddb407b400 con 0x7fddbc15ca40 2024-04-20T10:28:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.628+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fddbc074000 con 0x7fddbc15ca40 2024-04-20T10:28:28.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.629+0000 7fddb8c00640 1 -- 172.21.15.47:0/4172554331 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fddbc074000 con 0x7fddbc15ca40 2024-04-20T10:28:28.629 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.631+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd9806c8f0 msgr2=0x7fdd9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.631+0000 7fddc3c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd9806c8f0 0x7fdd9806edb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fddac0012e0 tx=0x7fddac007ed0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc15ca40 msgr2=0x7fddbc15b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc15ca40 0x7fddbc15b950 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fddb40098c0 tx=0x7fddb40046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 shutdown_connections 2024-04-20T10:28:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd9806c8f0 0x7fdd9806edb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fddbc15ca40 0x7fddbc15b950 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddbc156c20 0x7fddbc15b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 --2- 172.21.15.47:0/4172554331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddbc0ad8e0 0x7fddbc1f41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 >> 172.21.15.47:0/4172554331 conn(0x7fddbc0b3350 msgr2=0x7fddbc154e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 shutdown_connections 2024-04-20T10:28:28.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.632+0000 7fddc3c00640 1 -- 172.21.15.47:0/4172554331 wait complete. 2024-04-20T10:28:28.652 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.868+0000 7f2d6e000640 1 Processor -- start 2024-04-20T10:28:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.868+0000 7f2d6e000640 1 -- start start 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.869+0000 7f2d6e000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d6815b1f0 0x7f2d6815f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d6e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d6806cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d6e000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d6815b5d0 con 0x7f2d680d0010 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d6e000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2d6806e730 con 0x7f2d6815b1f0 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d6e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d680ae310 con 0x7f2d6806ca10 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d6cc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d6815b1f0 0x7f2d6815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d67e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d680d0010 0x7f2d680adc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40328/0 (socket says 172.21.15.47:40328) 2024-04-20T10:28:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d67e00640 1 -- 172.21.15.47:0/1147553329 learned_addr learned my addr 172.21.15.47:0/1147553329 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.870+0000 7f2d6d600640 1 --2- 172.21.15.47:0/1147553329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d6806cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.871+0000 7f2d6cc00640 1 -- 172.21.15.47:0/1147553329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 msgr2=0x7f2d6806cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.871+0000 7f2d6cc00640 1 --2- 172.21.15.47:0/1147553329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d6806cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.871+0000 7f2d6cc00640 1 -- 172.21.15.47:0/1147553329 >> v1:172.21.15.47:6789/0 conn(0x7f2d680d0010 legacy=0x7f2d680adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.871+0000 7f2d6cc00640 1 -- 172.21.15.47:0/1147553329 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d6815fc70 con 0x7f2d6815b1f0 2024-04-20T10:28:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.871+0000 7f2d67400640 1 -- 172.21.15.47:0/1147553329 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4204327311 0 0) 0x7f2d6815b5d0 con 0x7f2d680d0010 2024-04-20T10:28:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.872+0000 7f2d6cc00640 1 --2- 172.21.15.47:0/1147553329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d6815f730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d580097d0 tx=0x7f2d5802b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9211710425e21887 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.872+0000 7f2d67400640 1 -- 172.21.15.47:0/1147553329 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58031770 con 0x7f2d6815b1f0 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.872+0000 7f2d67400640 1 -- 172.21.15.47:0/1147553329 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d58037070 con 0x7f2d6815b1f0 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.872+0000 7f2d67400640 1 -- 172.21.15.47:0/1147553329 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58031b30 con 0x7f2d6815b1f0 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 -- 172.21.15.47:0/1147553329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 msgr2=0x7f2d6815f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 --2- 172.21.15.47:0/1147553329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d6815f730 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2d580097d0 tx=0x7f2d5802b810 comp rx=0 tx=0).stop 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 -- 172.21.15.47:0/1147553329 shutdown_connections 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 --2- 172.21.15.47:0/1147553329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d6806cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 --2- 172.21.15.47:0/1147553329 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d6815f730 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 -- 172.21.15.47:0/1147553329 >> 172.21.15.47:0/1147553329 conn(0x7f2d680b5270 msgr2=0x7f2d680b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:28.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 -- 172.21.15.47:0/1147553329 shutdown_connections 2024-04-20T10:28:28.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.873+0000 7f2d6e000640 1 -- 172.21.15.47:0/1147553329 wait complete. 2024-04-20T10:28:28.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.875+0000 7f2d6e000640 1 Processor -- start 2024-04-20T10:28:28.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.875+0000 7f2d6e000640 1 -- start start 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d680992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680d0010 0x7f2d681509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d68150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6e000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2d680d0b90 con 0x7f2d6815b1f0 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d680d08b0 con 0x7f2d680d0010 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d680d0e70 con 0x7f2d6806ca10 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d68150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d68150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40258/0 (socket says 172.21.15.47:40258) 2024-04-20T10:28:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 -- 172.21.15.47:0/2769902380 learned_addr learned my addr 172.21.15.47:0/2769902380 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d67e00640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680d0010 0x7f2d681509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6cc00640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d680992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 -- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 msgr2=0x7f2d680992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d680992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 -- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680d0010 msgr2=0x7f2d681509b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680d0010 0x7f2d681509b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.876+0000 7f2d6d600640 1 -- 172.21.15.47:0/2769902380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d681449e0 con 0x7f2d6815b1f0 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.877+0000 7f2d6cc00640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d680992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.877+0000 7f2d6d600640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d68150ef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d60007e60 tx=0x7f2d6000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.877+0000 7f2d64c00640 1 -- 172.21.15.47:0/2769902380 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60014420 con 0x7f2d6815b1f0 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.877+0000 7f2d64c00640 1 -- 172.21.15.47:0/2769902380 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d60019070 con 0x7f2d6815b1f0 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.877+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d68144700 con 0x7f2d6815b1f0 2024-04-20T10:28:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.877+0000 7f2d64c00640 1 -- 172.21.15.47:0/2769902380 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60014820 con 0x7f2d6815b1f0 2024-04-20T10:28:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.877+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6814ac50 con 0x7f2d6815b1f0 2024-04-20T10:28:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.878+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d680afa90 con 0x7f2d6815b1f0 2024-04-20T10:28:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.879+0000 7f2d64c00640 1 -- 172.21.15.47:0/2769902380 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d600149c0 con 0x7f2d6815b1f0 2024-04-20T10:28:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.880+0000 7f2d64c00640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4406c8f0 0x7f2d4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.880+0000 7f2d64c00640 1 -- 172.21.15.47:0/2769902380 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f2d6008c780 con 0x7f2d6815b1f0 2024-04-20T10:28:28.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.881+0000 7f2d6cc00640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4406c8f0 0x7f2d4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:28.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.882+0000 7f2d6cc00640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4406c8f0 0x7f2d4406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d58036000 tx=0x7f2d58008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:28.885+0000 7f2d64c00640 1 -- 172.21.15.47:0/2769902380 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2d6005a800 con 0x7f2d6815b1f0 2024-04-20T10:28:29.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.096+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2d68075b30 con 0x7f2d6815b1f0 2024-04-20T10:28:29.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.097+0000 7f2d64c00640 1 -- 172.21.15.47:0/2769902380 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f2d6005a100 con 0x7f2d6815b1f0 2024-04-20T10:28:29.097 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4406c8f0 msgr2=0x7f2d4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4406c8f0 0x7f2d4406edb0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f2d58036000 tx=0x7f2d58008030 comp rx=0 tx=0).stop 2024-04-20T10:28:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 msgr2=0x7f2d68150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d68150ef0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2d60007e60 tx=0x7f2d6000e4b0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 shutdown_connections 2024-04-20T10:28:29.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4406c8f0 0x7f2d4406edb0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d6815b1f0 0x7f2d68150ef0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680d0010 0x7f2d681509b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 --2- 172.21.15.47:0/2769902380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d6806ca10 0x7f2d680992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 >> 172.21.15.47:0/2769902380 conn(0x7f2d680b5270 msgr2=0x7f2d6806f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 shutdown_connections 2024-04-20T10:28:29.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.100+0000 7f2d6e000640 1 -- 172.21.15.47:0/2769902380 wait complete. 2024-04-20T10:28:29.118 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:29.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.330+0000 7f8aa3800640 1 Processor -- start 2024-04-20T10:28:29.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.331+0000 7f8aa3800640 1 -- start start 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.332+0000 7f8aa3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4270 0x7f8a9c0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.332+0000 7f8aa3800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a9c0d4650 0x7f8a9c0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.332+0000 7f8aa3800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a9c0d3820 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.332+0000 7f8aa3800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8a9c0d40c0 con 0x7f8a9c0d4650 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.332+0000 7f8aa3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c0d3b00 con 0x7f8a9c0d4270 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.332+0000 7f8a9be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8a9c0d4650 0x7f8a9c0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.333+0000 7f8aa1800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8a9c0b17d0 0x7f8a9c0d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40336/0 (socket says 172.21.15.47:40336) 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.333+0000 7f8aa1800640 1 -- 172.21.15.47:0/656633874 learned_addr learned my addr 172.21.15.47:0/656633874 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:29.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.333+0000 7f8aa0e00640 1 --2- 172.21.15.47:0/656633874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4270 0x7f8a9c0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.333+0000 7f8a9be00640 1 -- 172.21.15.47:0/656633874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4270 msgr2=0x7f8a9c0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:29.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.333+0000 7f8a9be00640 1 --2- 172.21.15.47:0/656633874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4270 0x7f8a9c0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.333+0000 7f8a9be00640 1 -- 172.21.15.47:0/656633874 >> v1:172.21.15.47:6789/0 conn(0x7f8a9c0b17d0 legacy=0x7f8a9c0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.333+0000 7f8a9be00640 1 -- 172.21.15.47:0/656633874 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c0d3de0 con 0x7f8a9c0d4650 2024-04-20T10:28:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.334+0000 7f8a9be00640 1 --2- 172.21.15.47:0/656633874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0d4650 0x7f8a9c0b1290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8a880097a0 tx=0x7f8a8802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b177f57baf6f002 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.334+0000 7f8a9b400640 1 -- 172.21.15.47:0/656633874 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88035020 con 0x7f8a9c0d4650 2024-04-20T10:28:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8a9b400640 1 -- 172.21.15.47:0/656633874 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a88031810 con 0x7f8a9c0d4650 2024-04-20T10:28:29.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8a9b400640 1 -- 172.21.15.47:0/656633874 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88031be0 con 0x7f8a9c0d4650 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8aa3800640 1 -- 172.21.15.47:0/656633874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0d4650 msgr2=0x7f8a9c0b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8aa3800640 1 --2- 172.21.15.47:0/656633874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0d4650 0x7f8a9c0b1290 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8a880097a0 tx=0x7f8a8802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8aa3800640 1 -- 172.21.15.47:0/656633874 shutdown_connections 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8aa3800640 1 --2- 172.21.15.47:0/656633874 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0d4650 0x7f8a9c0b1290 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8aa3800640 1 --2- 172.21.15.47:0/656633874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4270 0x7f8a9c0d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.335+0000 7f8aa3800640 1 -- 172.21.15.47:0/656633874 >> 172.21.15.47:0/656633874 conn(0x7f8a9c0b99c0 msgr2=0x7f8a9c0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.336+0000 7f8aa3800640 1 -- 172.21.15.47:0/656633874 shutdown_connections 2024-04-20T10:28:29.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.336+0000 7f8aa3800640 1 -- 172.21.15.47:0/656633874 wait complete. 2024-04-20T10:28:29.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.337+0000 7f8aa3800640 1 Processor -- start 2024-04-20T10:28:29.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.338+0000 7f8aa3800640 1 -- start start 2024-04-20T10:28:29.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.338+0000 7f8aa3800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0b17d0 0x7f8a9c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.338+0000 7f8aa3800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a9c0d4270 0x7f8a9c159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.338+0000 7f8aa3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4650 0x7f8a9c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa3800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c161f70 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa3800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a9c0d53b0 con 0x7f8a9c0d4270 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a9c1620f0 con 0x7f8a9c0d4650 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0b17d0 0x7f8a9c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0b17d0 0x7f8a9c159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40278/0 (socket says 172.21.15.47:40278) 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 -- 172.21.15.47:0/894925601 learned_addr learned my addr 172.21.15.47:0/894925601 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 -- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4650 msgr2=0x7f8a9c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4650 0x7f8a9c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 -- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a9c0d4270 msgr2=0x7f8a9c159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a9c0d4270 0x7f8a9c159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.339+0000 7f8aa0e00640 1 -- 172.21.15.47:0/894925601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a9c1f5000 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.340+0000 7f8aa0e00640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0b17d0 0x7f8a9c159c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000d830 tx=0x7f8a9000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.340+0000 7f8a98c00640 1 -- 172.21.15.47:0/894925601 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000b6c0 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.340+0000 7f8a98c00640 1 -- 172.21.15.47:0/894925601 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a90019070 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.340+0000 7f8a98c00640 1 -- 172.21.15.47:0/894925601 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9000ba80 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.340+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a9c1f52e0 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.340+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a9c0cc650 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.342+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a9c068980 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.343+0000 7f8a98c00640 1 -- 172.21.15.47:0/894925601 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a9001e460 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.344+0000 7f8a98c00640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.344+0000 7f8a98c00640 1 -- 172.21.15.47:0/894925601 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f8a9008c650 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.345+0000 7f8a9be00640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.346+0000 7f8a9be00640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a88036000 tx=0x7f8a88030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:29.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.349+0000 7f8a98c00640 1 -- 172.21.15.47:0/894925601 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a9005a6d0 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.556+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8a9c15a1d0 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.557+0000 7f8a98c00640 1 -- 172.21.15.47:0/894925601 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f8a90059fd0 con 0x7f8a9c0b17d0 2024-04-20T10:28:29.557 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:29.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 msgr2=0x7f8a7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a88036000 tx=0x7f8a88030000 comp rx=0 tx=0).stop 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0b17d0 msgr2=0x7f8a9c159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0b17d0 0x7f8a9c159c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000d830 tx=0x7f8a9000dcf0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 shutdown_connections 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a9c0d4650 0x7f8a9c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a9c0d4270 0x7f8a9c159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 --2- 172.21.15.47:0/894925601 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a9c0b17d0 0x7f8a9c159c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 >> 172.21.15.47:0/894925601 conn(0x7f8a9c0b99c0 msgr2=0x7f8a9c0b1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.560+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 shutdown_connections 2024-04-20T10:28:29.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.561+0000 7f8aa3800640 1 -- 172.21.15.47:0/894925601 wait complete. 2024-04-20T10:28:29.580 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.786+0000 7fa117200640 1 Processor -- start 2024-04-20T10:28:29.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.786+0000 7fa117200640 1 -- start start 2024-04-20T10:28:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa117200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1180c8560 0x7fa1180cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa117200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa1180c8940 0x7fa1180cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa117200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa1180d0da0 con 0x7fa1180d0050 2024-04-20T10:28:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa117200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa1180d19b0 con 0x7fa1180c8940 2024-04-20T10:28:29.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa117200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1180c6d40 con 0x7fa1180c8560 2024-04-20T10:28:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa116800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa1180d0050 0x7fa1180d04e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40346/0 (socket says 172.21.15.47:40346) 2024-04-20T10:28:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa116800640 1 -- 172.21.15.47:0/1279171696 learned_addr learned my addr 172.21.15.47:0/1279171696 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:29.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa115e00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1180c8560 0x7fa1180cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.788+0000 7fa115400640 1 --2- 172.21.15.47:0/1279171696 >> v2:172.21.15.175:3300/0 conn(0x7fa1180c8940 0x7fa1180cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 664852001 0 0) 0x7fa1180d0da0 con 0x7fa1180d0050 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa0fc003520 con 0x7fa1180d0050 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3544007769 0 0) 0x7fa0fc003520 con 0x7fa1180d0050 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1180c8560 msgr2=0x7fa1180cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1180c8560 0x7fa1180cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 >> v2:172.21.15.175:3300/0 conn(0x7fa1180c8940 msgr2=0x7fa1180cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 --2- 172.21.15.47:0/1279171696 >> v2:172.21.15.175:3300/0 conn(0x7fa1180c8940 0x7fa1180cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1180d1080 con 0x7fa1180d0050 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fa1100019f0 con 0x7fa1180d0050 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 >> v1:172.21.15.47:6789/0 conn(0x7fa1180d0050 legacy=0x7fa1180d04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc0039b0 0x7fa0fc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0fc004800 0x7fa0fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1180c8560 0x7fa0fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa1180d19b0 con 0x7fa0fc0039b0 2024-04-20T10:28:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1180c6d40 con 0x7fa0fc004800 2024-04-20T10:28:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.789+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa0fc00b9c0 con 0x7fa1180c8560 2024-04-20T10:28:29.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115e00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc0039b0 0x7fa0fc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115400640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0fc004800 0x7fa0fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115e00640 1 -- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1180c8560 msgr2=0x7fa0fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115e00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1180c8560 0x7fa0fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115e00640 1 -- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0fc004800 msgr2=0x7fa0fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115e00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0fc004800 0x7fa0fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115e00640 1 -- 172.21.15.47:0/1279171696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc003520 con 0x7fa0fc0039b0 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.790+0000 7fa115400640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0fc004800 0x7fa0fc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.791+0000 7fa115e00640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc0039b0 0x7fa0fc003dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa108009ac0 tx=0x7fa0fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=653d19e107e94e21 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.791+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108039070 con 0x7fa0fc0039b0 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.791+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10802c710 con 0x7fa0fc0039b0 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.791+0000 7fa114a00640 1 -- 172.21.15.47:0/1279171696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa108009690 con 0x7fa0fc0039b0 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.792+0000 7fa117200640 1 -- 172.21.15.47:0/1279171696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc0039b0 msgr2=0x7fa0fc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.792+0000 7fa117200640 1 --2- 172.21.15.47:0/1279171696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0fc0039b0 0x7fa0fc003dd0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa108009ac0 tx=0x7fa0fc005830 comp rx=0 tx=0).stop 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.792+0000 7fa116800640 1 -- 172.21.15.47:0/1279171696 reap_dead start 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.792+0000 7fa117200640 1 -- 172.21.15.47:0/1279171696 shutdown_connections 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.792+0000 7fa117200640 1 -- 172.21.15.47:0/1279171696 >> 172.21.15.47:0/1279171696 conn(0x7fa1180b3d10 msgr2=0x7fa1180b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.792+0000 7fa117200640 1 -- 172.21.15.47:0/1279171696 shutdown_connections 2024-04-20T10:28:29.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.792+0000 7fa117200640 1 -- 172.21.15.47:0/1279171696 wait complete. 2024-04-20T10:28:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.794+0000 7fa117200640 1 Processor -- start 2024-04-20T10:28:29.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.794+0000 7fa117200640 1 -- start start 2024-04-20T10:28:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.795+0000 7fa117200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0fc0039b0 0x7fa1180cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.795+0000 7fa117200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180cb450 0x7fa1181e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.795+0000 7fa117200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1181e7af0 0x7fa1181e7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.795+0000 7fa117200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa1180d1e40 con 0x7fa1180cb450 2024-04-20T10:28:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.795+0000 7fa117200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1180d1640 con 0x7fa1181e7af0 2024-04-20T10:28:29.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.795+0000 7fa117200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1180cebf0 con 0x7fa0fc0039b0 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180cb450 0x7fa1181e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180cb450 0x7fa1181e5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40292/0 (socket says 172.21.15.47:40292) 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 -- 172.21.15.47:0/450133718 learned_addr learned my addr 172.21.15.47:0/450133718 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115e00640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0fc0039b0 0x7fa1180cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 -- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0fc0039b0 msgr2=0x7fa1180cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0fc0039b0 0x7fa1180cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa116800640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1181e7af0 0x7fa1181e7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 -- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1181e7af0 msgr2=0x7fa1181e7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1181e7af0 0x7fa1181e7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 -- 172.21.15.47:0/450133718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1181e8970 con 0x7fa1180cb450 2024-04-20T10:28:29.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.796+0000 7fa115400640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180cb450 0x7fa1181e5ec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1000109d0 tx=0x7fa100010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.797+0000 7fa10e000640 1 -- 172.21.15.47:0/450133718 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10000f040 con 0x7fa1180cb450 2024-04-20T10:28:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.797+0000 7fa10e000640 1 -- 172.21.15.47:0/450133718 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa10000ec20 con 0x7fa1180cb450 2024-04-20T10:28:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.797+0000 7fa10e000640 1 -- 172.21.15.47:0/450133718 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa10001c400 con 0x7fa1180cb450 2024-04-20T10:28:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.797+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1181e8c50 con 0x7fa1180cb450 2024-04-20T10:28:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.797+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1180cb8b0 con 0x7fa1180cb450 2024-04-20T10:28:29.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.798+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa118075b30 con 0x7fa1180cb450 2024-04-20T10:28:29.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.799+0000 7fa10e000640 1 -- 172.21.15.47:0/450133718 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa10001c5a0 con 0x7fa1180cb450 2024-04-20T10:28:29.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.800+0000 7fa10e000640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:29.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.801+0000 7fa10e000640 1 -- 172.21.15.47:0/450133718 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fa10008c5b0 con 0x7fa1180cb450 2024-04-20T10:28:29.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.801+0000 7fa115e00640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:29.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.802+0000 7fa115e00640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa108036000 tx=0x7fa108030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:29.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:29.805+0000 7fa10e000640 1 -- 172.21.15.47:0/450133718 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa100018030 con 0x7fa1180cb450 2024-04-20T10:28:30.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.007+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fa1180ce180 con 0x7fa1180cb450 2024-04-20T10:28:30.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.008+0000 7fa10e000640 1 -- 172.21.15.47:0/450133718 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fa10005a620 con 0x7fa1180cb450 2024-04-20T10:28:30.008 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 msgr2=0x7fa0f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa108036000 tx=0x7fa108030000 comp rx=0 tx=0).stop 2024-04-20T10:28:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180cb450 msgr2=0x7fa1181e5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180cb450 0x7fa1181e5ec0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa1000109d0 tx=0x7fa100010e90 comp rx=0 tx=0).stop 2024-04-20T10:28:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 shutdown_connections 2024-04-20T10:28:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1181e7af0 0x7fa1181e7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0f406c8f0 0x7fa0f406edb0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa1180cb450 0x7fa1181e5ec0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 --2- 172.21.15.47:0/450133718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0fc0039b0 0x7fa1180cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 >> 172.21.15.47:0/450133718 conn(0x7fa1180b3d10 msgr2=0x7fa1180d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 shutdown_connections 2024-04-20T10:28:30.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.011+0000 7fa117200640 1 -- 172.21.15.47:0/450133718 wait complete. 2024-04-20T10:28:30.031 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.242+0000 7f57b7200640 1 Processor -- start 2024-04-20T10:28:30.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.243+0000 7f57b7200640 1 -- start start 2024-04-20T10:28:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.244+0000 7f57b7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57b006cde0 0x7f57b00d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.244+0000 7f57b7200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b006c190 con 0x7f57b0161d50 2024-04-20T10:28:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.244+0000 7f57b7200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57b006c470 con 0x7f57b00b3330 2024-04-20T10:28:30.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.244+0000 7f57b7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57b0162130 con 0x7f57b006cde0 2024-04-20T10:28:30.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.245+0000 7f57b5e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f57b0161d50 0x7f57b006c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40360/0 (socket says 172.21.15.47:40360) 2024-04-20T10:28:30.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.245+0000 7f57b5e00640 1 -- 172.21.15.47:0/2184713323 learned_addr learned my addr 172.21.15.47:0/2184713323 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:30.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.245+0000 7f57b5400640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57b006cde0 0x7f57b00d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.245+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 509625963 0 0) 0x7f57b006c190 con 0x7f57b0161d50 2024-04-20T10:28:30.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.245+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5798003520 con 0x7f57b0161d50 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2941307951 0 0) 0x7f5798003520 con 0x7f57b0161d50 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57b006cde0 msgr2=0x7f57b00d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57b006cde0 0x7f57b00d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 >> v1:172.21.15.175:6789/0 conn(0x7f57b00b3330 legacy=0x7f57b00b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b00d2e00 con 0x7f57b0161d50 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f57a4001670 con 0x7f57b0161d50 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 >> v1:172.21.15.47:6789/0 conn(0x7f57b0161d50 legacy=0x7f57b006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57b006cde0 0x7f57980039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5798003f60 0x7f5798004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798004d40 0x7f57980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f579800ba70 con 0x7f5798004d40 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f579800bc20 con 0x7f5798003f60 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f579800be20 con 0x7f57b006cde0 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b6800640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798004d40 0x7f57980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.246+0000 7f57b5400640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5798003f60 0x7f5798004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.247+0000 7f57b6800640 1 -- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57b006cde0 msgr2=0x7f57980039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.247+0000 7f57b6800640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57b006cde0 0x7f57980039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.247+0000 7f57b6800640 1 -- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5798003f60 msgr2=0x7f5798004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.247+0000 7f57b6800640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5798003f60 0x7f5798004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.247+0000 7f57b6800640 1 -- 172.21.15.47:0/2184713323 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5798003520 con 0x7f5798004d40 2024-04-20T10:28:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.248+0000 7f57b6800640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798004d40 0x7f57980051c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f57ac001e50 tx=0x7f5798005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eed88131d6fa8c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.248+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac001a70 con 0x7f5798004d40 2024-04-20T10:28:30.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.248+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ac003b70 con 0x7f5798004d40 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.248+0000 7f57b4a00640 1 -- 172.21.15.47:0/2184713323 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ac0050d0 con 0x7f5798004d40 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.249+0000 7f57b7200640 1 -- 172.21.15.47:0/2184713323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798004d40 msgr2=0x7f57980051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.249+0000 7f57b7200640 1 --2- 172.21.15.47:0/2184713323 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5798004d40 0x7f57980051c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f57ac001e50 tx=0x7f5798005830 comp rx=0 tx=0).stop 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.249+0000 7f57b6800640 1 -- 172.21.15.47:0/2184713323 reap_dead start 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.249+0000 7f57b7200640 1 -- 172.21.15.47:0/2184713323 shutdown_connections 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.249+0000 7f57b7200640 1 -- 172.21.15.47:0/2184713323 >> 172.21.15.47:0/2184713323 conn(0x7f57b00d4450 msgr2=0x7f57b00d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.249+0000 7f57b7200640 1 -- 172.21.15.47:0/2184713323 shutdown_connections 2024-04-20T10:28:30.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.249+0000 7f57b7200640 1 -- 172.21.15.47:0/2184713323 wait complete. 2024-04-20T10:28:30.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.251+0000 7f57b7200640 1 Processor -- start 2024-04-20T10:28:30.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.251+0000 7f57b7200640 1 -- start start 2024-04-20T10:28:30.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5798004d40 0x7f57b0147740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b00b61b0 0x7f57b0159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57b00b6630 0x7f57b0149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f57b00d38e0 con 0x7f57b00b61b0 2024-04-20T10:28:30.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57b00d33c0 con 0x7f57b00b6630 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57b00d3a60 con 0x7f5798004d40 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b00b61b0 0x7f57b0159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b5400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b00b61b0 0x7f57b0159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40310/0 (socket says 172.21.15.47:40310) 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b5400640 1 -- 172.21.15.47:0/2185146552 learned_addr learned my addr 172.21.15.47:0/2185146552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.252+0000 7f57b5e00640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5798004d40 0x7f57b0147740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b5400640 1 -- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5798004d40 msgr2=0x7f57b0147740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b6800640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57b00b6630 0x7f57b0149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b5400640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5798004d40 0x7f57b0147740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b5400640 1 -- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57b00b6630 msgr2=0x7f57b0149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b5400640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57b00b6630 0x7f57b0149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b5400640 1 -- 172.21.15.47:0/2185146552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57b00cadb0 con 0x7f57b00b61b0 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b6800640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57b00b6630 0x7f57b0149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b5400640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b00b61b0 0x7f57b0159c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f57a0010890 tx=0x7f57a0010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:30.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57aa000640 1 -- 172.21.15.47:0/2185146552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a000f040 con 0x7f57b00b61b0 2024-04-20T10:28:30.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57aa000640 1 -- 172.21.15.47:0/2185146552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57a000eb60 con 0x7f57b00b61b0 2024-04-20T10:28:30.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57aa000640 1 -- 172.21.15.47:0/2185146552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57a001c400 con 0x7f57b00b61b0 2024-04-20T10:28:30.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57b015a730 con 0x7f57b00b61b0 2024-04-20T10:28:30.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.253+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57b00cce00 con 0x7f57b00b61b0 2024-04-20T10:28:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.255+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57b0075b60 con 0x7f57b00b61b0 2024-04-20T10:28:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.256+0000 7f57aa000640 1 -- 172.21.15.47:0/2185146552 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57a001c5a0 con 0x7f57b00b61b0 2024-04-20T10:28:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.256+0000 7f57aa000640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f579406c8f0 0x7f579406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.257+0000 7f57aa000640 1 -- 172.21.15.47:0/2185146552 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f57a008c5a0 con 0x7f57b00b61b0 2024-04-20T10:28:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.257+0000 7f57b5e00640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f579406c8f0 0x7f579406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.258+0000 7f57b5e00640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f579406c8f0 0x7f579406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57a40011e0 tx=0x7f57a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:30.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.261+0000 7f57aa000640 1 -- 172.21.15.47:0/2185146552 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f57a0018030 con 0x7f57b00b61b0 2024-04-20T10:28:30.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.472+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57b00cb2e0 con 0x7f57b00b61b0 2024-04-20T10:28:30.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.475+0000 7f57aa000640 1 -- 172.21.15.47:0/2185146552 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f57a005a580 con 0x7f57b00b61b0 2024-04-20T10:28:30.476 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:30.476 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:30.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.478+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f579406c8f0 msgr2=0x7f579406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.478+0000 7f57b7200640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f579406c8f0 0x7f579406edb0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f57a40011e0 tx=0x7f57a4009040 comp rx=0 tx=0).stop 2024-04-20T10:28:30.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.478+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b00b61b0 msgr2=0x7f57b0159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.478+0000 7f57b7200640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b00b61b0 0x7f57b0159c70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f57a0010890 tx=0x7f57a0010d50 comp rx=0 tx=0).stop 2024-04-20T10:28:30.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 shutdown_connections 2024-04-20T10:28:30.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57b00b6630 0x7f57b0149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f579406c8f0 0x7f579406edb0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57b00b61b0 0x7f57b0159c70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 --2- 172.21.15.47:0/2185146552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5798004d40 0x7f57b0147740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 >> 172.21.15.47:0/2185146552 conn(0x7f57b00d4450 msgr2=0x7f57b00b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:30.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 shutdown_connections 2024-04-20T10:28:30.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.479+0000 7f57b7200640 1 -- 172.21.15.47:0/2185146552 wait complete. 2024-04-20T10:28:30.497 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:30.498 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:30.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.708+0000 7f6a01e00640 1 Processor -- start 2024-04-20T10:28:30.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.708+0000 7f6a01e00640 1 -- start start 2024-04-20T10:28:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.710+0000 7f6a01e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69fc15e770 0x7f69fc0b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.710+0000 7f6a01e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69fc0d3590 con 0x7f69fc162210 2024-04-20T10:28:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.710+0000 7f6a01e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69fc0d3870 con 0x7f69fc162670 2024-04-20T10:28:30.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.710+0000 7f6a01e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f69fc162ae0 con 0x7f69fc15e770 2024-04-20T10:28:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.710+0000 7f6a00a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f69fc15e770 0x7f69fc0b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.710+0000 7f69fbe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f69fc162210 0x7f69fc0b6400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40362/0 (socket says 172.21.15.47:40362) 2024-04-20T10:28:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.710+0000 7f69fbe00640 1 -- 172.21.15.47:0/2508343325 learned_addr learned my addr 172.21.15.47:0/2508343325 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.711+0000 7f6a00a00640 1 -- 172.21.15.47:0/2508343325 >> v1:172.21.15.175:6789/0 conn(0x7f69fc162670 legacy=0x7f69fc0d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.711+0000 7f6a00a00640 1 -- 172.21.15.47:0/2508343325 >> v1:172.21.15.47:6789/0 conn(0x7f69fc162210 legacy=0x7f69fc0b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.711+0000 7f6a00a00640 1 -- 172.21.15.47:0/2508343325 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0d3b50 con 0x7f69fc15e770 2024-04-20T10:28:30.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.712+0000 7f6a00a00640 1 --2- 172.21.15.47:0/2508343325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc15e770 0x7f69fc0b3490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69f00097a0 tx=0x7f69f002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15361c96d245e00f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.713+0000 7f69fb400640 1 -- 172.21.15.47:0/2508343325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0035020 con 0x7f69fc15e770 2024-04-20T10:28:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.713+0000 7f69fb400640 1 -- 172.21.15.47:0/2508343325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69f0031830 con 0x7f69fc15e770 2024-04-20T10:28:30.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.713+0000 7f69fb400640 1 -- 172.21.15.47:0/2508343325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f0031c00 con 0x7f69fc15e770 2024-04-20T10:28:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.713+0000 7f6a01e00640 1 -- 172.21.15.47:0/2508343325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc15e770 msgr2=0x7f69fc0b3490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.713+0000 7f6a01e00640 1 --2- 172.21.15.47:0/2508343325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc15e770 0x7f69fc0b3490 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f69f00097a0 tx=0x7f69f002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.714+0000 7f6a01e00640 1 -- 172.21.15.47:0/2508343325 shutdown_connections 2024-04-20T10:28:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.714+0000 7f6a01e00640 1 --2- 172.21.15.47:0/2508343325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc15e770 0x7f69fc0b3490 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.714+0000 7f6a01e00640 1 -- 172.21.15.47:0/2508343325 >> 172.21.15.47:0/2508343325 conn(0x7f69fc09d6d0 msgr2=0x7f69fc0b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.714+0000 7f6a01e00640 1 -- 172.21.15.47:0/2508343325 shutdown_connections 2024-04-20T10:28:30.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.714+0000 7f6a01e00640 1 -- 172.21.15.47:0/2508343325 wait complete. 2024-04-20T10:28:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.715+0000 7f6a01e00640 1 Processor -- start 2024-04-20T10:28:30.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.715+0000 7f6a01e00640 1 -- start start 2024-04-20T10:28:30.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.716+0000 7f6a01e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69fc15e770 0x7f69fc15fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.716+0000 7f6a01e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc162210 0x7f69fc0b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.716+0000 7f6a01e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69fc162670 0x7f69fc1616e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.716+0000 7f6a01e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc0d3e30 con 0x7f69fc162210 2024-04-20T10:28:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.716+0000 7f6a01e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15eb50 con 0x7f69fc162670 2024-04-20T10:28:30.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.716+0000 7f6a01e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc0b6b10 con 0x7f69fc15e770 2024-04-20T10:28:30.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc162210 0x7f69fc0b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc162210 0x7f69fc0b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40326/0 (socket says 172.21.15.47:40326) 2024-04-20T10:28:30.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 -- 172.21.15.47:0/1516837971 learned_addr learned my addr 172.21.15.47:0/1516837971 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:30.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f6a01400640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69fc162670 0x7f69fc1616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f6a00a00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69fc15e770 0x7f69fc15fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 -- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69fc15e770 msgr2=0x7f69fc15fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69fc15e770 0x7f69fc15fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 -- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69fc162670 msgr2=0x7f69fc1616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69fc162670 0x7f69fc1616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f69fbe00640 1 -- 172.21.15.47:0/1516837971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0cb530 con 0x7f69fc162210 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f6a01400640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69fc162670 0x7f69fc1616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.717+0000 7f6a00a00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69fc15e770 0x7f69fc15fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.718+0000 7f69fbe00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc162210 0x7f69fc0b17a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c480 tx=0x7f69e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.718+0000 7f69f8c00640 1 -- 172.21.15.47:0/1516837971 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e8018070 con 0x7f69fc162210 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.718+0000 7f69f8c00640 1 -- 172.21.15.47:0/1516837971 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69e8005ce0 con 0x7f69fc162210 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.718+0000 7f69f8c00640 1 -- 172.21.15.47:0/1516837971 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69e80072d0 con 0x7f69fc162210 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.718+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc0cb810 con 0x7f69fc162210 2024-04-20T10:28:30.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.718+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc0cbdd0 con 0x7f69fc162210 2024-04-20T10:28:30.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.719+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc075b30 con 0x7f69fc162210 2024-04-20T10:28:30.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.720+0000 7f69f8c00640 1 -- 172.21.15.47:0/1516837971 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69e8007470 con 0x7f69fc162210 2024-04-20T10:28:30.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.721+0000 7f69f8c00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69dc06c8f0 0x7f69dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:30.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.721+0000 7f69f8c00640 1 -- 172.21.15.47:0/1516837971 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f69e808b140 con 0x7f69fc162210 2024-04-20T10:28:30.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.722+0000 7f6a00a00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69dc06c8f0 0x7f69dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:30.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.723+0000 7f6a00a00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69dc06c8f0 0x7f69dc06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f69f0036000 tx=0x7f69f0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:30.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.726+0000 7f69f8c00640 1 -- 172.21.15.47:0/1516837971 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f69e8058ae0 con 0x7f69fc162210 2024-04-20T10:28:30.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.931+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f69fc0cb170 con 0x7f69fc162210 2024-04-20T10:28:30.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.932+0000 7f69f8c00640 1 -- 172.21.15.47:0/1516837971 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f69e805cde0 con 0x7f69fc162210 2024-04-20T10:28:30.933 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69dc06c8f0 msgr2=0x7f69dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69dc06c8f0 0x7f69dc06edb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f69f0036000 tx=0x7f69f0008030 comp rx=0 tx=0).stop 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc162210 msgr2=0x7f69fc0b17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc162210 0x7f69fc0b17a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f69e800c480 tx=0x7f69e800c940 comp rx=0 tx=0).stop 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 shutdown_connections 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69fc162670 0x7f69fc1616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69dc06c8f0 0x7f69dc06edb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69fc162210 0x7f69fc0b17a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 --2- 172.21.15.47:0/1516837971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69fc15e770 0x7f69fc15fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 >> 172.21.15.47:0/1516837971 conn(0x7f69fc09d6d0 msgr2=0x7f69fc15c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 shutdown_connections 2024-04-20T10:28:30.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:30.935+0000 7f6a01e00640 1 -- 172.21.15.47:0/1516837971 wait complete. 2024-04-20T10:28:30.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:31.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.163+0000 7f0104600640 1 Processor -- start 2024-04-20T10:28:31.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.163+0000 7f0104600640 1 -- start start 2024-04-20T10:28:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.164+0000 7f0104600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc15e8a0 0x7f00fc06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.165+0000 7f0104600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00fc0d0a70 0x7f00fc0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.165+0000 7f0104600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f00fc06f060 con 0x7f00fc06dbb0 2024-04-20T10:28:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.165+0000 7f0104600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f00fc06e7c0 con 0x7f00fc0d0a70 2024-04-20T10:28:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.165+0000 7f0104600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc06eaa0 con 0x7f00fc15e8a0 2024-04-20T10:28:31.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.165+0000 7f0102600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00fc0d0a70 0x7f00fc0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.165+0000 7f0101c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc15e8a0 0x7f00fc06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.166+0000 7f0102600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f00fc0d0a70 0x7f00fc0ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:48218/0 (socket says 172.21.15.47:48218) 2024-04-20T10:28:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.166+0000 7f0102600640 1 -- 172.21.15.47:0/1447720399 learned_addr learned my addr 172.21.15.47:0/1447720399 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.166+0000 7f0102600640 1 -- 172.21.15.47:0/1447720399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc15e8a0 msgr2=0x7f00fc06e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.166+0000 7f0102600640 1 --2- 172.21.15.47:0/1447720399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc15e8a0 0x7f00fc06e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.166+0000 7f0102600640 1 -- 172.21.15.47:0/1447720399 >> v1:172.21.15.175:6789/0 conn(0x7f00fc06dbb0 legacy=0x7f00fc0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.166+0000 7f0102600640 1 -- 172.21.15.47:0/1447720399 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc06ed80 con 0x7f00fc0d0a70 2024-04-20T10:28:31.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.168+0000 7f0102600640 1 --2- 172.21.15.47:0/1447720399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc0d0a70 0x7f00fc0ad680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009e50 tx=0x7f00f402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=849ee9ef27a73faa server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.168+0000 7f00fbe00640 1 -- 172.21.15.47:0/1447720399 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f402cb00 con 0x7f00fc0d0a70 2024-04-20T10:28:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.168+0000 7f00fbe00640 1 -- 172.21.15.47:0/1447720399 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f402cca0 con 0x7f00fc0d0a70 2024-04-20T10:28:31.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.168+0000 7f00fbe00640 1 -- 172.21.15.47:0/1447720399 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4034610 con 0x7f00fc0d0a70 2024-04-20T10:28:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 -- 172.21.15.47:0/1447720399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc0d0a70 msgr2=0x7f00fc0ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 --2- 172.21.15.47:0/1447720399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc0d0a70 0x7f00fc0ad680 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009e50 tx=0x7f00f402bf60 comp rx=0 tx=0).stop 2024-04-20T10:28:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 -- 172.21.15.47:0/1447720399 shutdown_connections 2024-04-20T10:28:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 --2- 172.21.15.47:0/1447720399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc0d0a70 0x7f00fc0ad680 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 --2- 172.21.15.47:0/1447720399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc15e8a0 0x7f00fc06e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 -- 172.21.15.47:0/1447720399 >> 172.21.15.47:0/1447720399 conn(0x7f00fc0b3350 msgr2=0x7f00fc0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 -- 172.21.15.47:0/1447720399 shutdown_connections 2024-04-20T10:28:31.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.169+0000 7f0104600640 1 -- 172.21.15.47:0/1447720399 wait complete. 2024-04-20T10:28:31.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.171+0000 7f0104600640 1 Processor -- start 2024-04-20T10:28:31.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.171+0000 7f0104600640 1 -- start start 2024-04-20T10:28:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0104600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc06dbb0 0x7f00fc14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0104600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc0d0a70 0x7f00fc1563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0104600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00fc15e8a0 0x7f00fc153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0104600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc06fef0 con 0x7f00fc15e8a0 2024-04-20T10:28:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0104600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00fc06fd70 con 0x7f00fc06dbb0 2024-04-20T10:28:31.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0104600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00fc0adbc0 con 0x7f00fc0d0a70 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0102600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00fc15e8a0 0x7f00fc153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0102600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00fc15e8a0 0x7f00fc153220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40338/0 (socket says 172.21.15.47:40338) 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.172+0000 7f0102600640 1 -- 172.21.15.47:0/1135943251 learned_addr learned my addr 172.21.15.47:0/1135943251 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0101c00640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc06dbb0 0x7f00fc14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0101200640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc0d0a70 0x7f00fc1563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0102600640 1 -- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc0d0a70 msgr2=0x7f00fc1563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0102600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc0d0a70 0x7f00fc1563a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0102600640 1 -- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc06dbb0 msgr2=0x7f00fc14d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0102600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc06dbb0 0x7f00fc14d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0102600640 1 -- 172.21.15.47:0/1135943251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00fc156d10 con 0x7f00fc15e8a0 2024-04-20T10:28:31.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.173+0000 7f0102600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00fc15e8a0 0x7f00fc153220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f00f4036040 tx=0x7f00f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.174+0000 7f00f9600640 1 -- 172.21.15.47:0/1135943251 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4043020 con 0x7f00fc15e8a0 2024-04-20T10:28:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.174+0000 7f00f9600640 1 -- 172.21.15.47:0/1135943251 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f00f402ce50 con 0x7f00fc15e8a0 2024-04-20T10:28:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.174+0000 7f00f9600640 1 -- 172.21.15.47:0/1135943251 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00f4009510 con 0x7f00fc15e8a0 2024-04-20T10:28:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.174+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00fc156ff0 con 0x7f00fc15e8a0 2024-04-20T10:28:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.174+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00fc1575b0 con 0x7f00fc15e8a0 2024-04-20T10:28:31.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.175+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00fc068980 con 0x7f00fc15e8a0 2024-04-20T10:28:31.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.176+0000 7f00f9600640 1 -- 172.21.15.47:0/1135943251 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f00f4046020 con 0x7f00fc15e8a0 2024-04-20T10:28:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.177+0000 7f00f9600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f00dc06c8f0 0x7f00dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.177+0000 7f00f9600640 1 -- 172.21.15.47:0/1135943251 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f00f40ae480 con 0x7f00fc15e8a0 2024-04-20T10:28:31.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.178+0000 7f0101c00640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f00dc06c8f0 0x7f00dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.179+0000 7f0101c00640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f00dc06c8f0 0x7f00dc06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f00fc152a90 tx=0x7f00f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:31.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.181+0000 7f00f9600640 1 -- 172.21.15.47:0/1135943251 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f00f404d030 con 0x7f00fc15e8a0 2024-04-20T10:28:31.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.383+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f00fc0b2c00 con 0x7f00fc15e8a0 2024-04-20T10:28:31.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.384+0000 7f00f9600640 1 -- 172.21.15.47:0/1135943251 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f00f403c350 con 0x7f00fc15e8a0 2024-04-20T10:28:31.385 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f00dc06c8f0 msgr2=0x7f00dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f00dc06c8f0 0x7f00dc06edb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f00fc152a90 tx=0x7f00f0008040 comp rx=0 tx=0).stop 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00fc15e8a0 msgr2=0x7f00fc153220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00fc15e8a0 0x7f00fc153220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f00f4036040 tx=0x7f00f4004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 shutdown_connections 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f00dc06c8f0 0x7f00dc06edb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00fc15e8a0 0x7f00fc153220 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00fc0d0a70 0x7f00fc1563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 --2- 172.21.15.47:0/1135943251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00fc06dbb0 0x7f00fc14d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 >> 172.21.15.47:0/1135943251 conn(0x7f00fc0b3350 msgr2=0x7f00fc155710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 shutdown_connections 2024-04-20T10:28:31.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.387+0000 7f0104600640 1 -- 172.21.15.47:0/1135943251 wait complete. 2024-04-20T10:28:31.407 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.625+0000 7fb321c00640 1 Processor -- start 2024-04-20T10:28:31.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.625+0000 7fb321c00640 1 -- start start 2024-04-20T10:28:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.626+0000 7fb321c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb31c161b40 0x7fb31c0c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.626+0000 7fb321c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0ae3e0 0x7fb31c0c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.626+0000 7fb321c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb31c0aded0 con 0x7fb31c0cdea0 2024-04-20T10:28:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.626+0000 7fb321c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb31c0ad2c0 con 0x7fb31c161b40 2024-04-20T10:28:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.626+0000 7fb321c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c0ad5a0 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.627+0000 7fb31be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb31c161b40 0x7fb31c0c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.627+0000 7fb313400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0ae3e0 0x7fb31c0c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.627+0000 7fb313400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0ae3e0 0x7fb31c0c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:51772/0 (socket says 172.21.15.47:51772) 2024-04-20T10:28:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.627+0000 7fb31be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb31c161b40 0x7fb31c0c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:48246/0 (socket says 172.21.15.47:48246) 2024-04-20T10:28:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.627+0000 7fb313400640 1 -- 172.21.15.47:0/3760392874 learned_addr learned my addr 172.21.15.47:0/3760392874 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.628+0000 7fb31be00640 1 -- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0ae3e0 msgr2=0x7fb31c0c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.628+0000 7fb31be00640 1 --2- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0ae3e0 0x7fb31c0c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.628+0000 7fb31be00640 1 -- 172.21.15.47:0/3760392874 >> v1:172.21.15.175:6789/0 conn(0x7fb31c0cdea0 legacy=0x7fb31c1620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.628+0000 7fb31be00640 1 -- 172.21.15.47:0/3760392874 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c0ad880 con 0x7fb31c161b40 2024-04-20T10:28:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.628+0000 7fb313400640 1 --2- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0ae3e0 0x7fb31c0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:28:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.629+0000 7fb31be00640 1 --2- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0c92f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009d90 tx=0x7fb30c02bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c6942fad3b5956a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.631+0000 7fb31b400640 1 -- 172.21.15.47:0/3760392874 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c039070 con 0x7fb31c161b40 2024-04-20T10:28:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.631+0000 7fb31b400640 1 -- 172.21.15.47:0/3760392874 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c02cc50 con 0x7fb31c161b40 2024-04-20T10:28:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.631+0000 7fb31b400640 1 -- 172.21.15.47:0/3760392874 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c034440 con 0x7fb31c161b40 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.631+0000 7fb321c00640 1 -- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 msgr2=0x7fb31c0c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.631+0000 7fb321c00640 1 --2- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0c92f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009d90 tx=0x7fb30c02bea0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.632+0000 7fb321c00640 1 -- 172.21.15.47:0/3760392874 shutdown_connections 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.632+0000 7fb321c00640 1 --2- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0ae3e0 0x7fb31c0c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.632+0000 7fb321c00640 1 --2- 172.21.15.47:0/3760392874 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0c92f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.632+0000 7fb321c00640 1 -- 172.21.15.47:0/3760392874 >> 172.21.15.47:0/3760392874 conn(0x7fb31c0cffa0 msgr2=0x7fb31c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.632+0000 7fb321c00640 1 -- 172.21.15.47:0/3760392874 shutdown_connections 2024-04-20T10:28:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.632+0000 7fb321c00640 1 -- 172.21.15.47:0/3760392874 wait complete. 2024-04-20T10:28:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.634+0000 7fb321c00640 1 Processor -- start 2024-04-20T10:28:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.635+0000 7fb321c00640 1 -- start start 2024-04-20T10:28:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb321c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb31c0ae3e0 0x7fb31c1f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb321c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0cdea0 0x7fb31c09d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb321c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0b2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb321c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c06d0c0 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb321c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb31c0acf50 con 0x7fb31c161b40 2024-04-20T10:28:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb321c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb31c06d240 con 0x7fb31c0cdea0 2024-04-20T10:28:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb31be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb31c0ae3e0 0x7fb31c1f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb31be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb31c0ae3e0 0x7fb31c1f88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40340/0 (socket says 172.21.15.47:40340) 2024-04-20T10:28:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.636+0000 7fb31be00640 1 -- 172.21.15.47:0/153358516 learned_addr learned my addr 172.21.15.47:0/153358516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb31be00640 1 -- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0cdea0 msgr2=0x7fb31c09d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb31be00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0cdea0 0x7fb31c09d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb31be00640 1 -- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 msgr2=0x7fb31c0b2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb321200640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0b2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb31be00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0b2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb31be00640 1 -- 172.21.15.47:0/153358516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31c0b56f0 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb321200640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb31be00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb31c0ae3e0 0x7fb31c1f88d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009d90 tx=0x7fb30c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb318c00640 1 -- 172.21.15.47:0/153358516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c044020 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.637+0000 7fb318c00640 1 -- 172.21.15.47:0/153358516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb30c03a040 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.638+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31c0b59d0 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.638+0000 7fb318c00640 1 -- 172.21.15.47:0/153358516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30c039070 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.638+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb31c0b32a0 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.639+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb31c075b30 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.640+0000 7fb318c00640 1 -- 172.21.15.47:0/153358516 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb30c047020 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.641+0000 7fb318c00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb2f806c8f0 0x7fb2f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:31.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.642+0000 7fb318c00640 1 -- 172.21.15.47:0/153358516 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fb30c0ae730 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.642+0000 7fb313400640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb2f806c8f0 0x7fb2f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:31.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.643+0000 7fb313400640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb2f806c8f0 0x7fb2f806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb304009630 tx=0x7fb304006e90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:31.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.646+0000 7fb318c00640 1 -- 172.21.15.47:0/153358516 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb30c04e030 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.853+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb31c0ae110 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.854+0000 7fb318c00640 1 -- 172.21.15.47:0/153358516 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fb30c07c7a0 con 0x7fb31c0ae3e0 2024-04-20T10:28:31.855 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:31.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb2f806c8f0 msgr2=0x7fb2f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb2f806c8f0 0x7fb2f806edb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb304009630 tx=0x7fb304006e90 comp rx=0 tx=0).stop 2024-04-20T10:28:31.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb31c0ae3e0 msgr2=0x7fb31c1f88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb31c0ae3e0 0x7fb31c1f88d0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009d90 tx=0x7fb30c004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 shutdown_connections 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb31c161b40 0x7fb31c0b2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb31c0cdea0 0x7fb31c09d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb2f806c8f0 0x7fb2f806edb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 --2- 172.21.15.47:0/153358516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb31c0ae3e0 0x7fb31c1f88d0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 >> 172.21.15.47:0/153358516 conn(0x7fb31c0cffa0 msgr2=0x7fb31c15c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 shutdown_connections 2024-04-20T10:28:31.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:31.857+0000 7fb321c00640 1 -- 172.21.15.47:0/153358516 wait complete. 2024-04-20T10:28:31.878 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.094+0000 7f85bd000640 1 Processor -- start 2024-04-20T10:28:32.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.094+0000 7f85bd000640 1 -- start start 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85bd000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85b8161a50 0x7f85b80b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85bd000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b80d0980 con 0x7f85b806c010 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85bd000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f85b8162490 con 0x7f85b80d3b20 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85bd000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f85b80d0c60 con 0x7f85b8161a50 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b7400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85b8161a50 0x7f85b80b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b7400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f85b8161a50 0x7f85b80b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40356/0 (socket says 172.21.15.47:40356) 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b7400640 1 -- 172.21.15.47:0/779226113 learned_addr learned my addr 172.21.15.47:0/779226113 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b7e00640 1 --1- 172.21.15.47:0/779226113 >> v1:172.21.15.47:6789/0 conn(0x7f85b806c010 0x7f85b80ceaa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40368/0 (socket says 172.21.15.47:40368) 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b7400640 1 -- 172.21.15.47:0/779226113 >> v1:172.21.15.175:6789/0 conn(0x7f85b80d3b20 legacy=0x7f85b806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b7400640 1 -- 172.21.15.47:0/779226113 >> v1:172.21.15.47:6789/0 conn(0x7f85b806c010 legacy=0x7f85b80ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b6000640 1 -- 172.21.15.47:0/779226113 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 717758434 0 0) 0x7f85b80d0980 con 0x7f85b806c010 2024-04-20T10:28:32.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.096+0000 7f85b7400640 1 -- 172.21.15.47:0/779226113 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b8162770 con 0x7f85b8161a50 2024-04-20T10:28:32.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.098+0000 7f85b7400640 1 --2- 172.21.15.47:0/779226113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80b3180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85b80d0de0 tx=0x7f85a802bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f991f9c5d7776f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:32.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.098+0000 7f85b6000640 1 -- 172.21.15.47:0/779226113 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a802cb80 con 0x7f85b8161a50 2024-04-20T10:28:32.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.098+0000 7f85b6000640 1 -- 172.21.15.47:0/779226113 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a802cd20 con 0x7f85b8161a50 2024-04-20T10:28:32.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.098+0000 7f85b6000640 1 -- 172.21.15.47:0/779226113 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a8034610 con 0x7f85b8161a50 2024-04-20T10:28:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.099+0000 7f85bd000640 1 -- 172.21.15.47:0/779226113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 msgr2=0x7f85b80b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.099+0000 7f85bd000640 1 --2- 172.21.15.47:0/779226113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80b3180 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f85b80d0de0 tx=0x7f85a802bfe0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.099+0000 7f85bd000640 1 -- 172.21.15.47:0/779226113 shutdown_connections 2024-04-20T10:28:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.099+0000 7f85bd000640 1 --2- 172.21.15.47:0/779226113 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80b3180 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.099+0000 7f85bd000640 1 -- 172.21.15.47:0/779226113 >> 172.21.15.47:0/779226113 conn(0x7f85b80d0ed0 msgr2=0x7f85b80d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.099+0000 7f85bd000640 1 -- 172.21.15.47:0/779226113 shutdown_connections 2024-04-20T10:28:32.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.099+0000 7f85bd000640 1 -- 172.21.15.47:0/779226113 wait complete. 2024-04-20T10:28:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 Processor -- start 2024-04-20T10:28:32.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 -- start start 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85b806c010 0x7f85b814d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b80ab370 0x7f85b814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f85b8162ae0 con 0x7f85b8161a50 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85b80d0690 con 0x7f85b80ab370 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85bd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85b8161e30 con 0x7f85b806c010 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40372/0 (socket says 172.21.15.47:40372) 2024-04-20T10:28:32.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 -- 172.21.15.47:0/3330584446 learned_addr learned my addr 172.21.15.47:0/3330584446 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b7e00640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b80ab370 0x7f85b814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b7400640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85b806c010 0x7f85b814d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 -- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85b806c010 msgr2=0x7f85b814d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85b806c010 0x7f85b814d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 -- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b80ab370 msgr2=0x7f85b814e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b80ab370 0x7f85b814e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b6a00640 1 -- 172.21.15.47:0/3330584446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85b80cd090 con 0x7f85b8161a50 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.102+0000 7f85b7e00640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b80ab370 0x7f85b814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.103+0000 7f85b6a00640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80cc640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c400 tx=0x7f85a000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.103+0000 7f85af400640 1 -- 172.21.15.47:0/3330584446 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0017070 con 0x7f85b8161a50 2024-04-20T10:28:32.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.103+0000 7f85af400640 1 -- 172.21.15.47:0/3330584446 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f85a00052d0 con 0x7f85b8161a50 2024-04-20T10:28:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.103+0000 7f85af400640 1 -- 172.21.15.47:0/3330584446 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85a0003830 con 0x7f85b8161a50 2024-04-20T10:28:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.103+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85b80cd370 con 0x7f85b8161a50 2024-04-20T10:28:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.103+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85b80cdc10 con 0x7f85b8161a50 2024-04-20T10:28:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.104+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85b80cde10 con 0x7f85b8161a50 2024-04-20T10:28:32.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.105+0000 7f85af400640 1 -- 172.21.15.47:0/3330584446 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f85a001a070 con 0x7f85b8161a50 2024-04-20T10:28:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.107+0000 7f85af400640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f859406c8f0 0x7f859406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.107+0000 7f85af400640 1 -- 172.21.15.47:0/3330584446 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f85a008b430 con 0x7f85b8161a50 2024-04-20T10:28:32.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.107+0000 7f85b7400640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f859406c8f0 0x7f859406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.108+0000 7f85b7400640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f859406c8f0 0x7f859406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f85a8035000 tx=0x7f85a8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:32.111 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.111+0000 7f85af400640 1 -- 172.21.15.47:0/3330584446 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f85b80cde10 con 0x7f85b8161a50 2024-04-20T10:28:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.323+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f85b8075b30 con 0x7f85b8161a50 2024-04-20T10:28:32.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.323+0000 7f85af400640 1 -- 172.21.15.47:0/3330584446 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f85a0059380 con 0x7f85b8161a50 2024-04-20T10:28:32.324 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:32.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f859406c8f0 msgr2=0x7f859406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f859406c8f0 0x7f859406edb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f85a8035000 tx=0x7f85a8030000 comp rx=0 tx=0).stop 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 msgr2=0x7f85b80cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80cc640 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f85a000c400 tx=0x7f85a000c8c0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 shutdown_connections 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85b80ab370 0x7f85b814e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f859406c8f0 0x7f859406edb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f85b8161a50 0x7f85b80cc640 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 --2- 172.21.15.47:0/3330584446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85b806c010 0x7f85b814d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 >> 172.21.15.47:0/3330584446 conn(0x7f85b80d0ed0 msgr2=0x7f85b80d28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 shutdown_connections 2024-04-20T10:28:32.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.326+0000 7f85bd000640 1 -- 172.21.15.47:0/3330584446 wait complete. 2024-04-20T10:28:32.353 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) MDS should verify/fix backtrace on fetch dirfrag 2024-04-20T10:28:32.353 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) 2024-04-20T10:28:32.353 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:32.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.562+0000 7f5f05200640 1 Processor -- start 2024-04-20T10:28:32.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.562+0000 7f5f05200640 1 -- start start 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5f05200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5f000d1a00 0x7f5f000b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5f05200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000b17d0 0x7f5f000d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5f05200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5f000d1e90 con 0x7f5f000d4270 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5f05200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5f000d40c0 con 0x7f5f000d1a00 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5f05200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f000d3820 con 0x7f5f000b17d0 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efde00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5f000d4270 0x7f5f000d4650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40380/0 (socket says 172.21.15.47:40380) 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efde00640 1 -- 172.21.15.47:0/3479304047 learned_addr learned my addr 172.21.15.47:0/3479304047 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:32.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efd400640 1 --2- 172.21.15.47:0/3479304047 >> v2:172.21.15.47:3300/0 conn(0x7f5f000d1a00 0x7f5f000b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efe800640 1 --2- 172.21.15.47:0/3479304047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000b17d0 0x7f5f000d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efca00640 1 -- 172.21.15.47:0/3479304047 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1347510165 0 0) 0x7f5ef0001110 con 0x7f5f000d4270 2024-04-20T10:28:32.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efd400640 1 -- 172.21.15.47:0/3479304047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000b17d0 msgr2=0x7f5f000d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:32.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efd400640 1 --2- 172.21.15.47:0/3479304047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000b17d0 0x7f5f000d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efd400640 1 -- 172.21.15.47:0/3479304047 >> v1:172.21.15.47:6789/0 conn(0x7f5f000d4270 legacy=0x7f5f000d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efd400640 1 -- 172.21.15.47:0/3479304047 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f000d3b00 con 0x7f5f000d1a00 2024-04-20T10:28:32.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.565+0000 7f5efe800640 1 --2- 172.21.15.47:0/3479304047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000b17d0 0x7f5f000d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.566+0000 7f5efd400640 1 --2- 172.21.15.47:0/3479304047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d1a00 0x7f5f000b1290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80097a0 tx=0x7f5ee802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b6a34c1e5ddc79 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.566+0000 7f5efca00640 1 -- 172.21.15.47:0/3479304047 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8035020 con 0x7f5f000d1a00 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.566+0000 7f5efca00640 1 -- 172.21.15.47:0/3479304047 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee8031810 con 0x7f5f000d1a00 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.566+0000 7f5efca00640 1 -- 172.21.15.47:0/3479304047 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8031be0 con 0x7f5f000d1a00 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 -- 172.21.15.47:0/3479304047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d1a00 msgr2=0x7f5f000b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 --2- 172.21.15.47:0/3479304047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d1a00 0x7f5f000b1290 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80097a0 tx=0x7f5ee802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 -- 172.21.15.47:0/3479304047 shutdown_connections 2024-04-20T10:28:32.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 --2- 172.21.15.47:0/3479304047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000b17d0 0x7f5f000d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 --2- 172.21.15.47:0/3479304047 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d1a00 0x7f5f000b1290 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 -- 172.21.15.47:0/3479304047 >> 172.21.15.47:0/3479304047 conn(0x7f5f000b99c0 msgr2=0x7f5f000b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 -- 172.21.15.47:0/3479304047 shutdown_connections 2024-04-20T10:28:32.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.567+0000 7f5f05200640 1 -- 172.21.15.47:0/3479304047 wait complete. 2024-04-20T10:28:32.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.569+0000 7f5f05200640 1 Processor -- start 2024-04-20T10:28:32.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.569+0000 7f5f05200640 1 -- start start 2024-04-20T10:28:32.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5f05200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f000b17d0 0x7f5f0014ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5f05200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000d1a00 0x7f5f000c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5f05200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d4270 0x7f5f000cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5f05200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000d53b0 con 0x7f5f000d4270 2024-04-20T10:28:32.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5f05200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000cfe60 con 0x7f5f000b17d0 2024-04-20T10:28:32.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5f05200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f00161f70 con 0x7f5f000d1a00 2024-04-20T10:28:32.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d4270 0x7f5f000cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d4270 0x7f5f000cb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40388/0 (socket says 172.21.15.47:40388) 2024-04-20T10:28:32.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 -- 172.21.15.47:0/3013081605 learned_addr learned my addr 172.21.15.47:0/3013081605 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efde00640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f000b17d0 0x7f5f0014ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efd400640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000d1a00 0x7f5f000c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 -- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000d1a00 msgr2=0x7f5f000c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000d1a00 0x7f5f000c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 -- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f000b17d0 msgr2=0x7f5f0014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f000b17d0 0x7f5f0014ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 -- 172.21.15.47:0/3013081605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f000c98c0 con 0x7f5f000d4270 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efde00640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f000b17d0 0x7f5f0014ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efd400640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000d1a00 0x7f5f000c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:32.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5efe800640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d4270 0x7f5f000cb6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800cd40 tx=0x7f5ef800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:32.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5ef6000640 1 -- 172.21.15.47:0/3013081605 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8003b90 con 0x7f5f000d4270 2024-04-20T10:28:32.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5ef6000640 1 -- 172.21.15.47:0/3013081605 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ef8003d50 con 0x7f5f000d4270 2024-04-20T10:28:32.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.571+0000 7f5ef6000640 1 -- 172.21.15.47:0/3013081605 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8004260 con 0x7f5f000d4270 2024-04-20T10:28:32.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.572+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f000c9ba0 con 0x7f5f000d4270 2024-04-20T10:28:32.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.572+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f000ca160 con 0x7f5f000d4270 2024-04-20T10:28:32.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.573+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f00069090 con 0x7f5f000d4270 2024-04-20T10:28:32.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.574+0000 7f5ef6000640 1 -- 172.21.15.47:0/3013081605 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ef8004400 con 0x7f5f000d4270 2024-04-20T10:28:32.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.575+0000 7f5ef6000640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5edc06c8f0 0x7f5edc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:32.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.575+0000 7f5ef6000640 1 -- 172.21.15.47:0/3013081605 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f5ef808c970 con 0x7f5f000d4270 2024-04-20T10:28:32.577 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.576+0000 7f5efde00640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5edc06c8f0 0x7f5edc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:32.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.578+0000 7f5efde00640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5edc06c8f0 0x7f5edc06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00013b0 tx=0x7f5ef0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:32.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.580+0000 7f5ef6000640 1 -- 172.21.15.47:0/3013081605 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5ef8014030 con 0x7f5f000d4270 2024-04-20T10:28:32.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.790+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f000cae50 con 0x7f5f000d4270 2024-04-20T10:28:32.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.792+0000 7f5ef6000640 1 -- 172.21.15.47:0/3013081605 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f5ef805a9f0 con 0x7f5f000d4270 2024-04-20T10:28:32.793 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:32.793 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5edc06c8f0 msgr2=0x7f5edc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5edc06c8f0 0x7f5edc06edb0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00013b0 tx=0x7f5ef0009040 comp rx=0 tx=0).stop 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d4270 msgr2=0x7f5f000cb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d4270 0x7f5f000cb6b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5ef800cd40 tx=0x7f5ef800a650 comp rx=0 tx=0).stop 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 shutdown_connections 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5edc06c8f0 0x7f5edc06edb0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5f000d4270 0x7f5f000cb6b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f000d1a00 0x7f5f000c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 --2- 172.21.15.47:0/3013081605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f000b17d0 0x7f5f0014ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:32.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.795+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 >> 172.21.15.47:0/3013081605 conn(0x7f5f000b99c0 msgr2=0x7f5f000b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:32.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.796+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 shutdown_connections 2024-04-20T10:28:32.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:32.796+0000 7f5f05200640 1 -- 172.21.15.47:0/3013081605 wait complete. 2024-04-20T10:28:32.816 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:32.816 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.035+0000 7fa783200640 1 Processor -- start 2024-04-20T10:28:33.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.035+0000 7fa783200640 1 -- start start 2024-04-20T10:28:33.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.036+0000 7fa783200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa78415e770 0x7fa7840b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.036+0000 7fa783200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa784162210 0x7fa784162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.037+0000 7fa783200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7840d3590 con 0x7fa7840b6400 2024-04-20T10:28:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.037+0000 7fa783200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa78415eb50 con 0x7fa784162210 2024-04-20T10:28:33.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.037+0000 7fa783200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa7840d3870 con 0x7fa78415e770 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.037+0000 7fa782800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa7840b6400 0x7fa7840d42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40390/0 (socket says 172.21.15.47:40390) 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.037+0000 7fa782800640 1 -- 172.21.15.47:0/3828711072 learned_addr learned my addr 172.21.15.47:0/3828711072 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.037+0000 7fa781400640 1 --2- 172.21.15.47:0/3828711072 >> v2:172.21.15.47:3300/0 conn(0x7fa784162210 0x7fa784162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.037+0000 7fa781e00640 1 --2- 172.21.15.47:0/3828711072 >> v2:172.21.15.175:3300/0 conn(0x7fa78415e770 0x7fa7840b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.038+0000 7fa781400640 1 -- 172.21.15.47:0/3828711072 >> v2:172.21.15.175:3300/0 conn(0x7fa78415e770 msgr2=0x7fa7840b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.038+0000 7fa781400640 1 --2- 172.21.15.47:0/3828711072 >> v2:172.21.15.175:3300/0 conn(0x7fa78415e770 0x7fa7840b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.038+0000 7fa781400640 1 -- 172.21.15.47:0/3828711072 >> v1:172.21.15.47:6789/0 conn(0x7fa7840b6400 legacy=0x7fa7840d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.038+0000 7fa781400640 1 -- 172.21.15.47:0/3828711072 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7840d3b50 con 0x7fa784162210 2024-04-20T10:28:33.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.038+0000 7fa780a00640 1 -- 172.21.15.47:0/3828711072 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3752213444 0 0) 0x7fa7840d3590 con 0x7fa7840b6400 2024-04-20T10:28:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.039+0000 7fa781400640 1 --2- 172.21.15.47:0/3828711072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa784162210 0x7fa784162670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa78408e630 tx=0x7fa77002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3bedbf89fbe84e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.039+0000 7fa780a00640 1 -- 172.21.15.47:0/3828711072 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7700315a0 con 0x7fa784162210 2024-04-20T10:28:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.039+0000 7fa780a00640 1 -- 172.21.15.47:0/3828711072 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa770037070 con 0x7fa784162210 2024-04-20T10:28:33.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.039+0000 7fa780a00640 1 -- 172.21.15.47:0/3828711072 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770031960 con 0x7fa784162210 2024-04-20T10:28:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 -- 172.21.15.47:0/3828711072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa784162210 msgr2=0x7fa784162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 --2- 172.21.15.47:0/3828711072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa784162210 0x7fa784162670 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa78408e630 tx=0x7fa77002b840 comp rx=0 tx=0).stop 2024-04-20T10:28:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 -- 172.21.15.47:0/3828711072 shutdown_connections 2024-04-20T10:28:33.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 --2- 172.21.15.47:0/3828711072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa784162210 0x7fa784162670 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 --2- 172.21.15.47:0/3828711072 >> v2:172.21.15.175:3300/0 conn(0x7fa78415e770 0x7fa7840b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 -- 172.21.15.47:0/3828711072 >> 172.21.15.47:0/3828711072 conn(0x7fa78409d6d0 msgr2=0x7fa7840b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 -- 172.21.15.47:0/3828711072 shutdown_connections 2024-04-20T10:28:33.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.040+0000 7fa783200640 1 -- 172.21.15.47:0/3828711072 wait complete. 2024-04-20T10:28:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.042+0000 7fa783200640 1 Processor -- start 2024-04-20T10:28:33.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.042+0000 7fa783200640 1 -- start start 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa783200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7840b6400 0x7fa7840cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa783200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa78415e770 0x7fa784154380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa783200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa784162210 0x7fa78406cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa783200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa7840b39d0 con 0x7fa78415e770 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa783200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7840d3e30 con 0x7fa7840b6400 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa783200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7840b3b50 con 0x7fa784162210 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa781400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa78415e770 0x7fa784154380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa781400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa78415e770 0x7fa784154380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40402/0 (socket says 172.21.15.47:40402) 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.043+0000 7fa781400640 1 -- 172.21.15.47:0/3996496728 learned_addr learned my addr 172.21.15.47:0/3996496728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781e00640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7840b6400 0x7fa7840cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781400640 1 -- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa784162210 msgr2=0x7fa78406cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa782800640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa784162210 0x7fa78406cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781400640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa784162210 0x7fa78406cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781400640 1 -- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7840b6400 msgr2=0x7fa7840cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781400640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7840b6400 0x7fa7840cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781400640 1 -- 172.21.15.47:0/3996496728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa78406d370 con 0x7fa78415e770 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa782800640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa784162210 0x7fa78406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781e00640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7840b6400 0x7fa7840cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:33.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.044+0000 7fa781400640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa78415e770 0x7fa784154380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa770034040 tx=0x7fa7700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.045+0000 7fa77a000640 1 -- 172.21.15.47:0/3996496728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770004230 con 0x7fa78415e770 2024-04-20T10:28:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.045+0000 7fa77a000640 1 -- 172.21.15.47:0/3996496728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7700043d0 con 0x7fa78415e770 2024-04-20T10:28:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.045+0000 7fa77a000640 1 -- 172.21.15.47:0/3996496728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa770003850 con 0x7fa78415e770 2024-04-20T10:28:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.045+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa78406d650 con 0x7fa78415e770 2024-04-20T10:28:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.045+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa78406dc10 con 0x7fa78415e770 2024-04-20T10:28:33.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.046+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7840cea20 con 0x7fa78415e770 2024-04-20T10:28:33.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.047+0000 7fa77a000640 1 -- 172.21.15.47:0/3996496728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7700039f0 con 0x7fa78415e770 2024-04-20T10:28:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.048+0000 7fa77a000640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.048+0000 7fa77a000640 1 -- 172.21.15.47:0/3996496728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fa77002f030 con 0x7fa78415e770 2024-04-20T10:28:33.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.049+0000 7fa781e00640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.050+0000 7fa781e00640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76406c8f0 0x7fa76406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa774009af0 tx=0x7fa774008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.053+0000 7fa77a000640 1 -- 172.21.15.47:0/3996496728 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7840cea20 con 0x7fa78415e770 2024-04-20T10:28:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.262+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa784075b30 con 0x7fa78415e770 2024-04-20T10:28:33.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.262+0000 7fa77a000640 1 -- 172.21.15.47:0/3996496728 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fa770041070 con 0x7fa78415e770 2024-04-20T10:28:33.263 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.265+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76406c8f0 msgr2=0x7fa76406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.265+0000 7fa783200640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76406c8f0 0x7fa76406edb0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa774009af0 tx=0x7fa774008040 comp rx=0 tx=0).stop 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.265+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa78415e770 msgr2=0x7fa784154380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.265+0000 7fa783200640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa78415e770 0x7fa784154380 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa770034040 tx=0x7fa7700046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 shutdown_connections 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa784162210 0x7fa78406cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76406c8f0 0x7fa76406edb0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa78415e770 0x7fa784154380 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 --2- 172.21.15.47:0/3996496728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7840b6400 0x7fa7840cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 >> 172.21.15.47:0/3996496728 conn(0x7fa78409d6d0 msgr2=0x7fa7840d26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:33.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 shutdown_connections 2024-04-20T10:28:33.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.266+0000 7fa783200640 1 -- 172.21.15.47:0/3996496728 wait complete. 2024-04-20T10:28:33.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:33.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.495+0000 7fb78ca00640 1 Processor -- start 2024-04-20T10:28:33.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.495+0000 7fb78ca00640 1 -- start start 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.496+0000 7fb78ca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb78815d4f0 0x7fb7880ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.496+0000 7fb78ca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb78806e060 0x7fb78806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.496+0000 7fb78ca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb78806b900 con 0x7fb7880aee60 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb78ca00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb78806c510 con 0x7fb78815d4f0 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb78ca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb78806bbe0 con 0x7fb78806e060 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb77de00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb78815d4f0 0x7fb7880ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb77de00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb78815d4f0 0x7fb7880ad0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40412/0 (socket says 172.21.15.47:40412) 2024-04-20T10:28:33.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb77de00640 1 -- 172.21.15.47:0/4141448435 learned_addr learned my addr 172.21.15.47:0/4141448435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb787200640 1 --2- 172.21.15.47:0/4141448435 >> v2:172.21.15.175:3300/0 conn(0x7fb78806e060 0x7fb78806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb77de00640 1 -- 172.21.15.47:0/4141448435 >> v2:172.21.15.175:3300/0 conn(0x7fb78806e060 msgr2=0x7fb78806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb77de00640 1 --2- 172.21.15.47:0/4141448435 >> v2:172.21.15.175:3300/0 conn(0x7fb78806e060 0x7fb78806e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb77de00640 1 -- 172.21.15.47:0/4141448435 >> v1:172.21.15.175:6789/0 conn(0x7fb7880aee60 legacy=0x7fb7880af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.497+0000 7fb77de00640 1 -- 172.21.15.47:0/4141448435 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78806bec0 con 0x7fb78815d4f0 2024-04-20T10:28:33.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.499+0000 7fb77de00640 1 --2- 172.21.15.47:0/4141448435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78815d4f0 0x7fb7880ad0f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb7880719a0 tx=0x7fb77002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63a9d648b7c0b8ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.499+0000 7fb785e00640 1 -- 172.21.15.47:0/4141448435 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77002ca80 con 0x7fb78815d4f0 2024-04-20T10:28:33.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.499+0000 7fb785e00640 1 -- 172.21.15.47:0/4141448435 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb77002cc20 con 0x7fb78815d4f0 2024-04-20T10:28:33.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.499+0000 7fb785e00640 1 -- 172.21.15.47:0/4141448435 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7700343f0 con 0x7fb78815d4f0 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 -- 172.21.15.47:0/4141448435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78815d4f0 msgr2=0x7fb7880ad0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 --2- 172.21.15.47:0/4141448435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78815d4f0 0x7fb7880ad0f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb7880719a0 tx=0x7fb77002bee0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 -- 172.21.15.47:0/4141448435 shutdown_connections 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 --2- 172.21.15.47:0/4141448435 >> v2:172.21.15.175:3300/0 conn(0x7fb78806e060 0x7fb78806e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 --2- 172.21.15.47:0/4141448435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78815d4f0 0x7fb7880ad0f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 -- 172.21.15.47:0/4141448435 >> 172.21.15.47:0/4141448435 conn(0x7fb7880b5270 msgr2=0x7fb7880b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 -- 172.21.15.47:0/4141448435 shutdown_connections 2024-04-20T10:28:33.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.500+0000 7fb78ca00640 1 -- 172.21.15.47:0/4141448435 wait complete. 2024-04-20T10:28:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.502+0000 7fb78ca00640 1 Processor -- start 2024-04-20T10:28:33.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.502+0000 7fb78ca00640 1 -- start start 2024-04-20T10:28:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb78ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78806e060 0x7fb7881ef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb78ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7880aee60 0x7fb788154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb78ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb78815d4f0 0x7fb7880b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb78ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7880ad630 con 0x7fb78806e060 2024-04-20T10:28:33.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb78ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb78806c9a0 con 0x7fb7880aee60 2024-04-20T10:28:33.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb78ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7880ad7b0 con 0x7fb78815d4f0 2024-04-20T10:28:33.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb786800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78806e060 0x7fb7881ef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb786800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78806e060 0x7fb7881ef4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:40420/0 (socket says 172.21.15.47:40420) 2024-04-20T10:28:33.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.503+0000 7fb786800640 1 -- 172.21.15.47:0/3898703222 learned_addr learned my addr 172.21.15.47:0/3898703222 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb786800640 1 -- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb78815d4f0 msgr2=0x7fb7880b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb786800640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb78815d4f0 0x7fb7880b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb786800640 1 -- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7880aee60 msgr2=0x7fb788154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb786800640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7880aee60 0x7fb788154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb786800640 1 -- 172.21.15.47:0/3898703222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78809df20 con 0x7fb78806e060 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb786800640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78806e060 0x7fb7881ef4b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb77800c430 tx=0x7fb77800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb77f400640 1 -- 172.21.15.47:0/3898703222 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778011790 con 0x7fb78806e060 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb77f400640 1 -- 172.21.15.47:0/3898703222 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb778017070 con 0x7fb78806e060 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78806ed70 con 0x7fb78806e060 2024-04-20T10:28:33.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb77f400640 1 -- 172.21.15.47:0/3898703222 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778011bb0 con 0x7fb78806e060 2024-04-20T10:28:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.504+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb788154e00 con 0x7fb78806e060 2024-04-20T10:28:33.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.506+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb788075b30 con 0x7fb78806e060 2024-04-20T10:28:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.506+0000 7fb77f400640 1 -- 172.21.15.47:0/3898703222 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb77801c460 con 0x7fb78806e060 2024-04-20T10:28:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.507+0000 7fb77f400640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb76406c8f0 0x7fb76406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.508+0000 7fb77f400640 1 -- 172.21.15.47:0/3898703222 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fb77808bd70 con 0x7fb78806e060 2024-04-20T10:28:33.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.508+0000 7fb77de00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb76406c8f0 0x7fb76406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.509+0000 7fb77de00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb76406c8f0 0x7fb76406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb770035000 tx=0x7fb77002e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.512+0000 7fb77f400640 1 -- 172.21.15.47:0/3898703222 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb778059de0 con 0x7fb78806e060 2024-04-20T10:28:33.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.707+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fb78806ceb0 con 0x7fb78806e060 2024-04-20T10:28:33.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.707+0000 7fb77f400640 1 -- 172.21.15.47:0/3898703222 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fb7780596e0 con 0x7fb78806e060 2024-04-20T10:28:33.707 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:33.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb76406c8f0 msgr2=0x7fb76406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb76406c8f0 0x7fb76406edb0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb770035000 tx=0x7fb77002e000 comp rx=0 tx=0).stop 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78806e060 msgr2=0x7fb7881ef4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78806e060 0x7fb7881ef4b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb77800c430 tx=0x7fb77800c8f0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 shutdown_connections 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb78815d4f0 0x7fb7880b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7880aee60 0x7fb788154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb76406c8f0 0x7fb76406edb0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 --2- 172.21.15.47:0/3898703222 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb78806e060 0x7fb7881ef4b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 >> 172.21.15.47:0/3898703222 conn(0x7fb7880b5270 msgr2=0x7fb788071be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 shutdown_connections 2024-04-20T10:28:33.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.710+0000 7fb78ca00640 1 -- 172.21.15.47:0/3898703222 wait complete. 2024-04-20T10:28:33.731 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.936+0000 7f82da000640 1 Processor -- start 2024-04-20T10:28:33.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.936+0000 7f82da000640 1 -- start start 2024-04-20T10:28:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.938+0000 7f82da000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 0x7f82d40b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.938+0000 7f82da000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d40d3590 con 0x7f82d4162670 2024-04-20T10:28:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.938+0000 7f82da000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f82d40d3870 con 0x7f82d4162210 2024-04-20T10:28:33.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.938+0000 7f82da000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82d415eb50 con 0x7f82d415e770 2024-04-20T10:28:33.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.939+0000 7f82d9600640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f82d4162670 0x7f82d40d42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33202/0 (socket says 172.21.15.47:33202) 2024-04-20T10:28:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.939+0000 7f82d9600640 1 -- 172.21.15.47:0/3562438016 learned_addr learned my addr 172.21.15.47:0/3562438016 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.939+0000 7f82d8c00640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 0x7f82d40b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.939+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1557316603 0 0) 0x7f82d40d3590 con 0x7f82d4162670 2024-04-20T10:28:33.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.939+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82bc003520 con 0x7f82d4162670 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4178675108 0 0) 0x7f82d40d3870 con 0x7f82d4162210 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f82d40d3590 con 0x7f82d4162210 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3796892221 0 0) 0x7f82bc003520 con 0x7f82d4162670 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 msgr2=0x7f82d40b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 0x7f82d40b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 >> v1:172.21.15.175:6789/0 conn(0x7f82d4162210 legacy=0x7f82d40b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40d3b50 con 0x7f82d4162670 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f82cc0019f0 con 0x7f82d4162670 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 >> v1:172.21.15.47:6789/0 conn(0x7f82d4162670 legacy=0x7f82d40d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82bc003b70 0x7f82bc003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 0x7f82bc005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82d415eb50 con 0x7f82bc003b70 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc00bb50 con 0x7f82bc004800 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82bc00bd20 con 0x7f82d415e770 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.940+0000 7f82d8c00640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82bc003b70 0x7f82bc003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d3e00640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d9600640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 0x7f82bc005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d8c00640 1 -- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 msgr2=0x7f82bc005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d8c00640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82d415e770 0x7f82bc005220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d8c00640 1 -- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82bc004800 msgr2=0x7f82bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d8c00640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d8c00640 1 -- 172.21.15.47:0/3562438016 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc003520 con 0x7f82bc003b70 2024-04-20T10:28:33.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.941+0000 7f82d3e00640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82bc004800 0x7f82bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.942+0000 7f82d8c00640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82bc003b70 0x7f82bc003f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009af0 tx=0x7f82bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3a01aa2d9f28c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.942+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8039070 con 0x7f82bc003b70 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.942+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c802c6b0 con 0x7f82bc003b70 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.942+0000 7f82d3400640 1 -- 172.21.15.47:0/3562438016 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c8009690 con 0x7f82bc003b70 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.943+0000 7f82da000640 1 -- 172.21.15.47:0/3562438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82bc003b70 msgr2=0x7f82bc003f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.943+0000 7f82da000640 1 --2- 172.21.15.47:0/3562438016 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82bc003b70 0x7f82bc003f70 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f82c8009af0 tx=0x7f82bc005830 comp rx=0 tx=0).stop 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.943+0000 7f82d9600640 1 -- 172.21.15.47:0/3562438016 reap_dead start 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.943+0000 7f82da000640 1 -- 172.21.15.47:0/3562438016 shutdown_connections 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.943+0000 7f82da000640 1 -- 172.21.15.47:0/3562438016 >> 172.21.15.47:0/3562438016 conn(0x7f82d409d6d0 msgr2=0x7f82d40b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.943+0000 7f82da000640 1 -- 172.21.15.47:0/3562438016 shutdown_connections 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.943+0000 7f82da000640 1 -- 172.21.15.47:0/3562438016 wait complete. 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.945+0000 7f82da000640 1 Processor -- start 2024-04-20T10:28:33.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.945+0000 7f82da000640 1 -- start start 2024-04-20T10:28:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.946+0000 7f82da000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82bc003b70 0x7f82d40ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.946+0000 7f82da000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82d40cd5d0 0x7f82d4161790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.946+0000 7f82da000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d40cda50 0x7f82d415ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.946+0000 7f82da000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40b6b10 con 0x7f82d40cd5d0 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.946+0000 7f82da000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82d40d3e30 con 0x7f82d40cda50 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.946+0000 7f82da000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82d40b6c90 con 0x7f82bc003b70 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82d40cd5d0 0x7f82d4161790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82d40cd5d0 0x7f82d4161790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44542/0 (socket says 172.21.15.47:44542) 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 -- 172.21.15.47:0/1315761025 learned_addr learned my addr 172.21.15.47:0/1315761025 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d8c00640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82bc003b70 0x7f82d40ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d9600640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d40cda50 0x7f82d415ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 -- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82bc003b70 msgr2=0x7f82d40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82bc003b70 0x7f82d40ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 -- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d40cda50 msgr2=0x7f82d415ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:33.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d40cda50 0x7f82d415ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 -- 172.21.15.47:0/1315761025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82d40cad80 con 0x7f82d40cd5d0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d9600640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d40cda50 0x7f82d415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.947+0000 7f82d3e00640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82d40cd5d0 0x7f82d4161790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d7d0 tx=0x7f82c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.948+0000 7f82d0c00640 1 -- 172.21.15.47:0/1315761025 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c000bac0 con 0x7f82d40cd5d0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.948+0000 7f82d0c00640 1 -- 172.21.15.47:0/1315761025 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f82c000bc60 con 0x7f82d40cd5d0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.948+0000 7f82d0c00640 1 -- 172.21.15.47:0/1315761025 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c00069f0 con 0x7f82d40cd5d0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.948+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82d40cc270 con 0x7f82d40cd5d0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.948+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82d40cc810 con 0x7f82d40cd5d0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.949+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82d40b74b0 con 0x7f82d40cd5d0 2024-04-20T10:28:33.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.950+0000 7f82d0c00640 1 -- 172.21.15.47:0/1315761025 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f82c0006b90 con 0x7f82d40cd5d0 2024-04-20T10:28:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.951+0000 7f82d0c00640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f82b406c8f0 0x7f82b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.952+0000 7f82d0c00640 1 -- 172.21.15.47:0/1315761025 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f82c008c060 con 0x7f82d40cd5d0 2024-04-20T10:28:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.952+0000 7f82d8c00640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f82b406c8f0 0x7f82b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:33.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.953+0000 7f82d8c00640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f82b406c8f0 0x7f82b406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82c8036000 tx=0x7f82c8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:33.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:33.956+0000 7f82d0c00640 1 -- 172.21.15.47:0/1315761025 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f82d40b74b0 con 0x7f82d40cd5d0 2024-04-20T10:28:34.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.163+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f82d4075b30 con 0x7f82d40cd5d0 2024-04-20T10:28:34.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.164+0000 7f82d0c00640 1 -- 172.21.15.47:0/1315761025 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f82c0016020 con 0x7f82d40cd5d0 2024-04-20T10:28:34.164 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f82b406c8f0 msgr2=0x7f82b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f82b406c8f0 0x7f82b406edb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f82c8036000 tx=0x7f82c8030000 comp rx=0 tx=0).stop 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82d40cd5d0 msgr2=0x7f82d4161790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82d40cd5d0 0x7f82d4161790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f82c000d7d0 tx=0x7f82c000dc90 comp rx=0 tx=0).stop 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 shutdown_connections 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82d40cda50 0x7f82d415ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f82b406c8f0 0x7f82b406edb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f82d40cd5d0 0x7f82d4161790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 --2- 172.21.15.47:0/1315761025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82bc003b70 0x7f82d40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.166+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 >> 172.21.15.47:0/1315761025 conn(0x7f82d409d6d0 msgr2=0x7f82d40b21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:34.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.167+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 shutdown_connections 2024-04-20T10:28:34.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.167+0000 7f82da000640 1 -- 172.21.15.47:0/1315761025 wait complete. 2024-04-20T10:28:34.186 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:34.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.398+0000 7ff799a00640 1 Processor -- start 2024-04-20T10:28:34.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.399+0000 7ff799a00640 1 -- start start 2024-04-20T10:28:34.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.400+0000 7ff799a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7940d0010 0x7ff7940adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.400+0000 7ff799a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7940ae140 0x7ff79406ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.400+0000 7ff799a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff79415b5d0 con 0x7ff79415b1f0 2024-04-20T10:28:34.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.400+0000 7ff799a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff79406e730 con 0x7ff7940d0010 2024-04-20T10:28:34.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.400+0000 7ff799a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7940d05d0 con 0x7ff7940ae140 2024-04-20T10:28:34.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.401+0000 7ff793400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7940d0010 0x7ff7940adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.401+0000 7ff799000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7940ae140 0x7ff79406ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.401+0000 7ff799000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7940ae140 0x7ff79406ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:53816/0 (socket says 172.21.15.47:53816) 2024-04-20T10:28:34.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.401+0000 7ff799000640 1 -- 172.21.15.47:0/794969867 learned_addr learned my addr 172.21.15.47:0/794969867 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:34.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.402+0000 7ff793400640 1 -- 172.21.15.47:0/794969867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7940ae140 msgr2=0x7ff79406ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:34.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.402+0000 7ff793400640 1 --2- 172.21.15.47:0/794969867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7940ae140 0x7ff79406ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.402+0000 7ff793400640 1 -- 172.21.15.47:0/794969867 >> v1:172.21.15.175:6789/0 conn(0x7ff79415b1f0 legacy=0x7ff79415f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.402+0000 7ff793400640 1 -- 172.21.15.47:0/794969867 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7940d08b0 con 0x7ff7940d0010 2024-04-20T10:28:34.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.403+0000 7ff793400640 1 --2- 172.21.15.47:0/794969867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 0x7ff7940adc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff7800097d0 tx=0x7ff78002b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75c30461d59f2b5e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:34.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.404+0000 7ff792a00640 1 -- 172.21.15.47:0/794969867 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780031770 con 0x7ff7940d0010 2024-04-20T10:28:34.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.404+0000 7ff792a00640 1 -- 172.21.15.47:0/794969867 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff780037070 con 0x7ff7940d0010 2024-04-20T10:28:34.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.404+0000 7ff792a00640 1 -- 172.21.15.47:0/794969867 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780031b30 con 0x7ff7940d0010 2024-04-20T10:28:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 -- 172.21.15.47:0/794969867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 msgr2=0x7ff7940adc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 --2- 172.21.15.47:0/794969867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 0x7ff7940adc00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff7800097d0 tx=0x7ff78002b810 comp rx=0 tx=0).stop 2024-04-20T10:28:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 -- 172.21.15.47:0/794969867 shutdown_connections 2024-04-20T10:28:34.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 --2- 172.21.15.47:0/794969867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7940ae140 0x7ff79406ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 --2- 172.21.15.47:0/794969867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 0x7ff7940adc00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 -- 172.21.15.47:0/794969867 >> 172.21.15.47:0/794969867 conn(0x7ff7940b5270 msgr2=0x7ff7940b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:34.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 -- 172.21.15.47:0/794969867 shutdown_connections 2024-04-20T10:28:34.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.405+0000 7ff799a00640 1 -- 172.21.15.47:0/794969867 wait complete. 2024-04-20T10:28:34.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.407+0000 7ff799a00640 1 Processor -- start 2024-04-20T10:28:34.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.407+0000 7ff799a00640 1 -- start start 2024-04-20T10:28:34.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.408+0000 7ff799a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ae140 0x7ff79409d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.408+0000 7ff799a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 0x7ff79409a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.408+0000 7ff799a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff79415b1f0 0x7ff79409aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.408+0000 7ff799a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff79406cf50 con 0x7ff7940ae140 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.408+0000 7ff799a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7940d0e70 con 0x7ff7940d0010 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.408+0000 7ff799a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff79406d0d0 con 0x7ff79415b1f0 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.408+0000 7ff793e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ae140 0x7ff79409d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ae140 0x7ff79409d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44552/0 (socket says 172.21.15.47:44552) 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 -- 172.21.15.47:0/2725603175 learned_addr learned my addr 172.21.15.47:0/2725603175 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff799000640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff79415b1f0 0x7ff79409aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793400640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 0x7ff79409a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 -- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff79415b1f0 msgr2=0x7ff79409aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff79415b1f0 0x7ff79409aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 -- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 msgr2=0x7ff79409a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 0x7ff79409a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 -- 172.21.15.47:0/2725603175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79409b320 con 0x7ff7940ae140 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.409+0000 7ff793e00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ae140 0x7ff79409d060 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c3f0 tx=0x7ff78400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.410+0000 7ff78be00640 1 -- 172.21.15.47:0/2725603175 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7840052d0 con 0x7ff7940ae140 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.410+0000 7ff78be00640 1 -- 172.21.15.47:0/2725603175 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff784005490 con 0x7ff7940ae140 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.410+0000 7ff78be00640 1 -- 172.21.15.47:0/2725603175 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff784007260 con 0x7ff7940ae140 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.410+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79409b600 con 0x7ff7940ae140 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.410+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff79409bbc0 con 0x7ff7940ae140 2024-04-20T10:28:34.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.411+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff79409bff0 con 0x7ff7940ae140 2024-04-20T10:28:34.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.412+0000 7ff78be00640 1 -- 172.21.15.47:0/2725603175 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff784007400 con 0x7ff7940ae140 2024-04-20T10:28:34.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.413+0000 7ff78be00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff77406c8f0 0x7ff77406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.413+0000 7ff78be00640 1 -- 172.21.15.47:0/2725603175 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff78408b440 con 0x7ff7940ae140 2024-04-20T10:28:34.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.414+0000 7ff793400640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff77406c8f0 0x7ff77406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.415+0000 7ff793400640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff77406c8f0 0x7ff77406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff780036000 tx=0x7ff78003d040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:34.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.417+0000 7ff78be00640 1 -- 172.21.15.47:0/2725603175 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff784059420 con 0x7ff7940ae140 2024-04-20T10:28:34.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.622+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff794075b30 con 0x7ff7940ae140 2024-04-20T10:28:34.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.624+0000 7ff78be00640 1 -- 172.21.15.47:0/2725603175 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7ff784058d20 con 0x7ff7940ae140 2024-04-20T10:28:34.624 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.626+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff77406c8f0 msgr2=0x7ff77406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.626+0000 7ff799a00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff77406c8f0 0x7ff77406edb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ff780036000 tx=0x7ff78003d040 comp rx=0 tx=0).stop 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.626+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ae140 msgr2=0x7ff79409d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.626+0000 7ff799a00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ae140 0x7ff79409d060 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff78400c3f0 tx=0x7ff78400c8b0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 shutdown_connections 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff79415b1f0 0x7ff79409aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7940d0010 0x7ff79409a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff77406c8f0 0x7ff77406edb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 --2- 172.21.15.47:0/2725603175 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff7940ae140 0x7ff79409d060 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 >> 172.21.15.47:0/2725603175 conn(0x7ff7940b5270 msgr2=0x7ff794154550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 shutdown_connections 2024-04-20T10:28:34.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.627+0000 7ff799a00640 1 -- 172.21.15.47:0/2725603175 wait complete. 2024-04-20T10:28:34.647 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair' 2024-04-20T10:28:34.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.860+0000 7f5441600640 1 Processor -- start 2024-04-20T10:28:34.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.860+0000 7f5441600640 1 -- start start 2024-04-20T10:28:34.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.862+0000 7f5441600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f543c1651d0 0x7f543c071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.862+0000 7f5441600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.862+0000 7f5441600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f543c06c310 con 0x7f543c163070 2024-04-20T10:28:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.862+0000 7f5441600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f543c073cf0 con 0x7f543c1651d0 2024-04-20T10:28:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.862+0000 7f5441600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f543c06c5f0 con 0x7f543c163450 2024-04-20T10:28:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.862+0000 7f543be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f543c163070 0x7f543c0735e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33212/0 (socket says 172.21.15.47:33212) 2024-04-20T10:28:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.862+0000 7f543be00640 1 -- 172.21.15.47:0/2195236960 learned_addr learned my addr 172.21.15.47:0/2195236960 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:34.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f5433e00640 1 --2- 172.21.15.47:0/2195236960 >> v2:172.21.15.47:3300/0 conn(0x7f543c1651d0 0x7f543c071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f5440c00640 1 --2- 172.21.15.47:0/2195236960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f5433e00640 1 -- 172.21.15.47:0/2195236960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 msgr2=0x7f543c1638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f5433e00640 1 --2- 172.21.15.47:0/2195236960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f5433e00640 1 -- 172.21.15.47:0/2195236960 >> v1:172.21.15.47:6789/0 conn(0x7f543c163070 legacy=0x7f543c0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f5433e00640 1 -- 172.21.15.47:0/2195236960 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c06c8d0 con 0x7f543c1651d0 2024-04-20T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f543b400640 1 -- 172.21.15.47:0/2195236960 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3559839515 0 0) 0x7f543c06c310 con 0x7f543c163070 2024-04-20T10:28:34.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.863+0000 7f5440c00640 1 --2- 172.21.15.47:0/2195236960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.864+0000 7f5433e00640 1 --2- 172.21.15.47:0/2195236960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c1651d0 0x7f543c071d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54240097a0 tx=0x7f542402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f35f51cc21f3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f543b400640 1 -- 172.21.15.47:0/2195236960 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5424031780 con 0x7f543c1651d0 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f543b400640 1 -- 172.21.15.47:0/2195236960 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5424037070 con 0x7f543c1651d0 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f543b400640 1 -- 172.21.15.47:0/2195236960 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5424031b40 con 0x7f543c1651d0 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f5441600640 1 -- 172.21.15.47:0/2195236960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c1651d0 msgr2=0x7f543c071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f5441600640 1 --2- 172.21.15.47:0/2195236960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c1651d0 0x7f543c071d90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f54240097a0 tx=0x7f542402b820 comp rx=0 tx=0).stop 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f5441600640 1 -- 172.21.15.47:0/2195236960 shutdown_connections 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f5441600640 1 --2- 172.21.15.47:0/2195236960 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f5441600640 1 --2- 172.21.15.47:0/2195236960 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c1651d0 0x7f543c071d90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.865+0000 7f5441600640 1 -- 172.21.15.47:0/2195236960 >> 172.21.15.47:0/2195236960 conn(0x7f543c0d0040 msgr2=0x7f543c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.866+0000 7f5441600640 1 -- 172.21.15.47:0/2195236960 shutdown_connections 2024-04-20T10:28:34.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.866+0000 7f5441600640 1 -- 172.21.15.47:0/2195236960 wait complete. 2024-04-20T10:28:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.867+0000 7f5441600640 1 Processor -- start 2024-04-20T10:28:34.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.867+0000 7f5441600640 1 -- start start 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.868+0000 7f5441600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c163070 0x7f543c1f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.868+0000 7f5441600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1f8d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.868+0000 7f5441600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f543c1651d0 0x7f543c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.868+0000 7f5441600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f543c163e20 con 0x7f543c163070 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.868+0000 7f5441600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f543c06ce90 con 0x7f543c1651d0 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.868+0000 7f5441600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f543c1662a0 con 0x7f543c163450 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c163070 0x7f543c1f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c163070 0x7f543c1f94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44572/0 (socket says 172.21.15.47:44572) 2024-04-20T10:28:34.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 -- 172.21.15.47:0/3390348735 learned_addr learned my addr 172.21.15.47:0/3390348735 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f5433e00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1f8d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 -- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 msgr2=0x7f543c1f8d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1f8d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f5440c00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f543c1651d0 0x7f543c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 -- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f543c1651d0 msgr2=0x7f543c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f543c1651d0 0x7f543c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f543be00640 1 -- 172.21.15.47:0/3390348735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f543c1f9a00 con 0x7f543c163070 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.869+0000 7f5433e00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.870+0000 7f543be00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c163070 0x7f543c1f94c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f542c00c460 tx=0x7f542c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.870+0000 7f5438c00640 1 -- 172.21.15.47:0/3390348735 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c0074e0 con 0x7f543c163070 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.870+0000 7f5438c00640 1 -- 172.21.15.47:0/3390348735 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542c0047e0 con 0x7f543c163070 2024-04-20T10:28:34.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.870+0000 7f5438c00640 1 -- 172.21.15.47:0/3390348735 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542c004bd0 con 0x7f543c163070 2024-04-20T10:28:34.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.870+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f543c1f92c0 con 0x7f543c163070 2024-04-20T10:28:34.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.870+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f543c0c5fb0 con 0x7f543c163070 2024-04-20T10:28:34.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.871+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f543c075b60 con 0x7f543c163070 2024-04-20T10:28:34.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.872+0000 7f5438c00640 1 -- 172.21.15.47:0/3390348735 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f542c011070 con 0x7f543c163070 2024-04-20T10:28:34.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.874+0000 7f5438c00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f541806c920 0x7f541806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:34.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.874+0000 7f5438c00640 1 -- 172.21.15.47:0/3390348735 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f542c08ba10 con 0x7f543c163070 2024-04-20T10:28:34.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.875+0000 7f5433e00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f541806c920 0x7f541806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:34.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.876+0000 7f5433e00640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f541806c920 0x7f541806ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5424036000 tx=0x7f5424008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:34.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:34.879+0000 7f5438c00640 1 -- 172.21.15.47:0/3390348735 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f543c1f92c0 con 0x7f543c163070 2024-04-20T10:28:35.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.155+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]} v 0) -- 0x7f543c161e50 con 0x7f543c163070 2024-04-20T10:28:35.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.457+0000 7f5438c00640 1 -- 172.21.15.47:0/3390348735 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]}]=0 v82) ==== 151+0+0 (secure 0 0 0) 0x7f542c00e280 con 0x7f543c163070 2024-04-20T10:28:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.462+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f541806c920 msgr2=0x7f541806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.462+0000 7f5441600640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f541806c920 0x7f541806ede0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5424036000 tx=0x7f5424008030 comp rx=0 tx=0).stop 2024-04-20T10:28:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.462+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c163070 msgr2=0x7f543c1f94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.462+0000 7f5441600640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c163070 0x7f543c1f94c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f542c00c460 tx=0x7f542c00c920 comp rx=0 tx=0).stop 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 shutdown_connections 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f543c1651d0 0x7f543c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f543c163450 0x7f543c1f8d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f541806c920 0x7f541806ede0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 --2- 172.21.15.47:0/3390348735 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f543c163070 0x7f543c1f94c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 >> 172.21.15.47:0/3390348735 conn(0x7f543c0d0040 msgr2=0x7f543c0cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 shutdown_connections 2024-04-20T10:28:35.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.463+0000 7f5441600640 1 -- 172.21.15.47:0/3390348735 wait complete. 2024-04-20T10:28:35.487 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T10:28:35.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.706+0000 7f624e800640 1 Processor -- start 2024-04-20T10:28:35.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.706+0000 7f624e800640 1 -- start start 2024-04-20T10:28:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.708+0000 7f624e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480d3b20 0x7f62480b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.708+0000 7f624e800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f62480d0980 con 0x7f6248161a50 2024-04-20T10:28:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.708+0000 7f624e800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6248162490 con 0x7f624806c010 2024-04-20T10:28:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.708+0000 7f624e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62480d0c60 con 0x7f62480d3b20 2024-04-20T10:28:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.708+0000 7f624d400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6248161a50 0x7f624806b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33222/0 (socket says 172.21.15.47:33222) 2024-04-20T10:28:35.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.708+0000 7f624d400640 1 -- 172.21.15.47:0/1562261143 learned_addr learned my addr 172.21.15.47:0/1562261143 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f624ca00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480d3b20 0x7f62480b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3960380033 0 0) 0x7f62480d0980 con 0x7f6248161a50 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6230003520 con 0x7f6248161a50 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3948478881 0 0) 0x7f6230003520 con 0x7f6248161a50 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480d3b20 msgr2=0x7f62480b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480d3b20 0x7f62480b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 >> v1:172.21.15.175:6789/0 conn(0x7f624806c010 legacy=0x7f62480ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6248162770 con 0x7f6248161a50 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.709+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f623c001670 con 0x7f6248161a50 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 >> v1:172.21.15.47:6789/0 conn(0x7f6248161a50 legacy=0x7f624806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f6247e00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62300039e0 0x7f6230003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f6247e00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6230004800 0x7f6230004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f6247e00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6230005180 0x7f623000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f62480d0c60 con 0x7f6230005180 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f623000bf00 con 0x7f62300039e0 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f623000c0d0 con 0x7f6230004800 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f624de00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6230005180 0x7f623000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f624d400640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62300039e0 0x7f6230003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f624ca00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6230004800 0x7f6230004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.710+0000 7f624de00640 1 -- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6230004800 msgr2=0x7f6230004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.711+0000 7f624de00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6230004800 0x7f6230004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.711+0000 7f624de00640 1 -- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62300039e0 msgr2=0x7f6230003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.711+0000 7f624de00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62300039e0 0x7f6230003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.711+0000 7f624de00640 1 -- 172.21.15.47:0/1562261143 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6230003520 con 0x7f6230005180 2024-04-20T10:28:35.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.712+0000 7f624de00640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6230005180 0x7f623000b9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6240001e80 tx=0x7f6230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7532d6b26491278f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.712+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6240001a70 con 0x7f6230005180 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.712+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6240003b70 con 0x7f6230005180 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.712+0000 7f6247e00640 1 -- 172.21.15.47:0/1562261143 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62400050d0 con 0x7f6230005180 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.713+0000 7f624e800640 1 -- 172.21.15.47:0/1562261143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6230005180 msgr2=0x7f623000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.713+0000 7f624e800640 1 --2- 172.21.15.47:0/1562261143 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6230005180 0x7f623000b9c0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6240001e80 tx=0x7f6230005830 comp rx=0 tx=0).stop 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.713+0000 7f624de00640 1 -- 172.21.15.47:0/1562261143 reap_dead start 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.713+0000 7f624e800640 1 -- 172.21.15.47:0/1562261143 shutdown_connections 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.713+0000 7f624e800640 1 -- 172.21.15.47:0/1562261143 >> 172.21.15.47:0/1562261143 conn(0x7f62480d0ed0 msgr2=0x7f62480d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.713+0000 7f624e800640 1 -- 172.21.15.47:0/1562261143 shutdown_connections 2024-04-20T10:28:35.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.713+0000 7f624e800640 1 -- 172.21.15.47:0/1562261143 wait complete. 2024-04-20T10:28:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.715+0000 7f624e800640 1 Processor -- start 2024-04-20T10:28:35.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.715+0000 7f624e800640 1 -- start start 2024-04-20T10:28:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624815bd50 0x7f624815c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480cbd00 0x7f62480cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62480cc6a0 0x7f62480c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624e800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6248162ae0 con 0x7f624815bd50 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62480d06d0 con 0x7f62480cc6a0 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6248161e30 con 0x7f62480cbd00 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624815bd50 0x7f624815c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624815bd50 0x7f624815c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44590/0 (socket says 172.21.15.47:44590) 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 -- 172.21.15.47:0/4289893520 learned_addr learned my addr 172.21.15.47:0/4289893520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 -- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480cbd00 msgr2=0x7f62480cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480cbd00 0x7f62480cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 -- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62480cc6a0 msgr2=0x7f62480c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62480cc6a0 0x7f62480c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.716+0000 7f624d400640 1 -- 172.21.15.47:0/4289893520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62480ccb20 con 0x7f624815bd50 2024-04-20T10:28:35.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.717+0000 7f624d400640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624815bd50 0x7f624815c130 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f623c00eb20 tx=0x7f623c001690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.717+0000 7f6245600640 1 -- 172.21.15.47:0/4289893520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c00c850 con 0x7f624815bd50 2024-04-20T10:28:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.717+0000 7f6245600640 1 -- 172.21.15.47:0/4289893520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f623c018070 con 0x7f624815bd50 2024-04-20T10:28:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.717+0000 7f6245600640 1 -- 172.21.15.47:0/4289893520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c00cc50 con 0x7f624815bd50 2024-04-20T10:28:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.717+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f624815c670 con 0x7f624815bd50 2024-04-20T10:28:35.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.717+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62480c8510 con 0x7f624815bd50 2024-04-20T10:28:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.718+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6248075b30 con 0x7f624815bd50 2024-04-20T10:28:35.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.719+0000 7f6245600640 1 -- 172.21.15.47:0/4289893520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f623c01d460 con 0x7f624815bd50 2024-04-20T10:28:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.720+0000 7f6245600640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f622806c920 0x7f622806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.720+0000 7f6245600640 1 -- 172.21.15.47:0/4289893520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f623c08be80 con 0x7f624815bd50 2024-04-20T10:28:35.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.721+0000 7f624ca00640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f622806c920 0x7f622806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:35.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.722+0000 7f624ca00640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f622806c920 0x7f622806ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f624806da60 tx=0x7f6234008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:35.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.724+0000 7f6245600640 1 -- 172.21.15.47:0/4289893520 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f623c059fc0 con 0x7f624815bd50 2024-04-20T10:28:35.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.931+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f624815d070 con 0x7f624815bd50 2024-04-20T10:28:35.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.937+0000 7f6245600640 1 -- 172.21.15.47:0/4289893520 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) ==== 122+0+0 (secure 0 0 0) 0x7f623c0598c0 con 0x7f624815bd50 2024-04-20T10:28:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f622806c920 msgr2=0x7f622806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f622806c920 0x7f622806ede0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f624806da60 tx=0x7f6234008040 comp rx=0 tx=0).stop 2024-04-20T10:28:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624815bd50 msgr2=0x7f624815c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624815bd50 0x7f624815c130 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f623c00eb20 tx=0x7f623c001690 comp rx=0 tx=0).stop 2024-04-20T10:28:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 shutdown_connections 2024-04-20T10:28:35.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62480cc6a0 0x7f62480c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62480cbd00 0x7f62480cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f622806c920 0x7f622806ede0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 --2- 172.21.15.47:0/4289893520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f624815bd50 0x7f624815c130 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 >> 172.21.15.47:0/4289893520 conn(0x7f62480d0ed0 msgr2=0x7f62480b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 shutdown_connections 2024-04-20T10:28:35.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:35.940+0000 7f624e800640 1 -- 172.21.15.47:0/4289893520 wait complete. 2024-04-20T10:28:35.961 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T10:28:35.961 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:28:36.061 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:28:36.181 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T10:28:36.181 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:28:36.181 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:36.239 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:28:36.239 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:36.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.495+0000 7f625ec00640 1 Processor -- start 2024-04-20T10:28:36.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.495+0000 7f625ec00640 1 -- start start 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625ec00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6258159e10 0x7f6258071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625ec00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f625806da90 0x7f6258071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625ec00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f625815cc90 con 0x7f625815a1f0 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625ec00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6258072900 con 0x7f625806da90 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625ec00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f625815ca40 con 0x7f6258159e10 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625ce00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f625815a1f0 0x7f6258160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33228/0 (socket says 172.21.15.47:33228) 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625ce00640 1 -- 172.21.15.47:0/567274487 learned_addr learned my addr 172.21.15.47:0/567274487 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:36.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.497+0000 7f625e200640 1 --2- 172.21.15.47:0/567274487 >> v2:172.21.15.47:3300/0 conn(0x7f625806da90 0x7f6258071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.498+0000 7f625d800640 1 --2- 172.21.15.47:0/567274487 >> v2:172.21.15.175:3300/0 conn(0x7f6258159e10 0x7f6258071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.498+0000 7f625e200640 1 -- 172.21.15.47:0/567274487 >> v2:172.21.15.175:3300/0 conn(0x7f6258159e10 msgr2=0x7f6258071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.498+0000 7f625e200640 1 --2- 172.21.15.47:0/567274487 >> v2:172.21.15.175:3300/0 conn(0x7f6258159e10 0x7f6258071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.498+0000 7f625e200640 1 -- 172.21.15.47:0/567274487 >> v1:172.21.15.47:6789/0 conn(0x7f625815a1f0 legacy=0x7f6258160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.498+0000 7f6257e00640 1 -- 172.21.15.47:0/567274487 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1384079213 0 0) 0x7f625815cc90 con 0x7f625815a1f0 2024-04-20T10:28:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.498+0000 7f625e200640 1 -- 172.21.15.47:0/567274487 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f625815cf70 con 0x7f625806da90 2024-04-20T10:28:36.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.498+0000 7f625d800640 1 --2- 172.21.15.47:0/567274487 >> v2:172.21.15.175:3300/0 conn(0x7f6258159e10 0x7f6258071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.499+0000 7f625e200640 1 --2- 172.21.15.47:0/567274487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625806da90 0x7f6258071fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6258073f30 tx=0x7f625002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208abae0e5bb8b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f6257e00640 1 -- 172.21.15.47:0/567274487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250039070 con 0x7f625806da90 2024-04-20T10:28:36.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f6257e00640 1 -- 172.21.15.47:0/567274487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f625002cc80 con 0x7f625806da90 2024-04-20T10:28:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f6257e00640 1 -- 172.21.15.47:0/567274487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250034660 con 0x7f625806da90 2024-04-20T10:28:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 -- 172.21.15.47:0/567274487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625806da90 msgr2=0x7f6258071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 --2- 172.21.15.47:0/567274487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625806da90 0x7f6258071fc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f6258073f30 tx=0x7f625002bed0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 -- 172.21.15.47:0/567274487 shutdown_connections 2024-04-20T10:28:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 --2- 172.21.15.47:0/567274487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625806da90 0x7f6258071fc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 --2- 172.21.15.47:0/567274487 >> v2:172.21.15.175:3300/0 conn(0x7f6258159e10 0x7f6258071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 -- 172.21.15.47:0/567274487 >> 172.21.15.47:0/567274487 conn(0x7f62580d0040 msgr2=0x7f62580d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 -- 172.21.15.47:0/567274487 shutdown_connections 2024-04-20T10:28:36.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.500+0000 7f625ec00640 1 -- 172.21.15.47:0/567274487 wait complete. 2024-04-20T10:28:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.502+0000 7f625ec00640 1 Processor -- start 2024-04-20T10:28:36.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.502+0000 7f625ec00640 1 -- start start 2024-04-20T10:28:36.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f625806da90 0x7f62580cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6258159e10 0x7f625815ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625ec00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625815a1f0 0x7f625815b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625ec00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f625815d810 con 0x7f625815a1f0 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f625815d530 con 0x7f625806da90 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f625806b900 con 0x7f6258159e10 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625815a1f0 0x7f625815b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625815a1f0 0x7f625815b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44606/0 (socket says 172.21.15.47:44606) 2024-04-20T10:28:36.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.503+0000 7f625e200640 1 -- 172.21.15.47:0/3650266948 learned_addr learned my addr 172.21.15.47:0/3650266948 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.504+0000 7f625e200640 1 -- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6258159e10 msgr2=0x7f625815ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.504+0000 7f625e200640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6258159e10 0x7f625815ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.504+0000 7f625e200640 1 -- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f625806da90 msgr2=0x7f62580cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.504+0000 7f625e200640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f625806da90 0x7f62580cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.504+0000 7f625e200640 1 -- 172.21.15.47:0/3650266948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62580cd410 con 0x7f625815a1f0 2024-04-20T10:28:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.504+0000 7f625e200640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625815a1f0 0x7f625815b210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6250037040 tx=0x7f6250004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.504+0000 7f6255600640 1 -- 172.21.15.47:0/3650266948 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250039050 con 0x7f625815a1f0 2024-04-20T10:28:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.505+0000 7f6255600640 1 -- 172.21.15.47:0/3650266948 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6250045070 con 0x7f625815a1f0 2024-04-20T10:28:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.505+0000 7f6255600640 1 -- 172.21.15.47:0/3650266948 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625003e810 con 0x7f625815a1f0 2024-04-20T10:28:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.505+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f625815f390 con 0x7f625815a1f0 2024-04-20T10:28:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.505+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62581581c0 con 0x7f625815a1f0 2024-04-20T10:28:36.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.506+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6258075b60 con 0x7f625815a1f0 2024-04-20T10:28:36.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.507+0000 7f6255600640 1 -- 172.21.15.47:0/3650266948 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f625003e9b0 con 0x7f625815a1f0 2024-04-20T10:28:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.508+0000 7f6255600640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f623806c8f0 0x7f623806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.508+0000 7f6255600640 1 -- 172.21.15.47:0/3650266948 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f62500aef40 con 0x7f625815a1f0 2024-04-20T10:28:36.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.508+0000 7f625d800640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f623806c8f0 0x7f623806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.510+0000 7f625d800640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f623806c8f0 0x7f623806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f624c006fd0 tx=0x7f624c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:36.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.512+0000 7f6255600640 1 -- 172.21.15.47:0/3650266948 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f62580cd410 con 0x7f625815a1f0 2024-04-20T10:28:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.720+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62580cba90 con 0x7f625815a1f0 2024-04-20T10:28:36.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.723+0000 7f6255600640 1 -- 172.21.15.47:0/3650266948 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f625007d040 con 0x7f625815a1f0 2024-04-20T10:28:36.723 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:36.723 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:36.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f623806c8f0 msgr2=0x7f623806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f623806c8f0 0x7f623806edb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f624c006fd0 tx=0x7f624c008040 comp rx=0 tx=0).stop 2024-04-20T10:28:36.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625815a1f0 msgr2=0x7f625815b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625815a1f0 0x7f625815b210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f6250037040 tx=0x7f6250004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 shutdown_connections 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f623806c8f0 0x7f623806edb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f625815a1f0 0x7f625815b210 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6258159e10 0x7f625815ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 --2- 172.21.15.47:0/3650266948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f625806da90 0x7f62580cced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 >> 172.21.15.47:0/3650266948 conn(0x7f62580d0040 msgr2=0x7f625806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 shutdown_connections 2024-04-20T10:28:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.726+0000 7f625ec00640 1 -- 172.21.15.47:0/3650266948 wait complete. 2024-04-20T10:28:36.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:36.748 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.958+0000 7f8a12000640 1 Processor -- start 2024-04-20T10:28:36.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.958+0000 7f8a12000640 1 -- start start 2024-04-20T10:28:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.960+0000 7f8a12000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 0x7f8a0c13e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.960+0000 7f8a12000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8a0c0adef0 0x7f8a0c15b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.960+0000 7f8a12000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8a0c15e5d0 con 0x7f8a0c0cad70 2024-04-20T10:28:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.960+0000 7f8a12000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8a0c15e870 con 0x7f8a0c0adef0 2024-04-20T10:28:36.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.960+0000 7f8a12000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c15eb50 con 0x7f8a0c154fe0 2024-04-20T10:28:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.960+0000 7f8a0be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8a0c0cad70 0x7f8a0c0ad7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33234/0 (socket says 172.21.15.47:33234) 2024-04-20T10:28:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.960+0000 7f8a0be00640 1 -- 172.21.15.47:0/3162399154 learned_addr learned my addr 172.21.15.47:0/3162399154 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a10c00640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 0x7f8a0c13e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a11600640 1 --2- 172.21.15.47:0/3162399154 >> v2:172.21.15.175:3300/0 conn(0x7f8a0c0adef0 0x7f8a0c15b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2114681220 0 0) 0x7f8a0c15e5d0 con 0x7f8a0c0cad70 2024-04-20T10:28:36.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f89f4003520 con 0x7f8a0c0cad70 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 843715515 0 0) 0x7f89f4003520 con 0x7f8a0c0cad70 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 msgr2=0x7f8a0c13e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 0x7f8a0c13e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 >> v2:172.21.15.175:3300/0 conn(0x7f8a0c0adef0 msgr2=0x7f8a0c15b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 --2- 172.21.15.47:0/3162399154 >> v2:172.21.15.175:3300/0 conn(0x7f8a0c0adef0 0x7f8a0c15b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c15ee30 con 0x7f8a0c0cad70 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f89f8001690 con 0x7f8a0c0cad70 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 >> v1:172.21.15.47:6789/0 conn(0x7f8a0c0cad70 legacy=0x7f8a0c0ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f40039b0 0x7f89f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.961+0000 7f8a0b400640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 0x7f89f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a0b400640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f4004d40 0x7f89f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c15e870 con 0x7f89f40039b0 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c15eb50 con 0x7f89f4004d40 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f89f400dbd0 con 0x7f8a0c154fe0 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a10c00640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f40039b0 0x7f89f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a0be00640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 0x7f89f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a11600640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f4004d40 0x7f89f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a10c00640 1 -- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 msgr2=0x7f89f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a10c00640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a0c154fe0 0x7f89f4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a10c00640 1 -- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f4004d40 msgr2=0x7f89f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a10c00640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89f4004d40 0x7f89f40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.962+0000 7f8a10c00640 1 -- 172.21.15.47:0/3162399154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89f4003520 con 0x7f89f40039b0 2024-04-20T10:28:36.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.963+0000 7f8a10c00640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f40039b0 0x7f89f4003dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009ac0 tx=0x7f89f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd75abbf9aea0718 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:36.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00039070 con 0x7f89f40039b0 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a0002c6b0 con 0x7f89f40039b0 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a0b400640 1 -- 172.21.15.47:0/3162399154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a00009690 con 0x7f89f40039b0 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a12000640 1 -- 172.21.15.47:0/3162399154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f40039b0 msgr2=0x7f89f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a12000640 1 --2- 172.21.15.47:0/3162399154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f89f40039b0 0x7f89f4003dd0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009ac0 tx=0x7f89f4005830 comp rx=0 tx=0).stop 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a11600640 1 -- 172.21.15.47:0/3162399154 reap_dead start 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a12000640 1 -- 172.21.15.47:0/3162399154 shutdown_connections 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.964+0000 7f8a12000640 1 -- 172.21.15.47:0/3162399154 >> 172.21.15.47:0/3162399154 conn(0x7f8a0c0cffa0 msgr2=0x7f8a0c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.965+0000 7f8a12000640 1 -- 172.21.15.47:0/3162399154 shutdown_connections 2024-04-20T10:28:36.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.965+0000 7f8a12000640 1 -- 172.21.15.47:0/3162399154 wait complete. 2024-04-20T10:28:36.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.966+0000 7f8a12000640 1 Processor -- start 2024-04-20T10:28:36.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.966+0000 7f8a12000640 1 -- start start 2024-04-20T10:28:36.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.967+0000 7f8a12000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89f40039b0 0x7f8a0c1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.967+0000 7f8a12000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a0c06f4c0 0x7f8a0c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.967+0000 7f8a12000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0c1f4b00 0x7f8a0c1f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.967+0000 7f8a12000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c1553c0 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.967+0000 7f8a12000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a0c15f3f0 con 0x7f8a0c06f4c0 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.967+0000 7f8a12000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a0c0cb1f0 con 0x7f89f40039b0 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0c1f4b00 0x7f8a0c1f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0c1f4b00 0x7f8a0c1f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44616/0 (socket says 172.21.15.47:44616) 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 -- 172.21.15.47:0/860399065 learned_addr learned my addr 172.21.15.47:0/860399065 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a10c00640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89f40039b0 0x7f8a0c1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 -- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89f40039b0 msgr2=0x7f8a0c1f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89f40039b0 0x7f8a0c1f8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 -- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a0c06f4c0 msgr2=0x7f8a0c15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a0c06f4c0 0x7f8a0c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.968+0000 7f8a11600640 1 -- 172.21.15.47:0/860399065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a0c073e40 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.969+0000 7f8a11600640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0c1f4b00 0x7f8a0c1f4f80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c15afe0 tx=0x7f8a0400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:36.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.969+0000 7f8a08c00640 1 -- 172.21.15.47:0/860399065 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0401c070 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.969+0000 7f8a08c00640 1 -- 172.21.15.47:0/860399065 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a04017400 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.969+0000 7f8a08c00640 1 -- 172.21.15.47:0/860399065 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a040177f0 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.969+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a0c06fa80 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.969+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a0c1558c0 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.970+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a0c075b30 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.971+0000 7f8a08c00640 1 -- 172.21.15.47:0/860399065 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a04005d10 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.972+0000 7f8a08c00640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89ec06c8f0 0x7f89ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:36.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.972+0000 7f8a08c00640 1 -- 172.21.15.47:0/860399065 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f8a0408c350 con 0x7f8a0c1f4b00 2024-04-20T10:28:36.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.973+0000 7f8a10c00640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89ec06c8f0 0x7f89ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.974+0000 7f8a10c00640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89ec06c8f0 0x7f89ec06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a00036000 tx=0x7f8a00030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:36.977+0000 7f8a08c00640 1 -- 172.21.15.47:0/860399065 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a0c073e40 con 0x7f8a0c1f4b00 2024-04-20T10:28:37.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.189+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8a0c06bec0 con 0x7f8a0c1f4b00 2024-04-20T10:28:37.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.190+0000 7f8a08c00640 1 -- 172.21.15.47:0/860399065 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f8a04012070 con 0x7f8a0c1f4b00 2024-04-20T10:28:37.190 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:37.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89ec06c8f0 msgr2=0x7f89ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89ec06c8f0 0x7f89ec06edb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a00036000 tx=0x7f8a00030000 comp rx=0 tx=0).stop 2024-04-20T10:28:37.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0c1f4b00 msgr2=0x7f8a0c1f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0c1f4b00 0x7f8a0c1f4f80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c15afe0 tx=0x7f8a0400f4a0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 shutdown_connections 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f89ec06c8f0 0x7f89ec06edb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a0c1f4b00 0x7f8a0c1f4f80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a0c06f4c0 0x7f8a0c15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 --2- 172.21.15.47:0/860399065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89f40039b0 0x7f8a0c1f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 >> 172.21.15.47:0/860399065 conn(0x7f8a0c0cffa0 msgr2=0x7f8a0c0cf880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 shutdown_connections 2024-04-20T10:28:37.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.193+0000 7f8a12000640 1 -- 172.21.15.47:0/860399065 wait complete. 2024-04-20T10:28:37.213 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.428+0000 7fc299600640 1 Processor -- start 2024-04-20T10:28:37.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.429+0000 7fc299600640 1 -- start start 2024-04-20T10:28:37.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.430+0000 7fc299600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc29415b3b0 0x7fc29406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.430+0000 7fc299600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc29406dcb0 0x7fc294071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.430+0000 7fc299600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc2940ade40 con 0x7fc294072090 2024-04-20T10:28:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.430+0000 7fc299600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc2940ae0e0 con 0x7fc29406dcb0 2024-04-20T10:28:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.430+0000 7fc299600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc29415b790 con 0x7fc29415b3b0 2024-04-20T10:28:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.430+0000 7fc292000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc29406dcb0 0x7fc294071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.430+0000 7fc292000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc29406dcb0 0x7fc294071b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44632/0 (socket says 172.21.15.47:44632) 2024-04-20T10:28:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.431+0000 7fc292000640 1 -- 172.21.15.47:0/4111789895 learned_addr learned my addr 172.21.15.47:0/4111789895 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:37.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.431+0000 7fc292a00640 1 --2- 172.21.15.47:0/4111789895 >> v2:172.21.15.175:3300/0 conn(0x7fc29415b3b0 0x7fc29406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.431+0000 7fc292000640 1 -- 172.21.15.47:0/4111789895 >> v2:172.21.15.175:3300/0 conn(0x7fc29415b3b0 msgr2=0x7fc29406eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:37.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.431+0000 7fc292000640 1 --2- 172.21.15.47:0/4111789895 >> v2:172.21.15.175:3300/0 conn(0x7fc29415b3b0 0x7fc29406eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.431+0000 7fc292000640 1 -- 172.21.15.47:0/4111789895 >> v1:172.21.15.47:6789/0 conn(0x7fc294072090 legacy=0x7fc2940d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.431+0000 7fc292000640 1 -- 172.21.15.47:0/4111789895 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2940ae380 con 0x7fc29406dcb0 2024-04-20T10:28:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.432+0000 7fc292000640 1 --2- 172.21.15.47:0/4111789895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29406dcb0 0x7fc294071b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc29406cfa0 tx=0x7fc28002b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cc2bdd677a82bb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.432+0000 7fc291600640 1 -- 172.21.15.47:0/4111789895 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280034070 con 0x7fc29406dcb0 2024-04-20T10:28:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.432+0000 7fc291600640 1 -- 172.21.15.47:0/4111789895 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc280030870 con 0x7fc29406dcb0 2024-04-20T10:28:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc291600640 1 -- 172.21.15.47:0/4111789895 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280030c40 con 0x7fc29406dcb0 2024-04-20T10:28:37.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 -- 172.21.15.47:0/4111789895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29406dcb0 msgr2=0x7fc294071b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 --2- 172.21.15.47:0/4111789895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29406dcb0 0x7fc294071b50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc29406cfa0 tx=0x7fc28002b9b0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 -- 172.21.15.47:0/4111789895 shutdown_connections 2024-04-20T10:28:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 --2- 172.21.15.47:0/4111789895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29406dcb0 0x7fc294071b50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 --2- 172.21.15.47:0/4111789895 >> v2:172.21.15.175:3300/0 conn(0x7fc29415b3b0 0x7fc29406eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 -- 172.21.15.47:0/4111789895 >> 172.21.15.47:0/4111789895 conn(0x7fc2940b3350 msgr2=0x7fc2940b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 -- 172.21.15.47:0/4111789895 shutdown_connections 2024-04-20T10:28:37.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.433+0000 7fc299600640 1 -- 172.21.15.47:0/4111789895 wait complete. 2024-04-20T10:28:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.435+0000 7fc299600640 1 Processor -- start 2024-04-20T10:28:37.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.435+0000 7fc299600640 1 -- start start 2024-04-20T10:28:37.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc299600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29406dcb0 0x7fc2940b0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc299600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29415b3b0 0x7fc2940cf520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc299600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2940af4c0 0x7fc294151ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc299600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc2940aec20 con 0x7fc29415b3b0 2024-04-20T10:28:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc299600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2940ae940 con 0x7fc29406dcb0 2024-04-20T10:28:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc299600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc29406efe0 con 0x7fc2940af4c0 2024-04-20T10:28:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc292000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29415b3b0 0x7fc2940cf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc292000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29415b3b0 0x7fc2940cf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44638/0 (socket says 172.21.15.47:44638) 2024-04-20T10:28:37.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.436+0000 7fc292000640 1 -- 172.21.15.47:0/162515154 learned_addr learned my addr 172.21.15.47:0/162515154 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.437+0000 7fc292000640 1 -- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2940af4c0 msgr2=0x7fc294151ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.437+0000 7fc292000640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2940af4c0 0x7fc294151ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.437+0000 7fc292000640 1 -- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29406dcb0 msgr2=0x7fc2940b0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.437+0000 7fc292000640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29406dcb0 0x7fc2940b0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.437+0000 7fc292000640 1 -- 172.21.15.47:0/162515154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2940afa80 con 0x7fc29415b3b0 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.437+0000 7fc292000640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29415b3b0 0x7fc2940cf520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc280000c00 tx=0x7fc280004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.438+0000 7fc28aa00640 1 -- 172.21.15.47:0/162515154 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280048070 con 0x7fc29415b3b0 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.438+0000 7fc28aa00640 1 -- 172.21.15.47:0/162515154 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc280030670 con 0x7fc29415b3b0 2024-04-20T10:28:37.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.438+0000 7fc28aa00640 1 -- 172.21.15.47:0/162515154 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc280034040 con 0x7fc29415b3b0 2024-04-20T10:28:37.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.438+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2940afd60 con 0x7fc29415b3b0 2024-04-20T10:28:37.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.438+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2940b0320 con 0x7fc29415b3b0 2024-04-20T10:28:37.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.439+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc294068980 con 0x7fc29415b3b0 2024-04-20T10:28:37.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.440+0000 7fc28aa00640 1 -- 172.21.15.47:0/162515154 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc28004b070 con 0x7fc29415b3b0 2024-04-20T10:28:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.441+0000 7fc28aa00640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc27406c8f0 0x7fc27406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.441+0000 7fc28aa00640 1 -- 172.21.15.47:0/162515154 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fc2800ae090 con 0x7fc29415b3b0 2024-04-20T10:28:37.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.442+0000 7fc292a00640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc27406c8f0 0x7fc27406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.443+0000 7fc292a00640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc27406c8f0 0x7fc27406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc284006fd0 tx=0x7fc284008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:37.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.446+0000 7fc28aa00640 1 -- 172.21.15.47:0/162515154 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc28007c070 con 0x7fc29415b3b0 2024-04-20T10:28:37.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.654+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fc294072d20 con 0x7fc29415b3b0 2024-04-20T10:28:37.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.655+0000 7fc28aa00640 1 -- 172.21.15.47:0/162515154 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fc28007b970 con 0x7fc29415b3b0 2024-04-20T10:28:37.655 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:37.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc27406c8f0 msgr2=0x7fc27406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc27406c8f0 0x7fc27406edb0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc284006fd0 tx=0x7fc284008040 comp rx=0 tx=0).stop 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29415b3b0 msgr2=0x7fc2940cf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29415b3b0 0x7fc2940cf520 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc280000c00 tx=0x7fc280004730 comp rx=0 tx=0).stop 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 shutdown_connections 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2940af4c0 0x7fc294151ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc27406c8f0 0x7fc27406edb0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc29415b3b0 0x7fc2940cf520 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 --2- 172.21.15.47:0/162515154 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc29406dcb0 0x7fc2940b0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.657+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 >> 172.21.15.47:0/162515154 conn(0x7fc2940b3350 msgr2=0x7fc2941583f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.658+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 shutdown_connections 2024-04-20T10:28:37.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.658+0000 7fc299600640 1 -- 172.21.15.47:0/162515154 wait complete. 2024-04-20T10:28:37.677 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:37.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.891+0000 7f6ca9200640 1 Processor -- start 2024-04-20T10:28:37.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.891+0000 7f6ca9200640 1 -- start start 2024-04-20T10:28:37.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca415a130 0x7f6ca4073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca9200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6ca4071a60 0x7f6ca40739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca9200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ca406f160 0x7f6ca406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca9200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca415d7f0 con 0x7f6ca406f160 2024-04-20T10:28:37.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca9200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6ca415cc20 con 0x7f6ca4071a60 2024-04-20T10:28:37.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca415cec0 con 0x7f6ca415a130 2024-04-20T10:28:37.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ca406f160 0x7f6ca406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.893+0000 7f6ca3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ca406f160 0x7f6ca406f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44646/0 (socket says 172.21.15.47:44646) 2024-04-20T10:28:37.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca3e00640 1 -- 172.21.15.47:0/2682111067 learned_addr learned my addr 172.21.15.47:0/2682111067 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca3400640 1 --2- 172.21.15.47:0/2682111067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca415a130 0x7f6ca4073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca2a00640 1 --2- 172.21.15.47:0/2682111067 >> v2:172.21.15.175:3300/0 conn(0x7f6ca4071a60 0x7f6ca40739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca3e00640 1 -- 172.21.15.47:0/2682111067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca415a130 msgr2=0x7f6ca4073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca3e00640 1 --2- 172.21.15.47:0/2682111067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca415a130 0x7f6ca4073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca3e00640 1 -- 172.21.15.47:0/2682111067 >> v2:172.21.15.175:3300/0 conn(0x7f6ca4071a60 msgr2=0x7f6ca40739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca3e00640 1 --2- 172.21.15.47:0/2682111067 >> v2:172.21.15.175:3300/0 conn(0x7f6ca4071a60 0x7f6ca40739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca3e00640 1 -- 172.21.15.47:0/2682111067 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca415d1a0 con 0x7f6ca406f160 2024-04-20T10:28:37.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.894+0000 7f6ca2a00640 1 --2- 172.21.15.47:0/2682111067 >> v2:172.21.15.175:3300/0 conn(0x7f6ca4071a60 0x7f6ca40739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.895+0000 7f6ca3e00640 1 --2- 172.21.15.47:0/2682111067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca406f160 0x7f6ca406f5f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40cc8c0 tx=0x7f6c9c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1211d750da57049a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.896+0000 7f6ca2000640 1 -- 172.21.15.47:0/2682111067 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c02c9f0 con 0x7f6ca406f160 2024-04-20T10:28:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.896+0000 7f6ca2000640 1 -- 172.21.15.47:0/2682111067 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9c02cb90 con 0x7f6ca406f160 2024-04-20T10:28:37.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.896+0000 7f6ca2000640 1 -- 172.21.15.47:0/2682111067 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c0343f0 con 0x7f6ca406f160 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 -- 172.21.15.47:0/2682111067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca406f160 msgr2=0x7f6ca406f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 --2- 172.21.15.47:0/2682111067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca406f160 0x7f6ca406f5f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f6ca40cc8c0 tx=0x7f6c9c02be50 comp rx=0 tx=0).stop 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 -- 172.21.15.47:0/2682111067 shutdown_connections 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 --2- 172.21.15.47:0/2682111067 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca406f160 0x7f6ca406f5f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 --2- 172.21.15.47:0/2682111067 >> v2:172.21.15.175:3300/0 conn(0x7f6ca4071a60 0x7f6ca40739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 --2- 172.21.15.47:0/2682111067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca415a130 0x7f6ca4073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 -- 172.21.15.47:0/2682111067 >> 172.21.15.47:0/2682111067 conn(0x7f6ca40cffa0 msgr2=0x7f6ca40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 -- 172.21.15.47:0/2682111067 shutdown_connections 2024-04-20T10:28:37.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.897+0000 7f6ca9200640 1 -- 172.21.15.47:0/2682111067 wait complete. 2024-04-20T10:28:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.899+0000 7f6ca9200640 1 Processor -- start 2024-04-20T10:28:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.899+0000 7f6ca9200640 1 -- start start 2024-04-20T10:28:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406f160 0x7f6ca40b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca4071a60 0x7f6ca40cb9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca415a130 0x7f6ca40b0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca9200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca4160100 con 0x7f6ca415a130 2024-04-20T10:28:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ca406fb30 con 0x7f6ca406f160 2024-04-20T10:28:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ca4160280 con 0x7f6ca4071a60 2024-04-20T10:28:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca415a130 0x7f6ca40b0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca415a130 0x7f6ca40b0980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44652/0 (socket says 172.21.15.47:44652) 2024-04-20T10:28:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.900+0000 7f6ca3e00640 1 -- 172.21.15.47:0/3552286061 learned_addr learned my addr 172.21.15.47:0/3552286061 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca3400640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406f160 0x7f6ca40b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca2a00640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca4071a60 0x7f6ca40cb9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca3e00640 1 -- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca4071a60 msgr2=0x7f6ca40cb9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca3e00640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca4071a60 0x7f6ca40cb9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca3e00640 1 -- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406f160 msgr2=0x7f6ca40b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca3e00640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406f160 0x7f6ca40b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca3e00640 1 -- 172.21.15.47:0/3552286061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ca414c4f0 con 0x7f6ca415a130 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6ca3e00640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca415a130 0x7f6ca40b0980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c036040 tx=0x7f6c9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.901+0000 7f6c9b400640 1 -- 172.21.15.47:0/3552286061 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c043020 con 0x7f6ca415a130 2024-04-20T10:28:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.902+0000 7f6c9b400640 1 -- 172.21.15.47:0/3552286061 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6c9c03e450 con 0x7f6ca415a130 2024-04-20T10:28:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.902+0000 7f6c9b400640 1 -- 172.21.15.47:0/3552286061 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c9c009510 con 0x7f6ca415a130 2024-04-20T10:28:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.902+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ca414c210 con 0x7f6ca415a130 2024-04-20T10:28:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.902+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ca415e960 con 0x7f6ca415a130 2024-04-20T10:28:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.903+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ca40d0db0 con 0x7f6ca415a130 2024-04-20T10:28:37.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.904+0000 7f6c9b400640 1 -- 172.21.15.47:0/3552286061 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6c9c046020 con 0x7f6ca415a130 2024-04-20T10:28:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.905+0000 7f6c9b400640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6c8406c8f0 0x7f6c8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.905+0000 7f6c9b400640 1 -- 172.21.15.47:0/3552286061 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f6c9c0ae7d0 con 0x7f6ca415a130 2024-04-20T10:28:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.905+0000 7f6ca3400640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6c8406c8f0 0x7f6c8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.907+0000 7f6ca3400640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6c8406c8f0 0x7f6c8406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c94006fd0 tx=0x7f6c94008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:37.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:37.911+0000 7f6c9b400640 1 -- 172.21.15.47:0/3552286061 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6ca40d0db0 con 0x7f6ca415a130 2024-04-20T10:28:38.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.116+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6ca4075b30 con 0x7f6ca415a130 2024-04-20T10:28:38.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.117+0000 7f6c9b400640 1 -- 172.21.15.47:0/3552286061 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f6c9c04d030 con 0x7f6ca415a130 2024-04-20T10:28:38.117 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:38.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6c8406c8f0 msgr2=0x7f6c8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6c8406c8f0 0x7f6c8406edb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f6c94006fd0 tx=0x7f6c94008040 comp rx=0 tx=0).stop 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca415a130 msgr2=0x7f6ca40b0980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca415a130 0x7f6ca40b0980 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f6c9c036040 tx=0x7f6c9c004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 shutdown_connections 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6c8406c8f0 0x7f6c8406edb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ca415a130 0x7f6ca40b0980 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ca4071a60 0x7f6ca40cb9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 --2- 172.21.15.47:0/3552286061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ca406f160 0x7f6ca40b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 >> 172.21.15.47:0/3552286061 conn(0x7f6ca40cffa0 msgr2=0x7f6ca406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 shutdown_connections 2024-04-20T10:28:38.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.120+0000 7f6ca9200640 1 -- 172.21.15.47:0/3552286061 wait complete. 2024-04-20T10:28:38.140 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.357+0000 7fdd51000640 1 Processor -- start 2024-04-20T10:28:38.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.357+0000 7fdd51000640 1 -- start start 2024-04-20T10:28:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.359+0000 7fdd51000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 0x7fdd4c0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.359+0000 7fdd51000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdd4c06e060 0x7fdd4c06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.359+0000 7fdd51000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdd4c0af780 con 0x7fdd4c15d4f0 2024-04-20T10:28:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.359+0000 7fdd51000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdd4c06c510 con 0x7fdd4c06e060 2024-04-20T10:28:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.359+0000 7fdd51000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4c06b900 con 0x7fdd4c0aee60 2024-04-20T10:28:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.359+0000 7fdd4aa00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fdd4c15d4f0 0x7fdd4c0ad0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33252/0 (socket says 172.21.15.47:33252) 2024-04-20T10:28:38.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.359+0000 7fdd4aa00640 1 -- 172.21.15.47:0/3444135473 learned_addr learned my addr 172.21.15.47:0/3444135473 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:38.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4be00640 1 --2- 172.21.15.47:0/3444135473 >> v2:172.21.15.175:3300/0 conn(0x7fdd4c06e060 0x7fdd4c06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4b400640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 0x7fdd4c0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4174361722 0 0) 0x7fdd4c0af780 con 0x7fdd4c15d4f0 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fdd30003520 con 0x7fdd4c15d4f0 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3497378764 0 0) 0x7fdd30003520 con 0x7fdd4c15d4f0 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 msgr2=0x7fdd4c0af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 0x7fdd4c0af240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 >> v2:172.21.15.175:3300/0 conn(0x7fdd4c06e060 msgr2=0x7fdd4c06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 --2- 172.21.15.47:0/3444135473 >> v2:172.21.15.175:3300/0 conn(0x7fdd4c06e060 0x7fdd4c06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c06bbe0 con 0x7fdd4c15d4f0 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fdd38001670 con 0x7fdd4c15d4f0 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 >> v1:172.21.15.47:6789/0 conn(0x7fdd4c15d4f0 legacy=0x7fdd4c0ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 0x7fdd300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.360+0000 7fdd4a000640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30003f30 0x7fdd30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4a000640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd30004d40 0x7fdd300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c06c510 con 0x7fdd30004d40 2024-04-20T10:28:38.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c06b900 con 0x7fdd30003f30 2024-04-20T10:28:38.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3000b9c0 con 0x7fdd4c0aee60 2024-04-20T10:28:38.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4be00640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd30004d40 0x7fdd300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4aa00640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30003f30 0x7fdd30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4b400640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 0x7fdd300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4be00640 1 -- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 msgr2=0x7fdd300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4be00640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 0x7fdd300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4be00640 1 -- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30003f30 msgr2=0x7fdd30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4be00640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30003f30 0x7fdd30004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4be00640 1 -- 172.21.15.47:0/3444135473 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd30003520 con 0x7fdd30004d40 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.361+0000 7fdd4b400640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c0aee60 0x7fdd300039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.362+0000 7fdd4be00640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd30004d40 0x7fdd300051c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009fc0 tx=0x7fdd30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b0dac9e3176707 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.363+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44039070 con 0x7fdd30004d40 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.363+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd4402cc40 con 0x7fdd30004d40 2024-04-20T10:28:38.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.363+0000 7fdd4a000640 1 -- 172.21.15.47:0/3444135473 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd44009c10 con 0x7fdd30004d40 2024-04-20T10:28:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.363+0000 7fdd51000640 1 -- 172.21.15.47:0/3444135473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd30004d40 msgr2=0x7fdd300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.363+0000 7fdd51000640 1 --2- 172.21.15.47:0/3444135473 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd30004d40 0x7fdd300051c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fdd44009fc0 tx=0x7fdd30005830 comp rx=0 tx=0).stop 2024-04-20T10:28:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.363+0000 7fdd4be00640 1 -- 172.21.15.47:0/3444135473 reap_dead start 2024-04-20T10:28:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.364+0000 7fdd51000640 1 -- 172.21.15.47:0/3444135473 shutdown_connections 2024-04-20T10:28:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.364+0000 7fdd51000640 1 -- 172.21.15.47:0/3444135473 >> 172.21.15.47:0/3444135473 conn(0x7fdd4c0b5270 msgr2=0x7fdd4c0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:38.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.364+0000 7fdd51000640 1 -- 172.21.15.47:0/3444135473 shutdown_connections 2024-04-20T10:28:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.364+0000 7fdd51000640 1 -- 172.21.15.47:0/3444135473 wait complete. 2024-04-20T10:28:38.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.366+0000 7fdd51000640 1 Processor -- start 2024-04-20T10:28:38.366 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.366+0000 7fdd51000640 1 -- start start 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd51000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30004d40 0x7fdd4c14dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd51000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd4c14c0b0 0x7fdd4c14c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd51000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c15fb20 0x7fdd4c15ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd51000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c06c9a0 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd51000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdd4c06c1a0 con 0x7fdd30004d40 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd51000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdd4c0af240 con 0x7fdd4c15fb20 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd4aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd4c14c0b0 0x7fdd4c14c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd4aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd4c14c0b0 0x7fdd4c14c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44676/0 (socket says 172.21.15.47:44676) 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd4aa00640 1 -- 172.21.15.47:0/485065513 learned_addr learned my addr 172.21.15.47:0/485065513 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd4b400640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30004d40 0x7fdd4c14dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.367+0000 7fdd4be00640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c15fb20 0x7fdd4c15ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd4aa00640 1 -- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c15fb20 msgr2=0x7fdd4c15ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd4aa00640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c15fb20 0x7fdd4c15ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd4aa00640 1 -- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30004d40 msgr2=0x7fdd4c14dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd4aa00640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30004d40 0x7fdd4c14dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd4aa00640 1 -- 172.21.15.47:0/485065513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd4c1f6060 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd4be00640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c15fb20 0x7fdd4c15ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:38.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd4aa00640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd4c14c0b0 0x7fdd4c14c530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd38000f60 tx=0x7fdd3800dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd43400640 1 -- 172.21.15.47:0/485065513 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38019070 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd43400640 1 -- 172.21.15.47:0/485065513 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdd380074e0 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd43400640 1 -- 172.21.15.47:0/485065513 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd38007690 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd4c14cc20 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.368+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd4c14d1e0 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.370+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd4c075b30 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.371+0000 7fdd43400640 1 -- 172.21.15.47:0/485065513 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdd38004a00 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.372+0000 7fdd43400640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd2c06c920 0x7fdd2c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.372+0000 7fdd43400640 1 -- 172.21.15.47:0/485065513 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fdd3808b9e0 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.372+0000 7fdd4b400640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd2c06c920 0x7fdd2c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.373+0000 7fdd4b400640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd2c06c920 0x7fdd2c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009aa0 tx=0x7fdd3c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:38.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.376+0000 7fdd43400640 1 -- 172.21.15.47:0/485065513 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdd380599d0 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.586+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fdd4c099e40 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.587+0000 7fdd43400640 1 -- 172.21.15.47:0/485065513 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fdd380592d0 con 0x7fdd4c14c0b0 2024-04-20T10:28:38.587 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.589+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd2c06c920 msgr2=0x7fdd2c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.589+0000 7fdd51000640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd2c06c920 0x7fdd2c06ede0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c009aa0 tx=0x7fdd3c008040 comp rx=0 tx=0).stop 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.589+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd4c14c0b0 msgr2=0x7fdd4c14c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.589+0000 7fdd51000640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd4c14c0b0 0x7fdd4c14c530 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdd38000f60 tx=0x7fdd3800dd20 comp rx=0 tx=0).stop 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 shutdown_connections 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdd4c15fb20 0x7fdd4c15ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdd2c06c920 0x7fdd2c06ede0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdd4c14c0b0 0x7fdd4c14c530 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 --2- 172.21.15.47:0/485065513 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdd30004d40 0x7fdd4c14dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 >> 172.21.15.47:0/485065513 conn(0x7fdd4c0b5270 msgr2=0x7fdd4c0b1230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 shutdown_connections 2024-04-20T10:28:38.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.590+0000 7fdd51000640 1 -- 172.21.15.47:0/485065513 wait complete. 2024-04-20T10:28:38.610 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.826+0000 7f5902800640 1 Processor -- start 2024-04-20T10:28:38.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.826+0000 7f5902800640 1 -- start start 2024-04-20T10:28:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.828+0000 7f5902800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58fc15b7e0 0x7f58fc15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.828+0000 7f5902800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58fc06ccc0 con 0x7f58fc15c190 2024-04-20T10:28:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.828+0000 7f5902800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58fc06cfa0 con 0x7f58fc15b400 2024-04-20T10:28:38.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.828+0000 7f5902800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f58fc06d280 con 0x7f58fc15b7e0 2024-04-20T10:28:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.828+0000 7f5900a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58fc15b7e0 0x7f58fc15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.828+0000 7f5900a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f58fc15b7e0 0x7f58fc15bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44682/0 (socket says 172.21.15.47:44682) 2024-04-20T10:28:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.828+0000 7f5900a00640 1 -- 172.21.15.47:0/337504026 learned_addr learned my addr 172.21.15.47:0/337504026 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:38.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.829+0000 7f5900a00640 1 -- 172.21.15.47:0/337504026 >> v1:172.21.15.175:6789/0 conn(0x7f58fc15b400 legacy=0x7f58fc072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.829+0000 7f5900a00640 1 -- 172.21.15.47:0/337504026 >> v1:172.21.15.47:6789/0 conn(0x7f58fc15c190 legacy=0x7f58fc06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.829+0000 7f5900a00640 1 -- 172.21.15.47:0/337504026 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc06e440 con 0x7f58fc15b7e0 2024-04-20T10:28:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.830+0000 7f5900a00640 1 --2- 172.21.15.47:0/337504026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc15bc50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e802dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c17acd40448a6b73 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:38.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.830+0000 7f58fbe00640 1 -- 172.21.15.47:0/337504026 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e803b070 con 0x7f58fc15b7e0 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.830+0000 7f58fbe00640 1 -- 172.21.15.47:0/337504026 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e802e960 con 0x7f58fc15b7e0 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.830+0000 7f58fbe00640 1 -- 172.21.15.47:0/337504026 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e802ed30 con 0x7f58fc15b7e0 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.831+0000 7f5902800640 1 -- 172.21.15.47:0/337504026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 msgr2=0x7f58fc15bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.831+0000 7f5902800640 1 --2- 172.21.15.47:0/337504026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc15bc50 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e802dbb0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.831+0000 7f5902800640 1 -- 172.21.15.47:0/337504026 shutdown_connections 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.831+0000 7f5902800640 1 --2- 172.21.15.47:0/337504026 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc15bc50 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.831+0000 7f5902800640 1 -- 172.21.15.47:0/337504026 >> 172.21.15.47:0/337504026 conn(0x7f58fc0d0040 msgr2=0x7f58fc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.831+0000 7f5902800640 1 -- 172.21.15.47:0/337504026 shutdown_connections 2024-04-20T10:28:38.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.831+0000 7f5902800640 1 -- 172.21.15.47:0/337504026 wait complete. 2024-04-20T10:28:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.833+0000 7f5902800640 1 Processor -- start 2024-04-20T10:28:38.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.833+0000 7f5902800640 1 -- start start 2024-04-20T10:28:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5902800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58fc15b400 0x7f58fc0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5902800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5902800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc15c190 0x7f58fc0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5902800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc15c770 con 0x7f58fc15b7e0 2024-04-20T10:28:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5902800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc15c5f0 con 0x7f58fc15c190 2024-04-20T10:28:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5902800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58fc06f350 con 0x7f58fc15b400 2024-04-20T10:28:38.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5900a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5900a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc0cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44696/0 (socket says 172.21.15.47:44696) 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.834+0000 7f5900a00640 1 -- 172.21.15.47:0/1710771871 learned_addr learned my addr 172.21.15.47:0/1710771871 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5900a00640 1 -- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58fc15b400 msgr2=0x7f58fc0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5901400640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58fc15b400 0x7f58fc0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5901e00640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc15c190 0x7f58fc0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5900a00640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58fc15b400 0x7f58fc0cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5900a00640 1 -- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc15c190 msgr2=0x7f58fc0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5900a00640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc15c190 0x7f58fc0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5900a00640 1 -- 172.21.15.47:0/1710771871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc0ce710 con 0x7f58fc15b7e0 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5901e00640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc15c190 0x7f58fc0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:38.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f5900a00640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc0cd270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.835+0000 7f58f9600640 1 -- 172.21.15.47:0/1710771871 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8045020 con 0x7f58fc15b7e0 2024-04-20T10:28:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.836+0000 7f58f9600640 1 -- 172.21.15.47:0/1710771871 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f58e8036d90 con 0x7f58fc15b7e0 2024-04-20T10:28:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.836+0000 7f58f9600640 1 -- 172.21.15.47:0/1710771871 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e803b040 con 0x7f58fc15b7e0 2024-04-20T10:28:38.837 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.836+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc0ce9f0 con 0x7f58fc15b7e0 2024-04-20T10:28:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.836+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc0cefb0 con 0x7f58fc15b7e0 2024-04-20T10:28:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.837+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58fc075b60 con 0x7f58fc15b7e0 2024-04-20T10:28:38.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.838+0000 7f58f9600640 1 -- 172.21.15.47:0/1710771871 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f58e8048020 con 0x7f58fc15b7e0 2024-04-20T10:28:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.839+0000 7f58f9600640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f58dc06c8f0 0x7f58dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.839+0000 7f58f9600640 1 -- 172.21.15.47:0/1710771871 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f58e80b0030 con 0x7f58fc15b7e0 2024-04-20T10:28:38.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.840+0000 7f5901400640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f58dc06c8f0 0x7f58dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:38.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.841+0000 7f5901400640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f58dc06c8f0 0x7f58dc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007fd0 tx=0x7f58f0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:38.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:38.843+0000 7f58f9600640 1 -- 172.21.15.47:0/1710771871 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f58e804f030 con 0x7f58fc15b7e0 2024-04-20T10:28:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.058+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f58fc0ae630 con 0x7f58fc15b7e0 2024-04-20T10:28:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.058+0000 7f58f9600640 1 -- 172.21.15.47:0/1710771871 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f58e807e0b0 con 0x7f58fc15b7e0 2024-04-20T10:28:39.058 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f58dc06c8f0 msgr2=0x7f58dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f58dc06c8f0 0x7f58dc06edb0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007fd0 tx=0x7f58f0009040 comp rx=0 tx=0).stop 2024-04-20T10:28:39.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 msgr2=0x7f58fc0cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc0cd270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009870 tx=0x7f58e80046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 shutdown_connections 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58fc15c190 0x7f58fc0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f58dc06c8f0 0x7f58dc06edb0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f58fc15b7e0 0x7f58fc0cd270 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 --2- 172.21.15.47:0/1710771871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58fc15b400 0x7f58fc0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 >> 172.21.15.47:0/1710771871 conn(0x7f58fc0d0040 msgr2=0x7f58fc0afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 shutdown_connections 2024-04-20T10:28:39.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.061+0000 7f5902800640 1 -- 172.21.15.47:0/1710771871 wait complete. 2024-04-20T10:28:39.080 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:28:39.080 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:39.133 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.344+0000 7fccc1a00640 1 Processor -- start 2024-04-20T10:28:39.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.345+0000 7fccc1a00640 1 -- start start 2024-04-20T10:28:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.346+0000 7fccc1a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fccbc0b3330 0x7fccbc0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.346+0000 7fccc1a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccbc06c190 con 0x7fccbc161d50 2024-04-20T10:28:39.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.346+0000 7fccc1a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fccbc06c470 con 0x7fccbc06cde0 2024-04-20T10:28:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.346+0000 7fccc1a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fccbc162130 con 0x7fccbc0b3330 2024-04-20T10:28:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.346+0000 7fccbbe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fccbc161d50 0x7fccbc06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33266/0 (socket says 172.21.15.47:33266) 2024-04-20T10:28:39.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.346+0000 7fccbbe00640 1 -- 172.21.15.47:0/930940188 learned_addr learned my addr 172.21.15.47:0/930940188 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccc1000640 1 --2- 172.21.15.47:0/930940188 >> v2:172.21.15.175:3300/0 conn(0x7fccbc0b3330 0x7fccbc0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3996554877 0 0) 0x7fccbc06c190 con 0x7fccbc161d50 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcc9c003520 con 0x7fccbc161d50 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 297772721 0 0) 0x7fcc9c003520 con 0x7fccbc161d50 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 >> v2:172.21.15.175:3300/0 conn(0x7fccbc0b3330 msgr2=0x7fccbc0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 --2- 172.21.15.47:0/930940188 >> v2:172.21.15.175:3300/0 conn(0x7fccbc0b3330 0x7fccbc0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 >> v1:172.21.15.175:6789/0 conn(0x7fccbc06cde0 legacy=0x7fccbc0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc0d2e00 con 0x7fccbc161d50 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fccac001670 con 0x7fccbc161d50 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 >> v1:172.21.15.47:6789/0 conn(0x7fccbc161d50 legacy=0x7fccbc06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc9c003a00 0x7fcc9c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc9c004800 0x7fcc9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fcc9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c00c090 con 0x7fcc9c0051c0 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcc9c00c240 con 0x7fcc9c003a00 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.347+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcc9c00c440 con 0x7fcc9c004800 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccc1000640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fcc9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccbb400640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc9c004800 0x7fcc9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccbbe00640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc9c003a00 0x7fcc9c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccc1000640 1 -- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc9c004800 msgr2=0x7fcc9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccc1000640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcc9c004800 0x7fcc9c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccc1000640 1 -- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc9c003a00 msgr2=0x7fcc9c003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccc1000640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcc9c003a00 0x7fcc9c003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.348+0000 7fccc1000640 1 -- 172.21.15.47:0/930940188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9c003520 con 0x7fcc9c0051c0 2024-04-20T10:28:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.349+0000 7fccc1000640 1 -- 172.21.15.47:0/930940188 reap_dead start 2024-04-20T10:28:39.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.349+0000 7fccc1000640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fcc9c00b9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009ac0 tx=0x7fcc9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da5be4132449589c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4009690 con 0x7fcc9c0051c0 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccb4004020 con 0x7fcc9c0051c0 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccbaa00640 1 -- 172.21.15.47:0/930940188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4004410 con 0x7fcc9c0051c0 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccc1a00640 1 -- 172.21.15.47:0/930940188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 msgr2=0x7fcc9c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccc1a00640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fcc9c00b9c0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fccb4009ac0 tx=0x7fcc9c005830 comp rx=0 tx=0).stop 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccc1a00640 1 -- 172.21.15.47:0/930940188 shutdown_connections 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccc1a00640 1 --2- 172.21.15.47:0/930940188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fcc9c00b9c0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccc1a00640 1 -- 172.21.15.47:0/930940188 >> 172.21.15.47:0/930940188 conn(0x7fccbc0d4450 msgr2=0x7fccbc0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccc1a00640 1 -- 172.21.15.47:0/930940188 shutdown_connections 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.350+0000 7fccc1a00640 1 -- 172.21.15.47:0/930940188 wait complete. 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.352+0000 7fccc1a00640 1 Processor -- start 2024-04-20T10:28:39.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.352+0000 7fccc1a00640 1 -- start start 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.353+0000 7fccc1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fccbc1605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.353+0000 7fccc1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0cb520 0x7fccbc14f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.353+0000 7fccc1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccbc0cb9a0 0x7fccbc14f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.353+0000 7fccc1a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc0d38e0 con 0x7fcc9c0051c0 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.353+0000 7fccc1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fccbc0d33c0 con 0x7fccbc0cb520 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.353+0000 7fccc1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fccbc0d3a60 con 0x7fccbc0cb9a0 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fccbc1605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fccbc1605c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44712/0 (socket says 172.21.15.47:44712) 2024-04-20T10:28:39.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 -- 172.21.15.47:0/1879425340 learned_addr learned my addr 172.21.15.47:0/1879425340 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 -- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccbc0cb9a0 msgr2=0x7fccbc14f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccc1000640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccbc0cb9a0 0x7fccbc14f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbb400640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0cb520 0x7fccbc14f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccbc0cb9a0 0x7fccbc14f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 -- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0cb520 msgr2=0x7fccbc14f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0cb520 0x7fccbc14f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 -- 172.21.15.47:0/1879425340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccbc0cbf00 con 0x7fcc9c0051c0 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.354+0000 7fccbbe00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fccbc1605c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fccac00d7a0 tx=0x7fccac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.355+0000 7fccb3e00640 1 -- 172.21.15.47:0/1879425340 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac019070 con 0x7fcc9c0051c0 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.355+0000 7fccb3e00640 1 -- 172.21.15.47:0/1879425340 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fccac00de50 con 0x7fcc9c0051c0 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.355+0000 7fccb3e00640 1 -- 172.21.15.47:0/1879425340 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccac0047e0 con 0x7fcc9c0051c0 2024-04-20T10:28:39.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.355+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccbc0cc1e0 con 0x7fcc9c0051c0 2024-04-20T10:28:39.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.355+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccbc0cc7a0 con 0x7fcc9c0051c0 2024-04-20T10:28:39.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.356+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccbc075b60 con 0x7fcc9c0051c0 2024-04-20T10:28:39.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.357+0000 7fccb3e00640 1 -- 172.21.15.47:0/1879425340 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fccac004980 con 0x7fcc9c0051c0 2024-04-20T10:28:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.358+0000 7fccb3e00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcc9806c8f0 0x7fcc9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.358+0000 7fccb3e00640 1 -- 172.21.15.47:0/1879425340 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fccac08b860 con 0x7fcc9c0051c0 2024-04-20T10:28:39.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.359+0000 7fccbb400640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcc9806c8f0 0x7fcc9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.360+0000 7fccbb400640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcc9806c8f0 0x7fcc9806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcca4000f80 tx=0x7fcca4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:39.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.362+0000 7fccb3e00640 1 -- 172.21.15.47:0/1879425340 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fccac059840 con 0x7fcc9c0051c0 2024-04-20T10:28:39.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.567+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fccbc0cf4a0 con 0x7fcc9c0051c0 2024-04-20T10:28:39.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.569+0000 7fccb3e00640 1 -- 172.21.15.47:0/1879425340 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7fccac059140 con 0x7fcc9c0051c0 2024-04-20T10:28:39.570 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:39.570 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcc9806c8f0 msgr2=0x7fcc9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcc9806c8f0 0x7fcc9806edb0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcca4000f80 tx=0x7fcca4009040 comp rx=0 tx=0).stop 2024-04-20T10:28:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 msgr2=0x7fccbc1605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fccbc1605c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fccac00d7a0 tx=0x7fccac00dc60 comp rx=0 tx=0).stop 2024-04-20T10:28:39.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 shutdown_connections 2024-04-20T10:28:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccbc0cb9a0 0x7fccbc14f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccbc0cb520 0x7fccbc14f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcc9806c8f0 0x7fcc9806edb0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 --2- 172.21.15.47:0/1879425340 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcc9c0051c0 0x7fccbc1605c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 >> 172.21.15.47:0/1879425340 conn(0x7fccbc0d4450 msgr2=0x7fccbc0b8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 shutdown_connections 2024-04-20T10:28:39.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.572+0000 7fccc1a00640 1 -- 172.21.15.47:0/1879425340 wait complete. 2024-04-20T10:28:39.591 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:39.591 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.798+0000 7f87e1200640 1 Processor -- start 2024-04-20T10:28:39.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.799+0000 7f87e1200640 1 -- start start 2024-04-20T10:28:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.801+0000 7f87e1200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f87dc15dbf0 0x7f87dc0716e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.801+0000 7f87e1200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87dc15d520 con 0x7f87dc15f890 2024-04-20T10:28:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.801+0000 7f87e1200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f87dc15d800 con 0x7f87dc071c20 2024-04-20T10:28:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.801+0000 7f87e1200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f87dc0d00c0 con 0x7f87dc15dbf0 2024-04-20T10:28:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.801+0000 7f87d9e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f87dc15f890 0x7f87dc15ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33268/0 (socket says 172.21.15.47:33268) 2024-04-20T10:28:39.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.801+0000 7f87d9e00640 1 -- 172.21.15.47:0/2936371854 learned_addr learned my addr 172.21.15.47:0/2936371854 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.801+0000 7f87d9400640 1 --2- 172.21.15.47:0/2936371854 >> v2:172.21.15.175:3300/0 conn(0x7f87dc15dbf0 0x7f87dc0716e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3997836580 0 0) 0x7f87dc15d520 con 0x7f87dc15f890 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87c0003520 con 0x7f87dc15f890 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1492105373 0 0) 0x7f87c0003520 con 0x7f87dc15f890 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 >> v2:172.21.15.175:3300/0 conn(0x7f87dc15dbf0 msgr2=0x7f87dc0716e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 --2- 172.21.15.47:0/2936371854 >> v2:172.21.15.175:3300/0 conn(0x7f87dc15dbf0 0x7f87dc0716e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 >> v1:172.21.15.175:6789/0 conn(0x7f87dc071c20 legacy=0x7f87dc0d0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc0d0310 con 0x7f87dc15f890 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f87cc0019f0 con 0x7f87dc15f890 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 >> v1:172.21.15.47:6789/0 conn(0x7f87dc15f890 legacy=0x7f87dc15ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0003a30 0x7f87c0003e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87c0004800 0x7f87c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00051c0 0x7f87c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0d00c0 con 0x7f87c0004800 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87c000c0f0 con 0x7f87c0003a30 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87c000c2f0 con 0x7f87c00051c0 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.802+0000 7f87d9400640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87c0004800 0x7f87c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87d9e00640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0003a30 0x7f87c0003e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87da800640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00051c0 0x7f87c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87d9400640 1 -- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00051c0 msgr2=0x7f87c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87d9400640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00051c0 0x7f87c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87d9400640 1 -- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0003a30 msgr2=0x7f87c0003e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87d9400640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0003a30 0x7f87c0003e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87d9400640 1 -- 172.21.15.47:0/2936371854 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87c0003520 con 0x7f87c0004800 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87d9e00640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0003a30 0x7f87c0003e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87da800640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87c00051c0 0x7f87c000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:39.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.803+0000 7f87da800640 1 -- 172.21.15.47:0/2936371854 reap_dead start 2024-04-20T10:28:39.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.804+0000 7f87d9400640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87c0004800 0x7f87c0004c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009ac0 tx=0x7f87c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1a97b779541948 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4039070 con 0x7f87c0004800 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87c402c6b0 con 0x7f87c0004800 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87d8a00640 1 -- 172.21.15.47:0/2936371854 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87c4009690 con 0x7f87c0004800 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87e1200640 1 -- 172.21.15.47:0/2936371854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87c0004800 msgr2=0x7f87c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87e1200640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87c0004800 0x7f87c0004c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009ac0 tx=0x7f87c0005830 comp rx=0 tx=0).stop 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87e1200640 1 -- 172.21.15.47:0/2936371854 shutdown_connections 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87e1200640 1 --2- 172.21.15.47:0/2936371854 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87c0004800 0x7f87c0004c80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87e1200640 1 -- 172.21.15.47:0/2936371854 >> 172.21.15.47:0/2936371854 conn(0x7f87dc0cedf0 msgr2=0x7f87dc0cf1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.805+0000 7f87e1200640 1 -- 172.21.15.47:0/2936371854 shutdown_connections 2024-04-20T10:28:39.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.806+0000 7f87e1200640 1 -- 172.21.15.47:0/2936371854 wait complete. 2024-04-20T10:28:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.807+0000 7f87e1200640 1 Processor -- start 2024-04-20T10:28:39.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.807+0000 7f87e1200640 1 -- start start 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.808+0000 7f87e1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0004800 0x7f87dc152490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.808+0000 7f87e1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc1e8a50 0x7f87dc1512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.808+0000 7f87e1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87dc0b6e10 0x7f87dc0b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.808+0000 7f87e1200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc15fc70 con 0x7f87dc0b6e10 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.808+0000 7f87e1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87dc0b39f0 con 0x7f87c0004800 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.808+0000 7f87e1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87dc15fdf0 con 0x7f87dc1e8a50 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87dc0b6e10 0x7f87dc0b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87dc0b6e10 0x7f87dc0b72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44730/0 (socket says 172.21.15.47:44730) 2024-04-20T10:28:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 -- 172.21.15.47:0/1725880606 learned_addr learned my addr 172.21.15.47:0/1725880606 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 -- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc1e8a50 msgr2=0x7f87dc1512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc1e8a50 0x7f87dc1512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 -- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0004800 msgr2=0x7f87dc152490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0004800 0x7f87dc152490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:39.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 -- 172.21.15.47:0/1725880606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87dc1517e0 con 0x7f87dc0b6e10 2024-04-20T10:28:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.809+0000 7f87da800640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87dc0b6e10 0x7f87dc0b72c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007f90 tx=0x7f87d400d5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.810+0000 7f87d2000640 1 -- 172.21.15.47:0/1725880606 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4018070 con 0x7f87dc0b6e10 2024-04-20T10:28:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.810+0000 7f87d2000640 1 -- 172.21.15.47:0/1725880606 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87d40052d0 con 0x7f87dc0b6e10 2024-04-20T10:28:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.810+0000 7f87d2000640 1 -- 172.21.15.47:0/1725880606 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87d4003830 con 0x7f87dc0b6e10 2024-04-20T10:28:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.810+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87dc153ba0 con 0x7f87dc0b6e10 2024-04-20T10:28:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.810+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87dc152de0 con 0x7f87dc0b6e10 2024-04-20T10:28:39.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.811+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87dc069090 con 0x7f87dc0b6e10 2024-04-20T10:28:39.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.812+0000 7f87d2000640 1 -- 172.21.15.47:0/1725880606 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87d40039d0 con 0x7f87dc0b6e10 2024-04-20T10:28:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.813+0000 7f87d2000640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87b806c920 0x7f87b806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.813+0000 7f87d2000640 1 -- 172.21.15.47:0/1725880606 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f87d408b240 con 0x7f87dc0b6e10 2024-04-20T10:28:39.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.814+0000 7f87d9e00640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87b806c920 0x7f87b806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:39.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.815+0000 7f87d9e00640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87b806c920 0x7f87b806ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87cc001b60 tx=0x7f87cc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:39.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:39.817+0000 7f87d2000640 1 -- 172.21.15.47:0/1725880606 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f87d4059220 con 0x7f87dc0b6e10 2024-04-20T10:28:40.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.024+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f87dc15e480 con 0x7f87dc0b6e10 2024-04-20T10:28:40.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.025+0000 7f87d2000640 1 -- 172.21.15.47:0/1725880606 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f87d4058b20 con 0x7f87dc0b6e10 2024-04-20T10:28:40.025 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87b806c920 msgr2=0x7f87b806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87b806c920 0x7f87b806ede0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f87cc001b60 tx=0x7f87cc009040 comp rx=0 tx=0).stop 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87dc0b6e10 msgr2=0x7f87dc0b72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87dc0b6e10 0x7f87dc0b72c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f87d4007f90 tx=0x7f87d400d5e0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 shutdown_connections 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87b806c920 0x7f87b806ede0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87dc0b6e10 0x7f87dc0b72c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87dc1e8a50 0x7f87dc1512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 --2- 172.21.15.47:0/1725880606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87c0004800 0x7f87dc152490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 >> 172.21.15.47:0/1725880606 conn(0x7f87dc0cedf0 msgr2=0x7f87dc0cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 shutdown_connections 2024-04-20T10:28:40.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.028+0000 7f87e1200640 1 -- 172.21.15.47:0/1725880606 wait complete. 2024-04-20T10:28:40.049 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:40.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.266+0000 7f1a87400640 1 Processor -- start 2024-04-20T10:28:40.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.266+0000 7f1a87400640 1 -- start start 2024-04-20T10:28:40.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.267+0000 7f1a87400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1a8015e8a0 0x7f1a8006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.267+0000 7f1a87400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8006dbb0 0x7f1a800d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.267+0000 7f1a87400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1a8006f060 con 0x7f1a800d08a0 2024-04-20T10:28:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.267+0000 7f1a87400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1a8006e7c0 con 0x7f1a8015e8a0 2024-04-20T10:28:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.267+0000 7f1a87400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a8006eaa0 con 0x7f1a8006dbb0 2024-04-20T10:28:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.268+0000 7f1a84a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1a8015e8a0 0x7f1a8006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.268+0000 7f1a7fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8006dbb0 0x7f1a800d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.268+0000 7f1a84a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1a8015e8a0 0x7f1a8006e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:47962/0 (socket says 172.21.15.47:47962) 2024-04-20T10:28:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.268+0000 7f1a84a00640 1 -- 172.21.15.47:0/4128549676 learned_addr learned my addr 172.21.15.47:0/4128549676 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.269+0000 7f1a84a00640 1 -- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8006dbb0 msgr2=0x7f1a800d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.269+0000 7f1a84a00640 1 --2- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8006dbb0 0x7f1a800d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.269+0000 7f1a84a00640 1 -- 172.21.15.47:0/4128549676 >> v1:172.21.15.175:6789/0 conn(0x7f1a800d08a0 legacy=0x7f1a8006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.269+0000 7f1a84a00640 1 -- 172.21.15.47:0/4128549676 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8006ed80 con 0x7f1a8015e8a0 2024-04-20T10:28:40.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.269+0000 7f1a7fe00640 1 --2- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8006dbb0 0x7f1a800d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:28:40.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.271+0000 7f1a84a00640 1 --2- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a8015e8a0 0x7f1a8006e1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009870 tx=0x7f1a7402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c1567b6f279cec server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a7f400640 1 -- 172.21.15.47:0/4128549676 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74034070 con 0x7f1a8015e8a0 2024-04-20T10:28:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a7f400640 1 -- 172.21.15.47:0/4128549676 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a74030840 con 0x7f1a8015e8a0 2024-04-20T10:28:40.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a7f400640 1 -- 172.21.15.47:0/4128549676 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74030c10 con 0x7f1a8015e8a0 2024-04-20T10:28:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a87400640 1 -- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a8015e8a0 msgr2=0x7f1a8006e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a87400640 1 --2- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a8015e8a0 0x7f1a8006e1e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1a74009870 tx=0x7f1a7402b980 comp rx=0 tx=0).stop 2024-04-20T10:28:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a87400640 1 -- 172.21.15.47:0/4128549676 shutdown_connections 2024-04-20T10:28:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a87400640 1 --2- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8006dbb0 0x7f1a800d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a87400640 1 --2- 172.21.15.47:0/4128549676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a8015e8a0 0x7f1a8006e1e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.272+0000 7f1a87400640 1 -- 172.21.15.47:0/4128549676 >> 172.21.15.47:0/4128549676 conn(0x7f1a800b3350 msgr2=0x7f1a800b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:40.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.273+0000 7f1a87400640 1 -- 172.21.15.47:0/4128549676 shutdown_connections 2024-04-20T10:28:40.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.273+0000 7f1a87400640 1 -- 172.21.15.47:0/4128549676 wait complete. 2024-04-20T10:28:40.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.274+0000 7f1a87400640 1 Processor -- start 2024-04-20T10:28:40.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.274+0000 7f1a87400640 1 -- start start 2024-04-20T10:28:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.275+0000 7f1a87400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8006dbb0 0x7f1a80099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.275+0000 7f1a87400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a800d08a0 0x7f1a80145870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.275+0000 7f1a87400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8015e8a0 0x7f1a8009b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.275+0000 7f1a87400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8006fef0 con 0x7f1a8006dbb0 2024-04-20T10:28:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.275+0000 7f1a87400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1a8006fd70 con 0x7f1a800d08a0 2024-04-20T10:28:40.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.275+0000 7f1a87400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1a800ae690 con 0x7f1a8015e8a0 2024-04-20T10:28:40.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8006dbb0 0x7f1a80099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8006dbb0 0x7f1a80099a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44736/0 (socket says 172.21.15.47:44736) 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 -- 172.21.15.47:0/880291686 learned_addr learned my addr 172.21.15.47:0/880291686 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a7fe00640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a800d08a0 0x7f1a80145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a85400640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8015e8a0 0x7f1a8009b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 -- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8015e8a0 msgr2=0x7f1a8009b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8015e8a0 0x7f1a8009b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 -- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a800d08a0 msgr2=0x7f1a80145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a800d08a0 0x7f1a80145870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.276+0000 7f1a84a00640 1 -- 172.21.15.47:0/880291686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a8009a6d0 con 0x7f1a8006dbb0 2024-04-20T10:28:40.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.277+0000 7f1a84a00640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8006dbb0 0x7f1a80099a40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1a74000c00 tx=0x7f1a740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.277+0000 7f1a7cc00640 1 -- 172.21.15.47:0/880291686 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74034040 con 0x7f1a8006dbb0 2024-04-20T10:28:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.277+0000 7f1a7cc00640 1 -- 172.21.15.47:0/880291686 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1a74039d90 con 0x7f1a8006dbb0 2024-04-20T10:28:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.277+0000 7f1a7cc00640 1 -- 172.21.15.47:0/880291686 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a74043610 con 0x7f1a8006dbb0 2024-04-20T10:28:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.277+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a8009a9b0 con 0x7f1a8006dbb0 2024-04-20T10:28:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.277+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a8009af70 con 0x7f1a8006dbb0 2024-04-20T10:28:40.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.278+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a80068980 con 0x7f1a8006dbb0 2024-04-20T10:28:40.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.279+0000 7f1a7cc00640 1 -- 172.21.15.47:0/880291686 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1a7404a060 con 0x7f1a8006dbb0 2024-04-20T10:28:40.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.281+0000 7f1a7cc00640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a6006c8f0 0x7f1a6006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.281+0000 7f1a7cc00640 1 -- 172.21.15.47:0/880291686 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f1a740adfa0 con 0x7f1a8006dbb0 2024-04-20T10:28:40.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.282+0000 7f1a7fe00640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a6006c8f0 0x7f1a6006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.283+0000 7f1a7fe00640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a6006c8f0 0x7f1a6006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009560 tx=0x7f1a6c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:40.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.285+0000 7f1a7cc00640 1 -- 172.21.15.47:0/880291686 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1a7403e070 con 0x7f1a8006dbb0 2024-04-20T10:28:40.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.487+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1a800b19b0 con 0x7f1a8006dbb0 2024-04-20T10:28:40.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.488+0000 7f1a7cc00640 1 -- 172.21.15.47:0/880291686 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f1a7402d330 con 0x7f1a8006dbb0 2024-04-20T10:28:40.488 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.490+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a6006c8f0 msgr2=0x7f1a6006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.490+0000 7f1a87400640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a6006c8f0 0x7f1a6006edb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1a6c009560 tx=0x7f1a6c008040 comp rx=0 tx=0).stop 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.490+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8006dbb0 msgr2=0x7f1a80099a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.490+0000 7f1a87400640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8006dbb0 0x7f1a80099a40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1a74000c00 tx=0x7f1a740046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 shutdown_connections 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1a8015e8a0 0x7f1a8009b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1a800d08a0 0x7f1a80145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1a6006c8f0 0x7f1a6006edb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 --2- 172.21.15.47:0/880291686 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1a8006dbb0 0x7f1a80099a40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 >> 172.21.15.47:0/880291686 conn(0x7f1a800b3350 msgr2=0x7f1a80071b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 shutdown_connections 2024-04-20T10:28:40.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.491+0000 7f1a87400640 1 -- 172.21.15.47:0/880291686 wait complete. 2024-04-20T10:28:40.510 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.722+0000 7f3405a00640 1 Processor -- start 2024-04-20T10:28:40.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.722+0000 7f3405a00640 1 -- start start 2024-04-20T10:28:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f3405a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f340015a130 0x7f3400073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f3405a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f340006f870 0x7f34000739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f3405a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f340015cc20 con 0x7f3400071a60 2024-04-20T10:28:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f3405a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f340015d7f0 con 0x7f340006f870 2024-04-20T10:28:40.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f3405a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f340015cec0 con 0x7f340015a130 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f33ff400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3400071a60 0x7f340006f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33270/0 (socket says 172.21.15.47:33270) 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f33ff400640 1 -- 172.21.15.47:0/3025848777 learned_addr learned my addr 172.21.15.47:0/3025848777 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.724+0000 7f3405000640 1 --2- 172.21.15.47:0/3025848777 >> v2:172.21.15.47:3300/0 conn(0x7f340006f870 0x7f34000739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f33ffe00640 1 --2- 172.21.15.47:0/3025848777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f340015a130 0x7f3400073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f33fea00640 1 -- 172.21.15.47:0/3025848777 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 695599849 0 0) 0x7f340015cc20 con 0x7f3400071a60 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f33fea00640 1 -- 172.21.15.47:0/3025848777 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f33e4003520 con 0x7f3400071a60 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f3405000640 1 -- 172.21.15.47:0/3025848777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f340015a130 msgr2=0x7f3400073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:40.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f3405000640 1 --2- 172.21.15.47:0/3025848777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f340015a130 0x7f3400073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f3405000640 1 -- 172.21.15.47:0/3025848777 >> v1:172.21.15.47:6789/0 conn(0x7f3400071a60 legacy=0x7f340006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f3405000640 1 -- 172.21.15.47:0/3025848777 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340015d1a0 con 0x7f340006f870 2024-04-20T10:28:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.725+0000 7f33ffe00640 1 --2- 172.21.15.47:0/3025848777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f340015a130 0x7f3400073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:40.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.727+0000 7f3405000640 1 --2- 172.21.15.47:0/3025848777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340006f870 0x7f34000739b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009d90 tx=0x7f33f802bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c766c6443b2d78ae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.727+0000 7f33fea00640 1 -- 172.21.15.47:0/3025848777 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8039070 con 0x7f340006f870 2024-04-20T10:28:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.727+0000 7f33fea00640 1 -- 172.21.15.47:0/3025848777 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f802cb30 con 0x7f340006f870 2024-04-20T10:28:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.727+0000 7f33fea00640 1 -- 172.21.15.47:0/3025848777 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8034440 con 0x7f340006f870 2024-04-20T10:28:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 -- 172.21.15.47:0/3025848777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340006f870 msgr2=0x7f34000739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 --2- 172.21.15.47:0/3025848777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340006f870 0x7f34000739b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009d90 tx=0x7f33f802bd80 comp rx=0 tx=0).stop 2024-04-20T10:28:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 -- 172.21.15.47:0/3025848777 shutdown_connections 2024-04-20T10:28:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 --2- 172.21.15.47:0/3025848777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340006f870 0x7f34000739b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 --2- 172.21.15.47:0/3025848777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f340015a130 0x7f3400073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 -- 172.21.15.47:0/3025848777 >> 172.21.15.47:0/3025848777 conn(0x7f34000cffa0 msgr2=0x7f34000d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 -- 172.21.15.47:0/3025848777 shutdown_connections 2024-04-20T10:28:40.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.728+0000 7f3405a00640 1 -- 172.21.15.47:0/3025848777 wait complete. 2024-04-20T10:28:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.730+0000 7f3405a00640 1 Processor -- start 2024-04-20T10:28:40.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.730+0000 7f3405a00640 1 -- start start 2024-04-20T10:28:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.731+0000 7f3405a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f340006f870 0x7f3400155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.731+0000 7f3405a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3400071a60 0x7f34001563c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.731+0000 7f3405a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340015a130 0x7f3400156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.731+0000 7f3405a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f340006ba80 con 0x7f340015a130 2024-04-20T10:28:40.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.731+0000 7f3405a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f340006b900 con 0x7f340006f870 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.731+0000 7f3405a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f340006bc00 con 0x7f3400071a60 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340015a130 0x7f3400156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340015a130 0x7f3400156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44752/0 (socket says 172.21.15.47:44752) 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 -- 172.21.15.47:0/1555749720 learned_addr learned my addr 172.21.15.47:0/1555749720 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f33ff400640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3400071a60 0x7f34001563c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f33ffe00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f340006f870 0x7f3400155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 -- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3400071a60 msgr2=0x7f34001563c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3400071a60 0x7f34001563c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 -- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f340006f870 msgr2=0x7f3400155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f340006f870 0x7f3400155af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f3405000640 1 -- 172.21.15.47:0/1555749720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34001f77e0 con 0x7f340015a130 2024-04-20T10:28:40.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.732+0000 7f33ff400640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3400071a60 0x7f34001563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.733+0000 7f3405000640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340015a130 0x7f3400156d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009d90 tx=0x7f33f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.733+0000 7f33f7e00640 1 -- 172.21.15.47:0/1555749720 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f8039050 con 0x7f340015a130 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.733+0000 7f33f7e00640 1 -- 172.21.15.47:0/1555749720 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33f8045070 con 0x7f340015a130 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.733+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3400156030 con 0x7f340015a130 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.733+0000 7f33f7e00640 1 -- 172.21.15.47:0/1555749720 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33f803e850 con 0x7f340015a130 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.733+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34001f7d60 con 0x7f340015a130 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.734+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3400075b30 con 0x7f340015a130 2024-04-20T10:28:40.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.735+0000 7f33f7e00640 1 -- 172.21.15.47:0/1555749720 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33f803e9f0 con 0x7f340015a130 2024-04-20T10:28:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.736+0000 7f33f7e00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f33e006c8f0 0x7f33e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:40.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.736+0000 7f33f7e00640 1 -- 172.21.15.47:0/1555749720 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f33f80aec10 con 0x7f340015a130 2024-04-20T10:28:40.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.737+0000 7f33ffe00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f33e006c8f0 0x7f33e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:40.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.738+0000 7f33ffe00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f33e006c8f0 0x7f33e006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006fd0 tx=0x7f33f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:40.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.741+0000 7f33f7e00640 1 -- 172.21.15.47:0/1555749720 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f33f807cda0 con 0x7f340015a130 2024-04-20T10:28:40.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.950+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3400073ba0 con 0x7f340015a130 2024-04-20T10:28:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.951+0000 7f33f7e00640 1 -- 172.21.15.47:0/1555749720 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f33f807c6a0 con 0x7f340015a130 2024-04-20T10:28:40.951 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f33e006c8f0 msgr2=0x7f33e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f33e006c8f0 0x7f33e006edb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f33f0006fd0 tx=0x7f33f0008040 comp rx=0 tx=0).stop 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340015a130 msgr2=0x7f3400156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340015a130 0x7f3400156d40 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f33f8009d90 tx=0x7f33f8004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 shutdown_connections 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f33e006c8f0 0x7f33e006edb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f340015a130 0x7f3400156d40 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3400071a60 0x7f34001563c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 --2- 172.21.15.47:0/1555749720 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f340006f870 0x7f3400155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 >> 172.21.15.47:0/1555749720 conn(0x7f34000cffa0 msgr2=0x7f34000cf600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 shutdown_connections 2024-04-20T10:28:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:40.954+0000 7f3405a00640 1 -- 172.21.15.47:0/1555749720 wait complete. 2024-04-20T10:28:40.973 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:41.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.180+0000 7ff2ce800640 1 Processor -- start 2024-04-20T10:28:41.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.181+0000 7ff2ce800640 1 -- start start 2024-04-20T10:28:41.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.182+0000 7ff2ce800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8560 0x7ff2c80cebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.182+0000 7ff2ce800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff2c80d0760 0x7ff2c80cf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.182+0000 7ff2ce800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c80d0da0 con 0x7ff2c80c8940 2024-04-20T10:28:41.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.182+0000 7ff2ce800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c80d19b0 con 0x7ff2c80d0760 2024-04-20T10:28:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.182+0000 7ff2ce800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c80c6d40 con 0x7ff2c80c8560 2024-04-20T10:28:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.182+0000 7ff2cca00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff2c80c8940 0x7ff2c80d0050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33272/0 (socket says 172.21.15.47:33272) 2024-04-20T10:28:41.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.182+0000 7ff2cca00640 1 -- 172.21.15.47:0/3799356137 learned_addr learned my addr 172.21.15.47:0/3799356137 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2cde00640 1 --2- 172.21.15.47:0/3799356137 >> v2:172.21.15.47:3300/0 conn(0x7ff2c80d0760 0x7ff2c80cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2cd400640 1 --2- 172.21.15.47:0/3799356137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8560 0x7ff2c80cebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2c7e00640 1 -- 172.21.15.47:0/3799356137 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3926816382 0 0) 0x7ff2c80d0da0 con 0x7ff2c80c8940 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2c7e00640 1 -- 172.21.15.47:0/3799356137 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2ac003520 con 0x7ff2c80c8940 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2cde00640 1 -- 172.21.15.47:0/3799356137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8560 msgr2=0x7ff2c80cebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2cde00640 1 --2- 172.21.15.47:0/3799356137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8560 0x7ff2c80cebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2cde00640 1 -- 172.21.15.47:0/3799356137 >> v1:172.21.15.47:6789/0 conn(0x7ff2c80c8940 legacy=0x7ff2c80d0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.183+0000 7ff2cde00640 1 -- 172.21.15.47:0/3799356137 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c80d1080 con 0x7ff2c80d0760 2024-04-20T10:28:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.184+0000 7ff2cde00640 1 --2- 172.21.15.47:0/3799356137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80d0760 0x7ff2c80cf130 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009d90 tx=0x7ff2c002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f22cafd2def7be7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.185+0000 7ff2c7e00640 1 -- 172.21.15.47:0/3799356137 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0039070 con 0x7ff2c80d0760 2024-04-20T10:28:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.185+0000 7ff2c7e00640 1 -- 172.21.15.47:0/3799356137 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2c002cc60 con 0x7ff2c80d0760 2024-04-20T10:28:41.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.185+0000 7ff2c7e00640 1 -- 172.21.15.47:0/3799356137 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0034660 con 0x7ff2c80d0760 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.185+0000 7ff2ce800640 1 -- 172.21.15.47:0/3799356137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80d0760 msgr2=0x7ff2c80cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.185+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3799356137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80d0760 0x7ff2c80cf130 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff2c0009d90 tx=0x7ff2c002beb0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.186+0000 7ff2ce800640 1 -- 172.21.15.47:0/3799356137 shutdown_connections 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.186+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3799356137 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80d0760 0x7ff2c80cf130 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.186+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3799356137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8560 0x7ff2c80cebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.186+0000 7ff2ce800640 1 -- 172.21.15.47:0/3799356137 >> 172.21.15.47:0/3799356137 conn(0x7ff2c80b3d10 msgr2=0x7ff2c80b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.186+0000 7ff2ce800640 1 -- 172.21.15.47:0/3799356137 shutdown_connections 2024-04-20T10:28:41.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.186+0000 7ff2ce800640 1 -- 172.21.15.47:0/3799356137 wait complete. 2024-04-20T10:28:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.187+0000 7ff2ce800640 1 Processor -- start 2024-04-20T10:28:41.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.188+0000 7ff2ce800640 1 -- start start 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.188+0000 7ff2ce800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80c8560 0x7ff2c814e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.188+0000 7ff2ce800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8940 0x7ff2c806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.188+0000 7ff2ce800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c80d0760 0x7ff2c806d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.188+0000 7ff2ce800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c80d1e40 con 0x7ff2c80c8560 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2ce800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c80d1640 con 0x7ff2c80d0760 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2ce800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c80c5d30 con 0x7ff2c80c8940 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80c8560 0x7ff2c814e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80c8560 0x7ff2c814e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44776/0 (socket says 172.21.15.47:44776) 2024-04-20T10:28:41.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 -- 172.21.15.47:0/3693464125 learned_addr learned my addr 172.21.15.47:0/3693464125 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cca00640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8940 0x7ff2c806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cde00640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c80d0760 0x7ff2c806d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 -- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8940 msgr2=0x7ff2c806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8940 0x7ff2c806d2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 -- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c80d0760 msgr2=0x7ff2c806d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c80d0760 0x7ff2c806d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.189+0000 7ff2cd400640 1 -- 172.21.15.47:0/3693464125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c806de10 con 0x7ff2c80c8560 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.190+0000 7ff2cd400640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80c8560 0x7ff2c814e120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800d5f0 tx=0x7ff2b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.190+0000 7ff2c5600640 1 -- 172.21.15.47:0/3693464125 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80125f0 con 0x7ff2c80c8560 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.190+0000 7ff2c5600640 1 -- 172.21.15.47:0/3693464125 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b8019070 con 0x7ff2c80c8560 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.190+0000 7ff2c5600640 1 -- 172.21.15.47:0/3693464125 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b80129d0 con 0x7ff2c80c8560 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.190+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c806e0f0 con 0x7ff2c80c8560 2024-04-20T10:28:41.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.190+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c806e6b0 con 0x7ff2c80c8560 2024-04-20T10:28:41.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.192+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c815fe40 con 0x7ff2c80c8560 2024-04-20T10:28:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.192+0000 7ff2c5600640 1 -- 172.21.15.47:0/3693464125 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2b8012b70 con 0x7ff2c80c8560 2024-04-20T10:28:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.193+0000 7ff2c5600640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a806c920 0x7ff2a806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.194+0000 7ff2c5600640 1 -- 172.21.15.47:0/3693464125 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff2b808c2c0 con 0x7ff2c80c8560 2024-04-20T10:28:41.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.194+0000 7ff2cca00640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a806c920 0x7ff2a806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.195+0000 7ff2cca00640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a806c920 0x7ff2a806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2c814f620 tx=0x7ff2b4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:41.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.198+0000 7ff2c5600640 1 -- 172.21.15.47:0/3693464125 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff2c815fe40 con 0x7ff2c80c8560 2024-04-20T10:28:41.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.401+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff2c8075b30 con 0x7ff2c80c8560 2024-04-20T10:28:41.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.402+0000 7ff2c5600640 1 -- 172.21.15.47:0/3693464125 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7ff2b805a330 con 0x7ff2c80c8560 2024-04-20T10:28:41.402 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a806c920 msgr2=0x7ff2a806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a806c920 0x7ff2a806ede0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff2c814f620 tx=0x7ff2b4009040 comp rx=0 tx=0).stop 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80c8560 msgr2=0x7ff2c814e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80c8560 0x7ff2c814e120 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff2b800d5f0 tx=0x7ff2b800dab0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 shutdown_connections 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c80d0760 0x7ff2c806d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c80c8940 0x7ff2c806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a806c920 0x7ff2a806ede0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 --2- 172.21.15.47:0/3693464125 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2c80c8560 0x7ff2c814e120 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.404+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 >> 172.21.15.47:0/3693464125 conn(0x7ff2c80b3d10 msgr2=0x7ff2c80c6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.405+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 shutdown_connections 2024-04-20T10:28:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.405+0000 7ff2ce800640 1 -- 172.21.15.47:0/3693464125 wait complete. 2024-04-20T10:28:41.424 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.635+0000 7f0b86000640 1 Processor -- start 2024-04-20T10:28:41.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.635+0000 7f0b86000640 1 -- start start 2024-04-20T10:28:41.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b86000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b80161d50 0x7f0b800d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b86000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0b8006cde0 0x7f0b800b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b86000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0b8006c190 con 0x7f0b800b3330 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b86000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f0b80162130 con 0x7f0b80161d50 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b86000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0b8006c470 con 0x7f0b8006cde0 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b84c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b80161d50 0x7f0b800d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b84c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0b80161d50 0x7f0b800d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44788/0 (socket says 172.21.15.47:44788) 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.637+0000 7f0b84c00640 1 -- 172.21.15.47:0/3351917815 learned_addr learned my addr 172.21.15.47:0/3351917815 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.638+0000 7f0b84c00640 1 -- 172.21.15.47:0/3351917815 >> v2:172.21.15.175:3300/0 conn(0x7f0b8006cde0 msgr2=0x7f0b800b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.638+0000 7f0b84c00640 1 --2- 172.21.15.47:0/3351917815 >> v2:172.21.15.175:3300/0 conn(0x7f0b8006cde0 0x7f0b800b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.638+0000 7f0b84c00640 1 -- 172.21.15.47:0/3351917815 >> v1:172.21.15.47:6789/0 conn(0x7f0b800b3330 legacy=0x7f0b8006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.638+0000 7f0b84c00640 1 -- 172.21.15.47:0/3351917815 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b800d2e00 con 0x7f0b80161d50 2024-04-20T10:28:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.639+0000 7f0b84c00640 1 --2- 172.21.15.47:0/3351917815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b80161d50 0x7f0b800d38e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0b800cee50 tx=0x7f0b7402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3548e235b2d3bf91 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.639+0000 7f0b7f400640 1 -- 172.21.15.47:0/3351917815 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74034070 con 0x7f0b80161d50 2024-04-20T10:28:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.639+0000 7f0b7f400640 1 -- 172.21.15.47:0/3351917815 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b74030860 con 0x7f0b80161d50 2024-04-20T10:28:41.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.639+0000 7f0b7f400640 1 -- 172.21.15.47:0/3351917815 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b74030c30 con 0x7f0b80161d50 2024-04-20T10:28:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 -- 172.21.15.47:0/3351917815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b80161d50 msgr2=0x7f0b800d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 --2- 172.21.15.47:0/3351917815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b80161d50 0x7f0b800d38e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0b800cee50 tx=0x7f0b7402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 -- 172.21.15.47:0/3351917815 shutdown_connections 2024-04-20T10:28:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 --2- 172.21.15.47:0/3351917815 >> v2:172.21.15.175:3300/0 conn(0x7f0b8006cde0 0x7f0b800b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 --2- 172.21.15.47:0/3351917815 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b80161d50 0x7f0b800d38e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 -- 172.21.15.47:0/3351917815 >> 172.21.15.47:0/3351917815 conn(0x7f0b800d4450 msgr2=0x7f0b800d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 -- 172.21.15.47:0/3351917815 shutdown_connections 2024-04-20T10:28:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.640+0000 7f0b86000640 1 -- 172.21.15.47:0/3351917815 wait complete. 2024-04-20T10:28:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.641+0000 7f0b86000640 1 Processor -- start 2024-04-20T10:28:41.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.642+0000 7f0b86000640 1 -- start start 2024-04-20T10:28:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b86000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b8006cde0 0x7f0b800b4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b86000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b800b3330 0x7f0b800b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b86000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b80161d50 0x7f0b800b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b86000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800b7660 con 0x7f0b800b3330 2024-04-20T10:28:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b86000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800d33c0 con 0x7f0b8006cde0 2024-04-20T10:28:41.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b86000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b800b77e0 con 0x7f0b80161d50 2024-04-20T10:28:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b800b3330 0x7f0b800b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b800b3330 0x7f0b800b4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44798/0 (socket says 172.21.15.47:44798) 2024-04-20T10:28:41.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 -- 172.21.15.47:0/660417401 learned_addr learned my addr 172.21.15.47:0/660417401 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b85600640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b80161d50 0x7f0b800b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b84c00640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b8006cde0 0x7f0b800b4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 -- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b80161d50 msgr2=0x7f0b800b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b80161d50 0x7f0b800b4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 -- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b8006cde0 msgr2=0x7f0b800b4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b8006cde0 0x7f0b800b4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.643+0000 7f0b7fe00640 1 -- 172.21.15.47:0/660417401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b800cace0 con 0x7f0b800b3330 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.644+0000 7f0b7fe00640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b800b3330 0x7f0b800b4aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c400 tx=0x7f0b6c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.644+0000 7f0b7cc00640 1 -- 172.21.15.47:0/660417401 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0052d0 con 0x7f0b800b3330 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.644+0000 7f0b7cc00640 1 -- 172.21.15.47:0/660417401 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b6c005490 con 0x7f0b800b3330 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.644+0000 7f0b7cc00640 1 -- 172.21.15.47:0/660417401 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c007260 con 0x7f0b800b3330 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.644+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b800cdb50 con 0x7f0b800b3330 2024-04-20T10:28:41.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.644+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b800ce110 con 0x7f0b800b3330 2024-04-20T10:28:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.645+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b80075b60 con 0x7f0b800b3330 2024-04-20T10:28:41.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.646+0000 7f0b7cc00640 1 -- 172.21.15.47:0/660417401 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0b6c007400 con 0x7f0b800b3330 2024-04-20T10:28:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.647+0000 7f0b7cc00640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b6006c920 0x7f0b6006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:41.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.648+0000 7f0b7cc00640 1 -- 172.21.15.47:0/660417401 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f0b6c08b4b0 con 0x7f0b800b3330 2024-04-20T10:28:41.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.648+0000 7f0b84c00640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b6006c920 0x7f0b6006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:41.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.650+0000 7f0b84c00640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b6006c920 0x7f0b6006ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b74036000 tx=0x7f0b7403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:41.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.652+0000 7f0b7cc00640 1 -- 172.21.15.47:0/660417401 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0b6c059490 con 0x7f0b800b3330 2024-04-20T10:28:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.868+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0b800d4af0 con 0x7f0b800b3330 2024-04-20T10:28:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.869+0000 7f0b7cc00640 1 -- 172.21.15.47:0/660417401 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f0b6c058d90 con 0x7f0b800b3330 2024-04-20T10:28:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b6006c920 msgr2=0x7f0b6006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b6006c920 0x7f0b6006ede0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0b74036000 tx=0x7f0b7403b040 comp rx=0 tx=0).stop 2024-04-20T10:28:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b800b3330 msgr2=0x7f0b800b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b800b3330 0x7f0b800b4aa0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c400 tx=0x7f0b6c00c8c0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 shutdown_connections 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b80161d50 0x7f0b800b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0b6006c920 0x7f0b6006ede0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0b800b3330 0x7f0b800b4aa0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 --2- 172.21.15.47:0/660417401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b8006cde0 0x7f0b800b4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 >> 172.21.15.47:0/660417401 conn(0x7f0b800d4450 msgr2=0x7f0b800b9a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 shutdown_connections 2024-04-20T10:28:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:41.872+0000 7f0b86000640 1 -- 172.21.15.47:0/660417401 wait complete. 2024-04-20T10:28:41.890 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-04-20T10:28:41.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:41.891 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T10:28:41.967 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:28:41.967 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:28:41.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:41.967 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T10:28:42.018 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:28:42.018 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:42.076 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:28:42.076 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:42.127 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.346+0000 7f7d8ca00640 1 Processor -- start 2024-04-20T10:28:42.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.347+0000 7f7d8ca00640 1 -- start start 2024-04-20T10:28:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.348+0000 7f7d8ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d88161d50 0x7f7d880d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.348+0000 7f7d8ca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7d880b3330 0x7f7d880b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.348+0000 7f7d8ca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d8806c190 con 0x7f7d8806c6d0 2024-04-20T10:28:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.348+0000 7f7d8ca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7d88162130 con 0x7f7d880b3330 2024-04-20T10:28:42.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.348+0000 7f7d8ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8806c470 con 0x7f7d88161d50 2024-04-20T10:28:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.349+0000 7f7d87400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d88161d50 0x7f7d880d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.349+0000 7f7d87400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d88161d50 0x7f7d880d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:54082/0 (socket says 172.21.15.47:54082) 2024-04-20T10:28:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.349+0000 7f7d87400640 1 -- 172.21.15.47:0/3792098934 learned_addr learned my addr 172.21.15.47:0/3792098934 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:42.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.349+0000 7f7d86a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7d880b3330 0x7f7d880b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.350+0000 7f7d87400640 1 -- 172.21.15.47:0/3792098934 >> v2:172.21.15.175:3300/0 conn(0x7f7d880b3330 msgr2=0x7f7d880b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.350+0000 7f7d87400640 1 --2- 172.21.15.47:0/3792098934 >> v2:172.21.15.175:3300/0 conn(0x7f7d880b3330 0x7f7d880b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.350+0000 7f7d87400640 1 -- 172.21.15.47:0/3792098934 >> v1:172.21.15.175:6789/0 conn(0x7f7d8806c6d0 legacy=0x7f7d880cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.350+0000 7f7d87400640 1 -- 172.21.15.47:0/3792098934 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d880d2e00 con 0x7f7d88161d50 2024-04-20T10:28:42.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.352+0000 7f7d87400640 1 --2- 172.21.15.47:0/3792098934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d88161d50 0x7f7d880d38e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d7802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7270a50e3907df53 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.352+0000 7f7d86000640 1 -- 172.21.15.47:0/3792098934 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78034070 con 0x7f7d88161d50 2024-04-20T10:28:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d86000640 1 -- 172.21.15.47:0/3792098934 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d78030840 con 0x7f7d88161d50 2024-04-20T10:28:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d86000640 1 -- 172.21.15.47:0/3792098934 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78030c10 con 0x7f7d88161d50 2024-04-20T10:28:42.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3792098934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d88161d50 msgr2=0x7f7d880d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/3792098934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d88161d50 0x7f7d880d38e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d78009870 tx=0x7f7d7802b980 comp rx=0 tx=0).stop 2024-04-20T10:28:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3792098934 shutdown_connections 2024-04-20T10:28:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/3792098934 >> v2:172.21.15.175:3300/0 conn(0x7f7d880b3330 0x7f7d880b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/3792098934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d88161d50 0x7f7d880d38e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.353+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3792098934 >> 172.21.15.47:0/3792098934 conn(0x7f7d880d4450 msgr2=0x7f7d880d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.354+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3792098934 shutdown_connections 2024-04-20T10:28:42.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.354+0000 7f7d8ca00640 1 -- 172.21.15.47:0/3792098934 wait complete. 2024-04-20T10:28:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.355+0000 7f7d8ca00640 1 Processor -- start 2024-04-20T10:28:42.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.355+0000 7f7d8ca00640 1 -- start start 2024-04-20T10:28:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.356+0000 7f7d8ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d8806c6d0 0x7f7d880aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.356+0000 7f7d8ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d880b3330 0x7f7d880ab330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.356+0000 7f7d8ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d88161d50 0x7f7d881f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.356+0000 7f7d8ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8806cb60 con 0x7f7d8806c6d0 2024-04-20T10:28:42.357 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.356+0000 7f7d8ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d880d33c0 con 0x7f7d88161d50 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.356+0000 7f7d8ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8806cce0 con 0x7f7d880b3330 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d8806c6d0 0x7f7d880aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d8806c6d0 0x7f7d880aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44810/0 (socket says 172.21.15.47:44810) 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 -- 172.21.15.47:0/2296311268 learned_addr learned my addr 172.21.15.47:0/2296311268 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d86a00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d880b3330 0x7f7d880ab330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 -- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d880b3330 msgr2=0x7f7d880ab330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d880b3330 0x7f7d880ab330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 -- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d88161d50 msgr2=0x7f7d881f7d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d88161d50 0x7f7d881f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.357+0000 7f7d87400640 1 -- 172.21.15.47:0/2296311268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d880cbbf0 con 0x7f7d8806c6d0 2024-04-20T10:28:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.358+0000 7f7d87400640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d8806c6d0 0x7f7d880aadf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7d7802b9c0 tx=0x7f7d780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.358+0000 7f7d7f400640 1 -- 172.21.15.47:0/2296311268 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78034040 con 0x7f7d8806c6d0 2024-04-20T10:28:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.358+0000 7f7d7f400640 1 -- 172.21.15.47:0/2296311268 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d78039d90 con 0x7f7d8806c6d0 2024-04-20T10:28:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.358+0000 7f7d7f400640 1 -- 172.21.15.47:0/2296311268 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d78043630 con 0x7f7d8806c6d0 2024-04-20T10:28:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.358+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d880cbe90 con 0x7f7d8806c6d0 2024-04-20T10:28:42.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.358+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d880cc450 con 0x7f7d8806c6d0 2024-04-20T10:28:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.360+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d88075b60 con 0x7f7d8806c6d0 2024-04-20T10:28:42.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.360+0000 7f7d7f400640 1 -- 172.21.15.47:0/2296311268 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d7804a060 con 0x7f7d8806c6d0 2024-04-20T10:28:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.361+0000 7f7d7f400640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d6406c8f0 0x7f7d6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.362+0000 7f7d7f400640 1 -- 172.21.15.47:0/2296311268 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f7d780adf20 con 0x7f7d8806c6d0 2024-04-20T10:28:42.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.362+0000 7f7d86a00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d6406c8f0 0x7f7d6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.363+0000 7f7d86a00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d6406c8f0 0x7f7d6406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d881f7cc0 tx=0x7f7d70008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:42.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.366+0000 7f7d7f400640 1 -- 172.21.15.47:0/2296311268 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7d7803e070 con 0x7f7d8806c6d0 2024-04-20T10:28:42.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.574+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d880b9de0 con 0x7f7d8806c6d0 2024-04-20T10:28:42.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.575+0000 7f7d7f400640 1 -- 172.21.15.47:0/2296311268 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f7d7802d330 con 0x7f7d8806c6d0 2024-04-20T10:28:42.576 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:42.576 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.578+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d6406c8f0 msgr2=0x7f7d6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.578+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d6406c8f0 0x7f7d6406edb0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f7d881f7cc0 tx=0x7f7d70008040 comp rx=0 tx=0).stop 2024-04-20T10:28:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.578+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d8806c6d0 msgr2=0x7f7d880aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.578+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d8806c6d0 0x7f7d880aadf0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f7d7802b9c0 tx=0x7f7d780046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 shutdown_connections 2024-04-20T10:28:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d88161d50 0x7f7d881f7d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d880b3330 0x7f7d880ab330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d6406c8f0 0x7f7d6406edb0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 --2- 172.21.15.47:0/2296311268 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d8806c6d0 0x7f7d880aadf0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 >> 172.21.15.47:0/2296311268 conn(0x7f7d880d4450 msgr2=0x7f7d8815ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 shutdown_connections 2024-04-20T10:28:42.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.579+0000 7f7d8ca00640 1 -- 172.21.15.47:0/2296311268 wait complete. 2024-04-20T10:28:42.600 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:42.600 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:42.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.811+0000 7f7afdc00640 1 Processor -- start 2024-04-20T10:28:42.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.811+0000 7f7afdc00640 1 -- start start 2024-04-20T10:28:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7afdc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7af80b4610 0x7f7af815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7afdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af8072a20 0x7f7af8072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7afdc00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7af8160690 con 0x7f7af80b49f0 2024-04-20T10:28:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7afdc00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7af8160970 con 0x7f7af80b4610 2024-04-20T10:28:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7afdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7af8160c50 con 0x7f7af8072a20 2024-04-20T10:28:42.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7af7e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7af80b4610 0x7f7af815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7af7400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7af80b49f0 0x7f7af815d080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33290/0 (socket says 172.21.15.47:33290) 2024-04-20T10:28:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7af7400640 1 -- 172.21.15.47:0/2550540678 learned_addr learned my addr 172.21.15.47:0/2550540678 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.813+0000 7f7afd200640 1 --2- 172.21.15.47:0/2550540678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af8072a20 0x7f7af8072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.814+0000 7f7af7e00640 1 -- 172.21.15.47:0/2550540678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af8072a20 msgr2=0x7f7af8072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.814+0000 7f7af7e00640 1 --2- 172.21.15.47:0/2550540678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af8072a20 0x7f7af8072eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.814+0000 7f7af7e00640 1 -- 172.21.15.47:0/2550540678 >> v1:172.21.15.47:6789/0 conn(0x7f7af80b49f0 legacy=0x7f7af815d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.814+0000 7f7af7e00640 1 -- 172.21.15.47:0/2550540678 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af815cb60 con 0x7f7af80b4610 2024-04-20T10:28:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.814+0000 7f7af6a00640 1 -- 172.21.15.47:0/2550540678 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3186097268 0 0) 0x7f7af8160690 con 0x7f7af80b49f0 2024-04-20T10:28:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.815+0000 7f7af7e00640 1 --2- 172.21.15.47:0/2550540678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af80b4610 0x7f7af815fe40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097a0 tx=0x7f7ae802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b30a8a7e2606ec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.815+0000 7f7af6a00640 1 -- 172.21.15.47:0/2550540678 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8035020 con 0x7f7af80b4610 2024-04-20T10:28:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.815+0000 7f7af6a00640 1 -- 172.21.15.47:0/2550540678 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae8031830 con 0x7f7af80b4610 2024-04-20T10:28:42.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.815+0000 7f7af6a00640 1 -- 172.21.15.47:0/2550540678 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8031c00 con 0x7f7af80b4610 2024-04-20T10:28:42.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 -- 172.21.15.47:0/2550540678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af80b4610 msgr2=0x7f7af815fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 --2- 172.21.15.47:0/2550540678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af80b4610 0x7f7af815fe40 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7ae80097a0 tx=0x7f7ae802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 -- 172.21.15.47:0/2550540678 shutdown_connections 2024-04-20T10:28:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 --2- 172.21.15.47:0/2550540678 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af8072a20 0x7f7af8072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 --2- 172.21.15.47:0/2550540678 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af80b4610 0x7f7af815fe40 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 -- 172.21.15.47:0/2550540678 >> 172.21.15.47:0/2550540678 conn(0x7f7af80cdbc0 msgr2=0x7f7af80cdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 -- 172.21.15.47:0/2550540678 shutdown_connections 2024-04-20T10:28:42.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.816+0000 7f7afdc00640 1 -- 172.21.15.47:0/2550540678 wait complete. 2024-04-20T10:28:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.818+0000 7f7afdc00640 1 Processor -- start 2024-04-20T10:28:42.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.818+0000 7f7afdc00640 1 -- start start 2024-04-20T10:28:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.819+0000 7f7afdc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af8072a20 0x7f7af8148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.819+0000 7f7afdc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7af80b4610 0x7f7af80adec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.819+0000 7f7afdc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af80b49f0 0x7f7af8151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.819+0000 7f7afdc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7af8160380 con 0x7f7af8072a20 2024-04-20T10:28:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.819+0000 7f7afdc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7af815c850 con 0x7f7af80b4610 2024-04-20T10:28:42.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.819+0000 7f7afdc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7af80b4ea0 con 0x7f7af80b49f0 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af8072a20 0x7f7af8148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af8072a20 0x7f7af8148d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44836/0 (socket says 172.21.15.47:44836) 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 -- 172.21.15.47:0/3149670741 learned_addr learned my addr 172.21.15.47:0/3149670741 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7afd200640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af80b49f0 0x7f7af8151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7400640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7af80b4610 0x7f7af80adec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 -- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af80b49f0 msgr2=0x7f7af8151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af80b49f0 0x7f7af8151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 -- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7af80b4610 msgr2=0x7f7af80adec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7af80b4610 0x7f7af80adec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7af7e00640 1 -- 172.21.15.47:0/3149670741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af80c63f0 con 0x7f7af8072a20 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.820+0000 7f7afd200640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af80b49f0 0x7f7af8151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.821+0000 7f7af7e00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af8072a20 0x7f7af8148d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7ae802bee0 tx=0x7f7ae80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.821+0000 7f7aefe00640 1 -- 172.21.15.47:0/3149670741 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8041070 con 0x7f7af8072a20 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.821+0000 7f7aefe00640 1 -- 172.21.15.47:0/3149670741 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7ae803b920 con 0x7f7af8072a20 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.821+0000 7f7aefe00640 1 -- 172.21.15.47:0/3149670741 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8035040 con 0x7f7af8072a20 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.821+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af80ae810 con 0x7f7af8072a20 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.821+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af80aedd0 con 0x7f7af8072a20 2024-04-20T10:28:42.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.822+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7af8075b30 con 0x7f7af8072a20 2024-04-20T10:28:42.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.823+0000 7f7aefe00640 1 -- 172.21.15.47:0/3149670741 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7ae804c060 con 0x7f7af8072a20 2024-04-20T10:28:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.824+0000 7f7aefe00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7ad806c8f0 0x7f7ad806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:42.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.824+0000 7f7aefe00640 1 -- 172.21.15.47:0/3149670741 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f7ae80ad9c0 con 0x7f7af8072a20 2024-04-20T10:28:42.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.825+0000 7f7af7400640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7ad806c8f0 0x7f7ad806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:42.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.826+0000 7f7af7400640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7ad806c8f0 0x7f7ad806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7af8150fe0 tx=0x7f7ae4007b50 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:42.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:42.829+0000 7f7aefe00640 1 -- 172.21.15.47:0/3149670741 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7ae807b920 con 0x7f7af8072a20 2024-04-20T10:28:43.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.037+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7af80b5280 con 0x7f7af8072a20 2024-04-20T10:28:43.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.038+0000 7f7aefe00640 1 -- 172.21.15.47:0/3149670741 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f7ae807b220 con 0x7f7af8072a20 2024-04-20T10:28:43.038 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7ad806c8f0 msgr2=0x7f7ad806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7ad806c8f0 0x7f7ad806edb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7af8150fe0 tx=0x7f7ae4007b50 comp rx=0 tx=0).stop 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af8072a20 msgr2=0x7f7af8148d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af8072a20 0x7f7af8148d50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7ae802bee0 tx=0x7f7ae80046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 shutdown_connections 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7af80b49f0 0x7f7af8151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7af80b4610 0x7f7af80adec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7ad806c8f0 0x7f7ad806edb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 --2- 172.21.15.47:0/3149670741 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7af8072a20 0x7f7af8148d50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.040+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 >> 172.21.15.47:0/3149670741 conn(0x7f7af80cdbc0 msgr2=0x7f7af806c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.041+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 shutdown_connections 2024-04-20T10:28:43.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.041+0000 7f7afdc00640 1 -- 172.21.15.47:0/3149670741 wait complete. 2024-04-20T10:28:43.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:43.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.276+0000 7f4209c00640 1 Processor -- start 2024-04-20T10:28:43.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.276+0000 7f4209c00640 1 -- start start 2024-04-20T10:28:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.278+0000 7f4209c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4204161a50 0x7f42040b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.278+0000 7f4209c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420406c010 0x7f42040ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.278+0000 7f4209c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f42040d0980 con 0x7f42040d3b20 2024-04-20T10:28:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.278+0000 7f4209c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f42040d0c60 con 0x7f4204161a50 2024-04-20T10:28:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.278+0000 7f4209c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4204162ae0 con 0x7f420406c010 2024-04-20T10:28:43.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.278+0000 7f4203400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f42040d3b20 0x7f420406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33294/0 (socket says 172.21.15.47:33294) 2024-04-20T10:28:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.278+0000 7f4203400640 1 -- 172.21.15.47:0/2785864374 learned_addr learned my addr 172.21.15.47:0/2785864374 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4209200640 1 --2- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420406c010 0x7f42040ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4203e00640 1 --2- 172.21.15.47:0/2785864374 >> v2:172.21.15.175:3300/0 conn(0x7f4204161a50 0x7f42040b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4202a00640 1 -- 172.21.15.47:0/2785864374 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1671240542 0 0) 0x7f42040d0980 con 0x7f42040d3b20 2024-04-20T10:28:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4202a00640 1 -- 172.21.15.47:0/2785864374 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f41e8003520 con 0x7f42040d3b20 2024-04-20T10:28:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4203e00640 1 -- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420406c010 msgr2=0x7f42040ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4203e00640 1 --2- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420406c010 0x7f42040ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4203e00640 1 -- 172.21.15.47:0/2785864374 >> v1:172.21.15.47:6789/0 conn(0x7f42040d3b20 legacy=0x7f420406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.279+0000 7f4203e00640 1 -- 172.21.15.47:0/2785864374 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4204162490 con 0x7f4204161a50 2024-04-20T10:28:43.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.281+0000 7f4203e00640 1 --2- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4204161a50 0x7f42040b3180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41f40097a0 tx=0x7f41f402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9506fba826ce1af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:43.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.281+0000 7f4202a00640 1 -- 172.21.15.47:0/2785864374 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4035020 con 0x7f4204161a50 2024-04-20T10:28:43.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.281+0000 7f4202a00640 1 -- 172.21.15.47:0/2785864374 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f4031830 con 0x7f4204161a50 2024-04-20T10:28:43.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.281+0000 7f4202a00640 1 -- 172.21.15.47:0/2785864374 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f4031c00 con 0x7f4204161a50 2024-04-20T10:28:43.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 -- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4204161a50 msgr2=0x7f42040b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 --2- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4204161a50 0x7f42040b3180 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f41f40097a0 tx=0x7f41f402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 -- 172.21.15.47:0/2785864374 shutdown_connections 2024-04-20T10:28:43.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 --2- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420406c010 0x7f42040ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 --2- 172.21.15.47:0/2785864374 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4204161a50 0x7f42040b3180 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 -- 172.21.15.47:0/2785864374 >> 172.21.15.47:0/2785864374 conn(0x7f42040d0ed0 msgr2=0x7f42040d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 -- 172.21.15.47:0/2785864374 shutdown_connections 2024-04-20T10:28:43.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.282+0000 7f4209c00640 1 -- 172.21.15.47:0/2785864374 wait complete. 2024-04-20T10:28:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.284+0000 7f4209c00640 1 Processor -- start 2024-04-20T10:28:43.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.284+0000 7f4209c00640 1 -- start start 2024-04-20T10:28:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.285+0000 7f4209c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420406c010 0x7f420414c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.285+0000 7f4209c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4204161a50 0x7f42040c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.285+0000 7f4209c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420415ba10 0x7f420415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.285+0000 7f4209c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f42040d0600 con 0x7f4204161a50 2024-04-20T10:28:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.285+0000 7f4209c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42040d0480 con 0x7f420406c010 2024-04-20T10:28:43.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.285+0000 7f4209c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4204161e30 con 0x7f420415ba10 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4204161a50 0x7f42040c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4204161a50 0x7f42040c8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44838/0 (socket says 172.21.15.47:44838) 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 -- 172.21.15.47:0/3524275326 learned_addr learned my addr 172.21.15.47:0/3524275326 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4209200640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420415ba10 0x7f420415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203e00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420406c010 0x7f420414c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 -- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420415ba10 msgr2=0x7f420415bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420415ba10 0x7f420415bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 -- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420406c010 msgr2=0x7f420414c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420406c010 0x7f420414c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.286+0000 7f4203400640 1 -- 172.21.15.47:0/3524275326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42040c8070 con 0x7f4204161a50 2024-04-20T10:28:43.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.287+0000 7f4203400640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4204161a50 0x7f42040c8e30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f41f000c430 tx=0x7f41f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.287+0000 7f41fbe00640 1 -- 172.21.15.47:0/3524275326 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0017070 con 0x7f4204161a50 2024-04-20T10:28:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.287+0000 7f41fbe00640 1 -- 172.21.15.47:0/3524275326 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41f0005d00 con 0x7f4204161a50 2024-04-20T10:28:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.287+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42040c8350 con 0x7f4204161a50 2024-04-20T10:28:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.287+0000 7f41fbe00640 1 -- 172.21.15.47:0/3524275326 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f0007310 con 0x7f4204161a50 2024-04-20T10:28:43.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.287+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42040c9370 con 0x7f4204161a50 2024-04-20T10:28:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.289+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4204075b30 con 0x7f4204161a50 2024-04-20T10:28:43.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.290+0000 7f41fbe00640 1 -- 172.21.15.47:0/3524275326 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41f001a070 con 0x7f4204161a50 2024-04-20T10:28:43.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.290+0000 7f41fbe00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41e406c8f0 0x7f41e406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.291+0000 7f41fbe00640 1 -- 172.21.15.47:0/3524275326 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f41f008b300 con 0x7f4204161a50 2024-04-20T10:28:43.291 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.291+0000 7f4203e00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41e406c8f0 0x7f41e406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.292 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.293+0000 7f4203e00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41e406c8f0 0x7f41e406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41f4036000 tx=0x7f41f403c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:43.295 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.295+0000 7f41fbe00640 1 -- 172.21.15.47:0/3524275326 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41f0058d20 con 0x7f4204161a50 2024-04-20T10:28:43.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.498+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f420406d0a0 con 0x7f4204161a50 2024-04-20T10:28:43.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.498+0000 7f41fbe00640 1 -- 172.21.15.47:0/3524275326 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f41f005f070 con 0x7f4204161a50 2024-04-20T10:28:43.499 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:43.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41e406c8f0 msgr2=0x7f41e406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41e406c8f0 0x7f41e406edb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f41f4036000 tx=0x7f41f403c040 comp rx=0 tx=0).stop 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4204161a50 msgr2=0x7f42040c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4204161a50 0x7f42040c8e30 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f41f000c430 tx=0x7f41f000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 shutdown_connections 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f420415ba10 0x7f420415bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41e406c8f0 0x7f41e406edb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4204161a50 0x7f42040c8e30 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 --2- 172.21.15.47:0/3524275326 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f420406c010 0x7f420414c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 >> 172.21.15.47:0/3524275326 conn(0x7f42040d0ed0 msgr2=0x7f420415f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 shutdown_connections 2024-04-20T10:28:43.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.501+0000 7f4209c00640 1 -- 172.21.15.47:0/3524275326 wait complete. 2024-04-20T10:28:43.519 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:43.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.708+0000 7fa0fe800640 1 Processor -- start 2024-04-20T10:28:43.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.708+0000 7fa0fe800640 1 -- start start 2024-04-20T10:28:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fe800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0f8154fe0 0x7fa0f813e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fe800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f815e5d0 con 0x7fa0f80adef0 2024-04-20T10:28:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fe800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa0f815e870 con 0x7fa0f80cad70 2024-04-20T10:28:43.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fe800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa0f815eb50 con 0x7fa0f8154fe0 2024-04-20T10:28:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0f8154fe0 0x7fa0f813e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa0f8154fe0 0x7fa0f813e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44842/0 (socket says 172.21.15.47:44842) 2024-04-20T10:28:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fd400640 1 -- 172.21.15.47:0/3869673708 learned_addr learned my addr 172.21.15.47:0/3869673708 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.710+0000 7fa0fde00640 1 --1- 172.21.15.47:0/3869673708 >> v1:172.21.15.47:6789/0 conn(0x7fa0f80adef0 0x7fa0f815b290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:33298/0 (socket says 172.21.15.47:33298) 2024-04-20T10:28:43.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.711+0000 7fa0fd400640 1 -- 172.21.15.47:0/3869673708 >> v1:172.21.15.175:6789/0 conn(0x7fa0f80cad70 legacy=0x7fa0f80ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.711+0000 7fa0fd400640 1 -- 172.21.15.47:0/3869673708 >> v1:172.21.15.47:6789/0 conn(0x7fa0f80adef0 legacy=0x7fa0f815b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.711+0000 7fa0fd400640 1 -- 172.21.15.47:0/3869673708 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f815ee30 con 0x7fa0f8154fe0 2024-04-20T10:28:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.712+0000 7fa0fd400640 1 --2- 172.21.15.47:0/3869673708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f8154fe0 0x7fa0f813e1b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380903a94f7b8846 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.712+0000 7fa0f7e00640 1 -- 172.21.15.47:0/3869673708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0034070 con 0x7fa0f8154fe0 2024-04-20T10:28:43.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.712+0000 7fa0f7e00640 1 -- 172.21.15.47:0/3869673708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f0030860 con 0x7fa0f8154fe0 2024-04-20T10:28:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.712+0000 7fa0f7e00640 1 -- 172.21.15.47:0/3869673708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0030c30 con 0x7fa0f8154fe0 2024-04-20T10:28:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.713+0000 7fa0fe800640 1 -- 172.21.15.47:0/3869673708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f8154fe0 msgr2=0x7fa0f813e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.713+0000 7fa0fe800640 1 --2- 172.21.15.47:0/3869673708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f8154fe0 0x7fa0f813e1b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f002b9a0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.713+0000 7fa0fe800640 1 -- 172.21.15.47:0/3869673708 shutdown_connections 2024-04-20T10:28:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.713+0000 7fa0fe800640 1 --2- 172.21.15.47:0/3869673708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f8154fe0 0x7fa0f813e1b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.713+0000 7fa0fe800640 1 -- 172.21.15.47:0/3869673708 >> 172.21.15.47:0/3869673708 conn(0x7fa0f80cffa0 msgr2=0x7fa0f80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.713+0000 7fa0fe800640 1 -- 172.21.15.47:0/3869673708 shutdown_connections 2024-04-20T10:28:43.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.713+0000 7fa0fe800640 1 -- 172.21.15.47:0/3869673708 wait complete. 2024-04-20T10:28:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.715+0000 7fa0fe800640 1 Processor -- start 2024-04-20T10:28:43.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.715+0000 7fa0fe800640 1 -- start start 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80adef0 0x7fa0f8153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f80cad70 0x7fa0f806b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8154fe0 0x7fa0f806be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fe800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f81553c0 con 0x7fa0f80adef0 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fe800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa0f815f3f0 con 0x7fa0f8154fe0 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fe800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa0f80cb220 con 0x7fa0f80cad70 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80adef0 0x7fa0f8153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fd400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80adef0 0x7fa0f8153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:44856/0 (socket says 172.21.15.47:44856) 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.716+0000 7fa0fd400640 1 -- 172.21.15.47:0/1885804936 learned_addr learned my addr 172.21.15.47:0/1885804936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fd400640 1 -- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f80cad70 msgr2=0x7fa0f806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fd400640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f80cad70 0x7fa0f806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fd400640 1 -- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8154fe0 msgr2=0x7fa0f806be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fd400640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8154fe0 0x7fa0f806be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fd400640 1 -- 172.21.15.47:0/1885804936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0f8154380 con 0x7fa0f80adef0 2024-04-20T10:28:43.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fd400640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80adef0 0x7fa0f8153e40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0f5600640 1 -- 172.21.15.47:0/1885804936 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0034040 con 0x7fa0f80adef0 2024-04-20T10:28:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0f5600640 1 -- 172.21.15.47:0/1885804936 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa0f00433f0 con 0x7fa0f80adef0 2024-04-20T10:28:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0f5600640 1 -- 172.21.15.47:0/1885804936 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00437e0 con 0x7fa0f80adef0 2024-04-20T10:28:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0f813d350 con 0x7fa0f80adef0 2024-04-20T10:28:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.717+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0f813d910 con 0x7fa0f80adef0 2024-04-20T10:28:43.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.719+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0f8075b30 con 0x7fa0f80adef0 2024-04-20T10:28:43.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.719+0000 7fa0f5600640 1 -- 172.21.15.47:0/1885804936 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa0f004a060 con 0x7fa0f80adef0 2024-04-20T10:28:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.721+0000 7fa0f5600640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0d806c920 0x7fa0d806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.721+0000 7fa0f5600640 1 -- 172.21.15.47:0/1885804936 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fa0f00ae1d0 con 0x7fa0f80adef0 2024-04-20T10:28:43.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.722+0000 7fa0fca00640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0d806c920 0x7fa0d806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:43.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.723+0000 7fa0fca00640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0d806c920 0x7fa0d806ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80c9c20 tx=0x7fa0e4007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:43.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.725+0000 7fa0f5600640 1 -- 172.21.15.47:0/1885804936 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa0f003e070 con 0x7fa0f80adef0 2024-04-20T10:28:43.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.928+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa0f80ccf70 con 0x7fa0f80adef0 2024-04-20T10:28:43.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.929+0000 7fa0f5600640 1 -- 172.21.15.47:0/1885804936 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fa0f007c240 con 0x7fa0f80adef0 2024-04-20T10:28:43.929 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:43.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0d806c920 msgr2=0x7fa0d806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0d806c920 0x7fa0d806ede0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa0f80c9c20 tx=0x7fa0e4007d30 comp rx=0 tx=0).stop 2024-04-20T10:28:43.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80adef0 msgr2=0x7fa0f8153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:43.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80adef0 0x7fa0f8153e40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fa0f0009870 tx=0x7fa0f00046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 shutdown_connections 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa0f8154fe0 0x7fa0f806be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa0f80cad70 0x7fa0f806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa0d806c920 0x7fa0d806ede0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 --2- 172.21.15.47:0/1885804936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa0f80adef0 0x7fa0f8153e40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 >> 172.21.15.47:0/1885804936 conn(0x7fa0f80cffa0 msgr2=0x7fa0f80ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 shutdown_connections 2024-04-20T10:28:43.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:43.932+0000 7fa0fe800640 1 -- 172.21.15.47:0/1885804936 wait complete. 2024-04-20T10:28:43.952 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.296+0000 7f4da5000640 1 Processor -- start 2024-04-20T10:28:44.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.296+0000 7f4da5000640 1 -- start start 2024-04-20T10:28:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.297+0000 7f4da5000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4da0072cc0 0x7f4da00ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.297+0000 7f4da5000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da006e840 con 0x7f4da0071b50 2024-04-20T10:28:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.297+0000 7f4da5000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4da015b350 con 0x7f4da015abb0 2024-04-20T10:28:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.297+0000 7f4da5000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4da006eb00 con 0x7f4da0072cc0 2024-04-20T10:28:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.298+0000 7f4d9e800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4da0071b50 0x7f4da00adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44538/0 (socket says 172.21.15.47:44538) 2024-04-20T10:28:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.298+0000 7f4d9e800640 1 -- 172.21.15.47:0/2611745895 learned_addr learned my addr 172.21.15.47:0/2611745895 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:44.298 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.298+0000 7f4d9d400640 1 --2- 172.21.15.47:0/2611745895 >> v2:172.21.15.175:3300/0 conn(0x7f4da0072cc0 0x7f4da00ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.298+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1119524673 0 0) 0x7f4da006e840 con 0x7f4da0071b50 2024-04-20T10:28:44.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.298+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4d80003520 con 0x7f4da0071b50 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1344953033 0 0) 0x7f4d80003520 con 0x7f4da0071b50 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 >> v2:172.21.15.175:3300/0 conn(0x7f4da0072cc0 msgr2=0x7f4da00ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 --2- 172.21.15.47:0/2611745895 >> v2:172.21.15.175:3300/0 conn(0x7f4da0072cc0 0x7f4da00ad8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 >> v1:172.21.15.175:6789/0 conn(0x7f4da015abb0 legacy=0x7f4da00725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da015b630 con 0x7f4da0071b50 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f4d98001670 con 0x7f4da0071b50 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 >> v1:172.21.15.47:6789/0 conn(0x7f4da0071b50 legacy=0x7f4da00adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d80003a00 0x7f4d80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d80004800 0x7f4d80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d800051c0 0x7f4d8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8000c090 con 0x7f4d80004800 2024-04-20T10:28:44.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8000c240 con 0x7f4d800051c0 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d8000c440 con 0x7f4d80003a00 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.299+0000 7f4d9d400640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d80004800 0x7f4d80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9de00640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d80003a00 0x7f4d80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9e800640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d800051c0 0x7f4d8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9d400640 1 -- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d80003a00 msgr2=0x7f4d80003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9d400640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d80003a00 0x7f4d80003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9d400640 1 -- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d800051c0 msgr2=0x7f4d8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9d400640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d800051c0 0x7f4d8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9d400640 1 -- 172.21.15.47:0/2611745895 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d80003520 con 0x7f4d80004800 2024-04-20T10:28:44.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.300+0000 7f4d9e800640 1 -- 172.21.15.47:0/2611745895 reap_dead start 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.301+0000 7f4d9d400640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d80004800 0x7f4d80004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009fc0 tx=0x7f4d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49eadc0b0c7e8e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.301+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88039070 con 0x7f4d80004800 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.301+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8802cc20 con 0x7f4d80004800 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.301+0000 7f4d9ca00640 1 -- 172.21.15.47:0/2611745895 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88009c10 con 0x7f4d80004800 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.302+0000 7f4da5000640 1 -- 172.21.15.47:0/2611745895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d80004800 msgr2=0x7f4d80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.302+0000 7f4da5000640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d80004800 0x7f4d80004c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4d88009fc0 tx=0x7f4d80005830 comp rx=0 tx=0).stop 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.302+0000 7f4da5000640 1 -- 172.21.15.47:0/2611745895 shutdown_connections 2024-04-20T10:28:44.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.302+0000 7f4da5000640 1 --2- 172.21.15.47:0/2611745895 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d80004800 0x7f4d80004c80 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.302+0000 7f4da5000640 1 -- 172.21.15.47:0/2611745895 >> 172.21.15.47:0/2611745895 conn(0x7f4da00b3350 msgr2=0x7f4da00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.302+0000 7f4da5000640 1 -- 172.21.15.47:0/2611745895 shutdown_connections 2024-04-20T10:28:44.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.302+0000 7f4da5000640 1 -- 172.21.15.47:0/2611745895 wait complete. 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.304+0000 7f4da5000640 1 Processor -- start 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.304+0000 7f4da5000640 1 -- start start 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.305+0000 7f4da5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d80004800 0x7f4da0154600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.305+0000 7f4da5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da009ded0 0x7f4da0157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.305+0000 7f4da5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4da01592e0 0x7f4da00b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.305+0000 7f4da5000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4da015bbf0 con 0x7f4da009ded0 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.305+0000 7f4da5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4da015bf60 con 0x7f4d80004800 2024-04-20T10:28:44.307 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.305+0000 7f4da5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4da015af90 con 0x7f4da01592e0 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da009ded0 0x7f4da0157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da009ded0 0x7f4da0157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34972/0 (socket says 172.21.15.47:34972) 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 -- 172.21.15.47:0/464317017 learned_addr learned my addr 172.21.15.47:0/464317017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9de00640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d80004800 0x7f4da0154600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9e800640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4da01592e0 0x7f4da00b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 -- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4da01592e0 msgr2=0x7f4da00b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4da01592e0 0x7f4da00b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 -- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d80004800 msgr2=0x7f4da0154600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d80004800 0x7f4da0154600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.306+0000 7f4d9d400640 1 -- 172.21.15.47:0/464317017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da0157fe0 con 0x7f4da009ded0 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.307+0000 7f4d9d400640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da009ded0 0x7f4da0157720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e200 tx=0x7f4d88009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:44.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.307+0000 7f4d96000640 1 -- 172.21.15.47:0/464317017 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88039050 con 0x7f4da009ded0 2024-04-20T10:28:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.307+0000 7f4d96000640 1 -- 172.21.15.47:0/464317017 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d8803a040 con 0x7f4da009ded0 2024-04-20T10:28:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.307+0000 7f4d96000640 1 -- 172.21.15.47:0/464317017 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d88009c10 con 0x7f4da009ded0 2024-04-20T10:28:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.307+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da01582c0 con 0x7f4da009ded0 2024-04-20T10:28:44.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.307+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da0158ae0 con 0x7f4da009ded0 2024-04-20T10:28:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.308+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da0068980 con 0x7f4da009ded0 2024-04-20T10:28:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.309+0000 7f4d96000640 1 -- 172.21.15.47:0/464317017 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d88003bb0 con 0x7f4da009ded0 2024-04-20T10:28:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.310+0000 7f4d96000640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d7c06c8c0 0x7f4d7c06ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.310+0000 7f4d96000640 1 -- 172.21.15.47:0/464317017 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f4d880ae900 con 0x7f4da009ded0 2024-04-20T10:28:44.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.310+0000 7f4d9de00640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d7c06c8c0 0x7f4d7c06ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.312+0000 7f4d9de00640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d7c06c8c0 0x7f4d7c06ed80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4da006a810 tx=0x7f4d90009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:44.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.314+0000 7f4d96000640 1 -- 172.21.15.47:0/464317017 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d8807c8e0 con 0x7f4da009ded0 2024-04-20T10:28:44.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.518+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f4da01553a0 con 0x7f4da009ded0 2024-04-20T10:28:44.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.520+0000 7f4d96000640 1 -- 172.21.15.47:0/464317017 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f4d8807c1e0 con 0x7f4da009ded0 2024-04-20T10:28:44.521 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d7c06c8c0 msgr2=0x7f4d7c06ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d7c06c8c0 0x7f4d7c06ed80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4da006a810 tx=0x7f4d90009040 comp rx=0 tx=0).stop 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da009ded0 msgr2=0x7f4da0157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da009ded0 0x7f4da0157720 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4d8800e200 tx=0x7f4d88009750 comp rx=0 tx=0).stop 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 shutdown_connections 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4da01592e0 0x7f4da00b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d7c06c8c0 0x7f4d7c06ed80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4da009ded0 0x7f4da0157720 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 --2- 172.21.15.47:0/464317017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d80004800 0x7f4da0154600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 >> 172.21.15.47:0/464317017 conn(0x7f4da00b3350 msgr2=0x7f4da00ae070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 shutdown_connections 2024-04-20T10:28:44.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.523+0000 7f4da5000640 1 -- 172.21.15.47:0/464317017 wait complete. 2024-04-20T10:28:44.543 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:44.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.739+0000 7ff1a2a00640 1 Processor -- start 2024-04-20T10:28:44.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.739+0000 7ff1a2a00640 1 -- start start 2024-04-20T10:28:44.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff1a2a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff19c156c20 0x7ff19c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff1a2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff1a2a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff19c141c60 con 0x7ff19c15ca40 2024-04-20T10:28:44.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff1a2a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff19c0adab0 con 0x7ff19c156c20 2024-04-20T10:28:44.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff1a2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff19c15cf90 con 0x7ff19c0d0360 2024-04-20T10:28:44.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff19be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff19c156c20 0x7ff19c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff19be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff19c156c20 0x7ff19c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34976/0 (socket says 172.21.15.47:34976) 2024-04-20T10:28:44.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.741+0000 7ff19be00640 1 -- 172.21.15.47:0/2378694228 learned_addr learned my addr 172.21.15.47:0/2378694228 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:44.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.742+0000 7ff1a0a00640 1 --2- 172.21.15.47:0/2378694228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.742+0000 7ff19be00640 1 -- 172.21.15.47:0/2378694228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 msgr2=0x7ff19c0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:44.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.742+0000 7ff19be00640 1 --2- 172.21.15.47:0/2378694228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c0d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.742+0000 7ff19be00640 1 -- 172.21.15.47:0/2378694228 >> v1:172.21.15.175:6789/0 conn(0x7ff19c15ca40 legacy=0x7ff19c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:44.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.742+0000 7ff19be00640 1 -- 172.21.15.47:0/2378694228 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c15d270 con 0x7ff19c156c20 2024-04-20T10:28:44.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.742+0000 7ff1a0a00640 1 --2- 172.21.15.47:0/2378694228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c0d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:44.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.743+0000 7ff19be00640 1 --2- 172.21.15.47:0/2378694228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c156c20 0x7ff19c06b900 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009870 tx=0x7ff18c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f457e3759d39ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:44.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.743+0000 7ff19b400640 1 -- 172.21.15.47:0/2378694228 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c035020 con 0x7ff19c156c20 2024-04-20T10:28:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.743+0000 7ff19b400640 1 -- 172.21.15.47:0/2378694228 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff18c031640 con 0x7ff19c156c20 2024-04-20T10:28:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.743+0000 7ff19b400640 1 -- 172.21.15.47:0/2378694228 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c02d040 con 0x7ff19c156c20 2024-04-20T10:28:44.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.744+0000 7ff1a2a00640 1 -- 172.21.15.47:0/2378694228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c156c20 msgr2=0x7ff19c06b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.744+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/2378694228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c156c20 0x7ff19c06b900 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff18c009870 tx=0x7ff18c02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.744+0000 7ff1a2a00640 1 -- 172.21.15.47:0/2378694228 shutdown_connections 2024-04-20T10:28:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.744+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/2378694228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c0d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.744+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/2378694228 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c156c20 0x7ff19c06b900 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.744+0000 7ff1a2a00640 1 -- 172.21.15.47:0/2378694228 >> 172.21.15.47:0/2378694228 conn(0x7ff19c0b3350 msgr2=0x7ff19c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.745+0000 7ff1a2a00640 1 -- 172.21.15.47:0/2378694228 shutdown_connections 2024-04-20T10:28:44.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.745+0000 7ff1a2a00640 1 -- 172.21.15.47:0/2378694228 wait complete. 2024-04-20T10:28:44.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.747+0000 7ff1a2a00640 1 Processor -- start 2024-04-20T10:28:44.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.747+0000 7ff1a2a00640 1 -- start start 2024-04-20T10:28:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.748+0000 7ff1a2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c098ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.748+0000 7ff1a2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff19c156c20 0x7ff19c151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a2a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c15ca40 0x7ff19c15e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a2a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c06be40 con 0x7ff19c15ca40 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff19c15d830 con 0x7ff19c156c20 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff19c06bfc0 con 0x7ff19c0d0360 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c15ca40 0x7ff19c15e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c15ca40 0x7ff19c15e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34992/0 (socket says 172.21.15.47:34992) 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 -- 172.21.15.47:0/466704029 learned_addr learned my addr 172.21.15.47:0/466704029 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff19be00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c098ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff193e00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff19c156c20 0x7ff19c151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 -- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 msgr2=0x7ff19c098ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c098ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 -- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff19c156c20 msgr2=0x7ff19c151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff19c156c20 0x7ff19c151ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.749+0000 7ff1a0a00640 1 -- 172.21.15.47:0/466704029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff19c099220 con 0x7ff19c15ca40 2024-04-20T10:28:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.750+0000 7ff1a0a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c15ca40 0x7ff19c15e640 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff19400ce70 tx=0x7ff19400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.750+0000 7ff198c00640 1 -- 172.21.15.47:0/466704029 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19401a070 con 0x7ff19c15ca40 2024-04-20T10:28:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.750+0000 7ff198c00640 1 -- 172.21.15.47:0/466704029 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff19400d840 con 0x7ff19c15ca40 2024-04-20T10:28:44.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.750+0000 7ff198c00640 1 -- 172.21.15.47:0/466704029 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19400dc30 con 0x7ff19c15ca40 2024-04-20T10:28:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.750+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff19c072050 con 0x7ff19c15ca40 2024-04-20T10:28:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.750+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff19c072610 con 0x7ff19c15ca40 2024-04-20T10:28:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.752+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff19c06ccf0 con 0x7ff19c15ca40 2024-04-20T10:28:44.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.752+0000 7ff198c00640 1 -- 172.21.15.47:0/466704029 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff19401d070 con 0x7ff19c15ca40 2024-04-20T10:28:44.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.753+0000 7ff198c00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff17806c8f0 0x7ff17806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:44.753 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.753+0000 7ff198c00640 1 -- 172.21.15.47:0/466704029 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff19408c740 con 0x7ff19c15ca40 2024-04-20T10:28:44.754 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.754+0000 7ff19be00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff17806c8f0 0x7ff17806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:44.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.755+0000 7ff19be00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff17806c8f0 0x7ff17806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff18c036000 tx=0x7ff18c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:44.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.759+0000 7ff198c00640 1 -- 172.21.15.47:0/466704029 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff19c06ccf0 con 0x7ff19c15ca40 2024-04-20T10:28:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.978+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff19c068980 con 0x7ff19c15ca40 2024-04-20T10:28:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.979+0000 7ff198c00640 1 -- 172.21.15.47:0/466704029 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7ff19405a7b0 con 0x7ff19c15ca40 2024-04-20T10:28:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:44.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff17806c8f0 msgr2=0x7ff17806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff17806c8f0 0x7ff17806edb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff18c036000 tx=0x7ff18c03c040 comp rx=0 tx=0).stop 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c15ca40 msgr2=0x7ff19c15e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c15ca40 0x7ff19c15e640 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff19400ce70 tx=0x7ff19400a710 comp rx=0 tx=0).stop 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 shutdown_connections 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff17806c8f0 0x7ff17806edb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff19c15ca40 0x7ff19c15e640 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff19c156c20 0x7ff19c151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 --2- 172.21.15.47:0/466704029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff19c0d0360 0x7ff19c098ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.981+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 >> 172.21.15.47:0/466704029 conn(0x7ff19c0b3350 msgr2=0x7ff19c0b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.982+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 shutdown_connections 2024-04-20T10:28:44.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:44.982+0000 7ff1a2a00640 1 -- 172.21.15.47:0/466704029 wait complete. 2024-04-20T10:28:45.001 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:28:45.001 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:45.059 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:45.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.270+0000 7f35db600640 1 Processor -- start 2024-04-20T10:28:45.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.271+0000 7f35db600640 1 -- start start 2024-04-20T10:28:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.272+0000 7f35db600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f35d40d0a70 0x7f35d406e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.272+0000 7f35db600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f35d406dbb0 0x7f35d40ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.272+0000 7f35db600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35d406f060 con 0x7f35d415e8a0 2024-04-20T10:28:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.272+0000 7f35db600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f35d406e7c0 con 0x7f35d40d0a70 2024-04-20T10:28:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.272+0000 7f35db600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f35d406eaa0 con 0x7f35d406dbb0 2024-04-20T10:28:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f35d40d0a70 0x7f35d406e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f35d40d0a70 0x7f35d406e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35008/0 (socket says 172.21.15.47:35008) 2024-04-20T10:28:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d3e00640 1 -- 172.21.15.47:0/3979165911 learned_addr learned my addr 172.21.15.47:0/3979165911 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d8c00640 1 --1- 172.21.15.47:0/3979165911 >> v1:172.21.15.47:6789/0 conn(0x7f35d415e8a0 0x7f35d40d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44542/0 (socket says 172.21.15.47:44542) 2024-04-20T10:28:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d3e00640 1 -- 172.21.15.47:0/3979165911 >> v2:172.21.15.175:3300/0 conn(0x7f35d406dbb0 msgr2=0x7f35d40ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d9600640 1 --2- 172.21.15.47:0/3979165911 >> v2:172.21.15.175:3300/0 conn(0x7f35d406dbb0 0x7f35d40ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d3e00640 1 --2- 172.21.15.47:0/3979165911 >> v2:172.21.15.175:3300/0 conn(0x7f35d406dbb0 0x7f35d40ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d3e00640 1 -- 172.21.15.47:0/3979165911 >> v1:172.21.15.47:6789/0 conn(0x7f35d415e8a0 legacy=0x7f35d40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.273+0000 7f35d3e00640 1 -- 172.21.15.47:0/3979165911 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d406ed80 con 0x7f35d40d0a70 2024-04-20T10:28:45.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.274+0000 7f35d3e00640 1 --2- 172.21.15.47:0/3979165911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d406e1e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f35d40719a0 tx=0x7f35c402b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=793a84c9fcdf84fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35d3400640 1 -- 172.21.15.47:0/3979165911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4034070 con 0x7f35d40d0a70 2024-04-20T10:28:45.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35d3400640 1 -- 172.21.15.47:0/3979165911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4030870 con 0x7f35d40d0a70 2024-04-20T10:28:45.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35d3400640 1 -- 172.21.15.47:0/3979165911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4030c40 con 0x7f35d40d0a70 2024-04-20T10:28:45.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35db600640 1 -- 172.21.15.47:0/3979165911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 msgr2=0x7f35d406e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35db600640 1 --2- 172.21.15.47:0/3979165911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d406e1e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f35d40719a0 tx=0x7f35c402b9b0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35db600640 1 -- 172.21.15.47:0/3979165911 shutdown_connections 2024-04-20T10:28:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35db600640 1 --2- 172.21.15.47:0/3979165911 >> v2:172.21.15.175:3300/0 conn(0x7f35d406dbb0 0x7f35d40ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35db600640 1 --2- 172.21.15.47:0/3979165911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d406e1e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35db600640 1 -- 172.21.15.47:0/3979165911 >> 172.21.15.47:0/3979165911 conn(0x7f35d40b3350 msgr2=0x7f35d40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.275+0000 7f35db600640 1 -- 172.21.15.47:0/3979165911 shutdown_connections 2024-04-20T10:28:45.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.276+0000 7f35db600640 1 -- 172.21.15.47:0/3979165911 wait complete. 2024-04-20T10:28:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.277+0000 7f35db600640 1 Processor -- start 2024-04-20T10:28:45.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.277+0000 7f35db600640 1 -- start start 2024-04-20T10:28:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.278+0000 7f35db600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d406dbb0 0x7f35d4099ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.278+0000 7f35db600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d4161300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.278+0000 7f35db600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d415e8a0 0x7f35d4157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.278+0000 7f35db600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f35d406fef0 con 0x7f35d40d0a70 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.278+0000 7f35db600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f35d406fd70 con 0x7f35d415e8a0 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.278+0000 7f35db600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f35d40adbc0 con 0x7f35d406dbb0 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d4161300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d4161300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35020/0 (socket says 172.21.15.47:35020) 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 -- 172.21.15.47:0/389519283 learned_addr learned my addr 172.21.15.47:0/389519283 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d8c00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d406dbb0 0x7f35d4099ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 -- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d406dbb0 msgr2=0x7f35d4099ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d406dbb0 0x7f35d4099ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 -- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d415e8a0 msgr2=0x7f35d4157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d9600640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d415e8a0 0x7f35d4157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d415e8a0 0x7f35d4157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d3e00640 1 -- 172.21.15.47:0/389519283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35d4156780 con 0x7f35d40d0a70 2024-04-20T10:28:45.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.279+0000 7f35d8c00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d406dbb0 0x7f35d4099ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:45.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.280+0000 7f35d3e00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d4161300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f35c402b9f0 tx=0x7f35c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.280+0000 7f35d0c00640 1 -- 172.21.15.47:0/389519283 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4048070 con 0x7f35d40d0a70 2024-04-20T10:28:45.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.280+0000 7f35d0c00640 1 -- 172.21.15.47:0/389519283 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35c4039d90 con 0x7f35d40d0a70 2024-04-20T10:28:45.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.280+0000 7f35d0c00640 1 -- 172.21.15.47:0/389519283 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35c4034040 con 0x7f35d40d0a70 2024-04-20T10:28:45.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.280+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35d4156a60 con 0x7f35d40d0a70 2024-04-20T10:28:45.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.280+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35d4157020 con 0x7f35d40d0a70 2024-04-20T10:28:45.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.281+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35d4068980 con 0x7f35d40d0a70 2024-04-20T10:28:45.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.282+0000 7f35d0c00640 1 -- 172.21.15.47:0/389519283 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f35c404b070 con 0x7f35d40d0a70 2024-04-20T10:28:45.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.283+0000 7f35d0c00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35b006c940 0x7f35b006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.283+0000 7f35d0c00640 1 -- 172.21.15.47:0/389519283 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f35c40ae2a0 con 0x7f35d40d0a70 2024-04-20T10:28:45.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.284+0000 7f35d8c00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35b006c940 0x7f35b006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.285+0000 7f35d8c00640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35b006c940 0x7f35b006ee00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000f60 tx=0x7f35bc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.288+0000 7f35d0c00640 1 -- 172.21.15.47:0/389519283 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f35c407c300 con 0x7f35d40d0a70 2024-04-20T10:28:45.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.494+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f35d406d160 con 0x7f35d40d0a70 2024-04-20T10:28:45.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.497+0000 7f35d0c00640 1 -- 172.21.15.47:0/389519283 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f35c407bc00 con 0x7f35d40d0a70 2024-04-20T10:28:45.498 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:45.498 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35b006c940 msgr2=0x7f35b006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35b006c940 0x7f35b006ee00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f35bc000f60 tx=0x7f35bc009040 comp rx=0 tx=0).stop 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 msgr2=0x7f35d4161300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d4161300 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f35c402b9f0 tx=0x7f35c4004730 comp rx=0 tx=0).stop 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 shutdown_connections 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f35d415e8a0 0x7f35d4157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f35b006c940 0x7f35b006ee00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f35d40d0a70 0x7f35d4161300 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 --2- 172.21.15.47:0/389519283 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f35d406dbb0 0x7f35d4099ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.500+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 >> 172.21.15.47:0/389519283 conn(0x7f35d40b3350 msgr2=0x7f35d40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:45.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.501+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 shutdown_connections 2024-04-20T10:28:45.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.501+0000 7f35db600640 1 -- 172.21.15.47:0/389519283 wait complete. 2024-04-20T10:28:45.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:45.520 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.737+0000 7fdae9400640 1 Processor -- start 2024-04-20T10:28:45.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.738+0000 7fdae9400640 1 -- start start 2024-04-20T10:28:45.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae9400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdae4156c20 0x7fdae406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae9400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdae415ca40 0x7fdae40ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae9400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdae4141c60 con 0x7fdae40ad8e0 2024-04-20T10:28:45.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae9400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fdae415cf90 con 0x7fdae4156c20 2024-04-20T10:28:45.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae9400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdae415d270 con 0x7fdae415ca40 2024-04-20T10:28:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae2800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdae4156c20 0x7fdae406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae2800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fdae4156c20 0x7fdae406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35024/0 (socket says 172.21.15.47:35024) 2024-04-20T10:28:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae2800640 1 -- 172.21.15.47:0/724840433 learned_addr learned my addr 172.21.15.47:0/724840433 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.739+0000 7fdae3200640 1 --1- 172.21.15.47:0/724840433 >> v1:172.21.15.47:6789/0 conn(0x7fdae40ad8e0 0x7fdae40d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44558/0 (socket says 172.21.15.47:44558) 2024-04-20T10:28:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.740+0000 7fdae1e00640 1 --2- 172.21.15.47:0/724840433 >> v2:172.21.15.175:3300/0 conn(0x7fdae415ca40 0x7fdae40ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.740+0000 7fdae2800640 1 -- 172.21.15.47:0/724840433 >> v2:172.21.15.175:3300/0 conn(0x7fdae415ca40 msgr2=0x7fdae40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.740+0000 7fdae2800640 1 --2- 172.21.15.47:0/724840433 >> v2:172.21.15.175:3300/0 conn(0x7fdae415ca40 0x7fdae40ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.740+0000 7fdae2800640 1 -- 172.21.15.47:0/724840433 >> v1:172.21.15.47:6789/0 conn(0x7fdae40ad8e0 legacy=0x7fdae40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.740+0000 7fdae2800640 1 -- 172.21.15.47:0/724840433 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae415d550 con 0x7fdae4156c20 2024-04-20T10:28:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.741+0000 7fdae2800640 1 --2- 172.21.15.47:0/724840433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae4156c20 0x7fdae406b900 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdae406fe40 tx=0x7fdad402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8c4fb46d589144e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.741+0000 7fdae1400640 1 -- 172.21.15.47:0/724840433 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4034070 con 0x7fdae4156c20 2024-04-20T10:28:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.741+0000 7fdae1400640 1 -- 172.21.15.47:0/724840433 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad4030860 con 0x7fdae4156c20 2024-04-20T10:28:45.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.741+0000 7fdae1400640 1 -- 172.21.15.47:0/724840433 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4030c30 con 0x7fdae4156c20 2024-04-20T10:28:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 -- 172.21.15.47:0/724840433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae4156c20 msgr2=0x7fdae406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 --2- 172.21.15.47:0/724840433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae4156c20 0x7fdae406b900 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fdae406fe40 tx=0x7fdad402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 -- 172.21.15.47:0/724840433 shutdown_connections 2024-04-20T10:28:45.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 --2- 172.21.15.47:0/724840433 >> v2:172.21.15.175:3300/0 conn(0x7fdae415ca40 0x7fdae40ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 --2- 172.21.15.47:0/724840433 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae4156c20 0x7fdae406b900 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 -- 172.21.15.47:0/724840433 >> 172.21.15.47:0/724840433 conn(0x7fdae40b3350 msgr2=0x7fdae40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 -- 172.21.15.47:0/724840433 shutdown_connections 2024-04-20T10:28:45.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.742+0000 7fdae9400640 1 -- 172.21.15.47:0/724840433 wait complete. 2024-04-20T10:28:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.744+0000 7fdae9400640 1 Processor -- start 2024-04-20T10:28:45.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.744+0000 7fdae9400640 1 -- start start 2024-04-20T10:28:45.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.745+0000 7fdae9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae40ad8e0 0x7fdae415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.745+0000 7fdae9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae4156c20 0x7fdae40ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.745+0000 7fdae9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae415ca40 0x7fdae406dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.745+0000 7fdae9400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdae406bfc0 con 0x7fdae40ad8e0 2024-04-20T10:28:45.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.745+0000 7fdae9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdae406be40 con 0x7fdae4156c20 2024-04-20T10:28:45.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.745+0000 7fdae9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdae40d0a70 con 0x7fdae415ca40 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.745+0000 7fdae2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae40ad8e0 0x7fdae415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae40ad8e0 0x7fdae415b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35030/0 (socket says 172.21.15.47:35030) 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 -- 172.21.15.47:0/1348007999 learned_addr learned my addr 172.21.15.47:0/1348007999 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae1e00640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae4156c20 0x7fdae40ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae3200640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae415ca40 0x7fdae406dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 -- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae415ca40 msgr2=0x7fdae406dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae415ca40 0x7fdae406dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 -- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae4156c20 msgr2=0x7fdae40ce6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae4156c20 0x7fdae40ce6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 -- 172.21.15.47:0/1348007999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae40ccb30 con 0x7fdae40ad8e0 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.746+0000 7fdae2800640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae40ad8e0 0x7fdae415b680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdad402b9e0 tx=0x7fdad4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.747+0000 7fdadaa00640 1 -- 172.21.15.47:0/1348007999 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4048070 con 0x7fdae40ad8e0 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.747+0000 7fdadaa00640 1 -- 172.21.15.47:0/1348007999 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdad4030660 con 0x7fdae40ad8e0 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.747+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae40ccdf0 con 0x7fdae40ad8e0 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.747+0000 7fdadaa00640 1 -- 172.21.15.47:0/1348007999 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdad4034040 con 0x7fdae40ad8e0 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.747+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae40cd690 con 0x7fdae40ad8e0 2024-04-20T10:28:45.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.748+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae40cd1e0 con 0x7fdae40ad8e0 2024-04-20T10:28:45.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.749+0000 7fdadaa00640 1 -- 172.21.15.47:0/1348007999 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdad4037070 con 0x7fdae40ad8e0 2024-04-20T10:28:45.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.750+0000 7fdadaa00640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:45.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.750+0000 7fdadaa00640 1 -- 172.21.15.47:0/1348007999 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fdad40adec0 con 0x7fdae40ad8e0 2024-04-20T10:28:45.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.751+0000 7fdae1e00640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:45.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.752+0000 7fdae1e00640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdac406c8f0 0x7fdac406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdad0006fd0 tx=0x7fdad0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:45.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.755+0000 7fdadaa00640 1 -- 172.21.15.47:0/1348007999 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdad407c050 con 0x7fdae40ad8e0 2024-04-20T10:28:45.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.968+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdae40cd890 con 0x7fdae40ad8e0 2024-04-20T10:28:45.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.969+0000 7fdadaa00640 1 -- 172.21.15.47:0/1348007999 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7fdae40cd890 con 0x7fdae40ad8e0 2024-04-20T10:28:45.969 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdac406c8f0 msgr2=0x7fdac406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdac406c8f0 0x7fdac406edb0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdad0006fd0 tx=0x7fdad0008040 comp rx=0 tx=0).stop 2024-04-20T10:28:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae40ad8e0 msgr2=0x7fdae415b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:45.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae40ad8e0 0x7fdae415b680 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fdad402b9e0 tx=0x7fdad4004730 comp rx=0 tx=0).stop 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 shutdown_connections 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdae415ca40 0x7fdae406dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdae4156c20 0x7fdae40ce6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdac406c8f0 0x7fdac406edb0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 --2- 172.21.15.47:0/1348007999 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdae40ad8e0 0x7fdae415b680 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 >> 172.21.15.47:0/1348007999 conn(0x7fdae40b3350 msgr2=0x7fdae40aef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 shutdown_connections 2024-04-20T10:28:45.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:45.972+0000 7fdae9400640 1 -- 172.21.15.47:0/1348007999 wait complete. 2024-04-20T10:28:45.992 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.198+0000 7ff726800640 1 Processor -- start 2024-04-20T10:28:46.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.198+0000 7ff726800640 1 -- start start 2024-04-20T10:28:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.200+0000 7ff726800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff7200d0360 0x7ff7200d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.200+0000 7ff726800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7200ade40 con 0x7ff72006dcb0 2024-04-20T10:28:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.200+0000 7ff726800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7200ae0e0 con 0x7ff72015b3b0 2024-04-20T10:28:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.200+0000 7ff726800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff72006f1b0 con 0x7ff7200d0360 2024-04-20T10:28:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.200+0000 7ff717e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff72006dcb0 0x7ff720071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44572/0 (socket says 172.21.15.47:44572) 2024-04-20T10:28:46.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.200+0000 7ff717e00640 1 -- 172.21.15.47:0/1049473691 learned_addr learned my addr 172.21.15.47:0/1049473691 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71fe00640 1 --2- 172.21.15.47:0/1049473691 >> v2:172.21.15.175:3300/0 conn(0x7ff7200d0360 0x7ff7200d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1300723057 0 0) 0x7ff7200ade40 con 0x7ff72006dcb0 2024-04-20T10:28:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff704003520 con 0x7ff72006dcb0 2024-04-20T10:28:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 811108378 0 0) 0x7ff704003520 con 0x7ff72006dcb0 2024-04-20T10:28:46.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 >> v2:172.21.15.175:3300/0 conn(0x7ff7200d0360 msgr2=0x7ff7200d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 --2- 172.21.15.47:0/1049473691 >> v2:172.21.15.175:3300/0 conn(0x7ff7200d0360 0x7ff7200d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 >> v1:172.21.15.175:6789/0 conn(0x7ff72015b3b0 legacy=0x7ff72006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72015b790 con 0x7ff72006dcb0 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7ff708001670 con 0x7ff72006dcb0 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 >> v1:172.21.15.47:6789/0 conn(0x7ff72006dcb0 legacy=0x7ff720071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7040039b0 0x7ff704003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff704004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7040051c0 0x7ff70400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff72006f1b0 con 0x7ff704004800 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff70400c060 con 0x7ff7040051c0 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.201+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff70400c260 con 0x7ff7040039b0 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff717e00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff704004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff71fe00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7040051c0 0x7ff70400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff71f400640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7040039b0 0x7ff704003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff717e00640 1 -- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7040039b0 msgr2=0x7ff704003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff717e00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff7040039b0 0x7ff704003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff717e00640 1 -- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7040051c0 msgr2=0x7ff70400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff717e00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7040051c0 0x7ff70400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff717e00640 1 -- 172.21.15.47:0/1049473691 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff704003520 con 0x7ff704004800 2024-04-20T10:28:46.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.202+0000 7ff71fe00640 1 -- 172.21.15.47:0/1049473691 reap_dead start 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.203+0000 7ff717e00640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff704004c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c240 tx=0x7ff704005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b1d5cf65d164cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.203+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708007d10 con 0x7ff704004800 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.203+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff708030070 con 0x7ff704004800 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.203+0000 7ff71ea00640 1 -- 172.21.15.47:0/1049473691 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff708005d70 con 0x7ff704004800 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.204+0000 7ff726800640 1 -- 172.21.15.47:0/1049473691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 msgr2=0x7ff704004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.204+0000 7ff726800640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff704004c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c240 tx=0x7ff704005830 comp rx=0 tx=0).stop 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.204+0000 7ff726800640 1 -- 172.21.15.47:0/1049473691 shutdown_connections 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.204+0000 7ff726800640 1 --2- 172.21.15.47:0/1049473691 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff704004c80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.204+0000 7ff726800640 1 -- 172.21.15.47:0/1049473691 >> 172.21.15.47:0/1049473691 conn(0x7ff7200b3350 msgr2=0x7ff7200b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.204+0000 7ff726800640 1 -- 172.21.15.47:0/1049473691 shutdown_connections 2024-04-20T10:28:46.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.204+0000 7ff726800640 1 -- 172.21.15.47:0/1049473691 wait complete. 2024-04-20T10:28:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.206+0000 7ff726800640 1 Processor -- start 2024-04-20T10:28:46.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.206+0000 7ff726800640 1 -- start start 2024-04-20T10:28:46.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.207+0000 7ff726800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff72014c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.207+0000 7ff726800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff72015d8e0 0x7ff7200b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.207+0000 7ff726800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff72009d8a0 0x7ff72009dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.207+0000 7ff726800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff7200ae940 con 0x7ff704004800 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.207+0000 7ff726800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7200ae660 con 0x7ff72015d8e0 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.207+0000 7ff726800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7200aec20 con 0x7ff72009d8a0 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff72014c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff72014c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35052/0 (socket says 172.21.15.47:35052) 2024-04-20T10:28:46.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 -- 172.21.15.47:0/191525289 learned_addr learned my addr 172.21.15.47:0/191525289 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff717e00640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff72015d8e0 0x7ff7200b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71fe00640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff72009d8a0 0x7ff72009dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 -- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff72009d8a0 msgr2=0x7ff72009dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff72009d8a0 0x7ff72009dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 -- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff72015d8e0 msgr2=0x7ff7200b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff72015d8e0 0x7ff7200b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71f400640 1 -- 172.21.15.47:0/191525289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7201f7820 con 0x7ff704004800 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff717e00640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff72015d8e0 0x7ff7200b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.208+0000 7ff71fe00640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff72009d8a0 0x7ff72009dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.209+0000 7ff71f400640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff72014c940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d3e0 tx=0x7ff71000d8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.209+0000 7ff717400640 1 -- 172.21.15.47:0/191525289 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710016020 con 0x7ff704004800 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.209+0000 7ff717400640 1 -- 172.21.15.47:0/191525289 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff710012800 con 0x7ff704004800 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.209+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7201f7b00 con 0x7ff704004800 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.209+0000 7ff717400640 1 -- 172.21.15.47:0/191525289 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710012bd0 con 0x7ff704004800 2024-04-20T10:28:46.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.209+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201f80c0 con 0x7ff704004800 2024-04-20T10:28:46.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.210+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff720068980 con 0x7ff704004800 2024-04-20T10:28:46.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.211+0000 7ff717400640 1 -- 172.21.15.47:0/191525289 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff710019020 con 0x7ff704004800 2024-04-20T10:28:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.212+0000 7ff717400640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff6fc06c920 0x7ff6fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.212+0000 7ff717400640 1 -- 172.21.15.47:0/191525289 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff71008bfd0 con 0x7ff704004800 2024-04-20T10:28:46.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.213+0000 7ff717e00640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff6fc06c920 0x7ff6fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.214 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.214+0000 7ff717e00640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff6fc06c920 0x7ff6fc06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff708034000 tx=0x7ff708039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:46.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.217+0000 7ff717400640 1 -- 172.21.15.47:0/191525289 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff710020020 con 0x7ff704004800 2024-04-20T10:28:46.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.424+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff720074dd0 con 0x7ff704004800 2024-04-20T10:28:46.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.425+0000 7ff717400640 1 -- 172.21.15.47:0/191525289 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7ff710059f20 con 0x7ff704004800 2024-04-20T10:28:46.426 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff6fc06c920 msgr2=0x7ff6fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff6fc06c920 0x7ff6fc06ede0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff708034000 tx=0x7ff708039040 comp rx=0 tx=0).stop 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 msgr2=0x7ff72014c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff72014c940 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ff71000d3e0 tx=0x7ff71000d8a0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 shutdown_connections 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff72009d8a0 0x7ff72009dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff72015d8e0 0x7ff7200b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff6fc06c920 0x7ff6fc06ede0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.428+0000 7ff726800640 1 --2- 172.21.15.47:0/191525289 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff704004800 0x7ff72014c940 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.429+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 >> 172.21.15.47:0/191525289 conn(0x7ff7200b3350 msgr2=0x7ff72006e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.429+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 shutdown_connections 2024-04-20T10:28:46.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.429+0000 7ff726800640 1 -- 172.21.15.47:0/191525289 wait complete. 2024-04-20T10:28:46.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:46.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.656+0000 7f94e8c00640 1 Processor -- start 2024-04-20T10:28:46.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.656+0000 7f94e8c00640 1 -- start start 2024-04-20T10:28:46.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.657+0000 7f94e8c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94e415b3b0 0x7f94e406eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e8c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e406dcb0 0x7f94e4071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e8c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94e40ade40 con 0x7f94e4072090 2024-04-20T10:28:46.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e8c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f94e40ae0e0 con 0x7f94e415b3b0 2024-04-20T10:28:46.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e8c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94e415b790 con 0x7f94e406dcb0 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e1e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94e415b3b0 0x7f94e406eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e1e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f94e415b3b0 0x7f94e406eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35056/0 (socket says 172.21.15.47:35056) 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e1e00640 1 -- 172.21.15.47:0/2741406203 learned_addr learned my addr 172.21.15.47:0/2741406203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e1400640 1 --2- 172.21.15.47:0/2741406203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e406dcb0 0x7f94e4071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.658+0000 7f94e1e00640 1 -- 172.21.15.47:0/2741406203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e406dcb0 msgr2=0x7f94e4071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.659+0000 7f94e1e00640 1 --2- 172.21.15.47:0/2741406203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e406dcb0 0x7f94e4071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.659+0000 7f94e1e00640 1 -- 172.21.15.47:0/2741406203 >> v1:172.21.15.175:6789/0 conn(0x7f94e4072090 legacy=0x7f94e40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.659+0000 7f94e1e00640 1 -- 172.21.15.47:0/2741406203 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e40ae380 con 0x7f94e415b3b0 2024-04-20T10:28:46.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.659+0000 7f94e1400640 1 --2- 172.21.15.47:0/2741406203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e406dcb0 0x7f94e4071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:46.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.660+0000 7f94e1e00640 1 --2- 172.21.15.47:0/2741406203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415b3b0 0x7f94e406eaa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f94e406cfa0 tx=0x7f94d402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b46ec310d893ea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:46.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.660+0000 7f94e0a00640 1 -- 172.21.15.47:0/2741406203 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d402ca50 con 0x7f94e415b3b0 2024-04-20T10:28:46.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.660+0000 7f94e0a00640 1 -- 172.21.15.47:0/2741406203 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94d402cbf0 con 0x7f94e415b3b0 2024-04-20T10:28:46.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.660+0000 7f94e0a00640 1 -- 172.21.15.47:0/2741406203 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d40343f0 con 0x7f94e415b3b0 2024-04-20T10:28:46.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 -- 172.21.15.47:0/2741406203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415b3b0 msgr2=0x7f94e406eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 --2- 172.21.15.47:0/2741406203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415b3b0 0x7f94e406eaa0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f94e406cfa0 tx=0x7f94d402beb0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 -- 172.21.15.47:0/2741406203 shutdown_connections 2024-04-20T10:28:46.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 --2- 172.21.15.47:0/2741406203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e406dcb0 0x7f94e4071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 --2- 172.21.15.47:0/2741406203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415b3b0 0x7f94e406eaa0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 -- 172.21.15.47:0/2741406203 >> 172.21.15.47:0/2741406203 conn(0x7f94e40b3350 msgr2=0x7f94e40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:46.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 -- 172.21.15.47:0/2741406203 shutdown_connections 2024-04-20T10:28:46.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.661+0000 7f94e8c00640 1 -- 172.21.15.47:0/2741406203 wait complete. 2024-04-20T10:28:46.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.664+0000 7f94e8c00640 1 Processor -- start 2024-04-20T10:28:46.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.664+0000 7f94e8c00640 1 -- start start 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.665+0000 7f94e8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e406dcb0 0x7f94e41508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.665+0000 7f94e8c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e415b3b0 0x7f94e415c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.665+0000 7f94e8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415d670 0x7f94e409d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.665+0000 7f94e8c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94e40aec20 con 0x7f94e415d670 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.665+0000 7f94e8c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94e40ae940 con 0x7f94e406dcb0 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.665+0000 7f94e8c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94e406efe0 con 0x7f94e415b3b0 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415d670 0x7f94e409d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415d670 0x7f94e409d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35070/0 (socket says 172.21.15.47:35070) 2024-04-20T10:28:46.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 -- 172.21.15.47:0/1517848221 learned_addr learned my addr 172.21.15.47:0/1517848221 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e1e00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e406dcb0 0x7f94e41508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e1400640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e415b3b0 0x7f94e415c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 -- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e415b3b0 msgr2=0x7f94e415c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e415b3b0 0x7f94e415c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 -- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e406dcb0 msgr2=0x7f94e41508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e406dcb0 0x7f94e41508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e2800640 1 -- 172.21.15.47:0/1517848221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e4150de0 con 0x7f94e415d670 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.666+0000 7f94e1400640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e415b3b0 0x7f94e415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:46.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.667+0000 7f94e2800640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415d670 0x7f94e409d5e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c410 tx=0x7f94dc00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:46.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.667+0000 7f94da000640 1 -- 172.21.15.47:0/1517848221 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc017070 con 0x7f94e415d670 2024-04-20T10:28:46.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.667+0000 7f94da000640 1 -- 172.21.15.47:0/1517848221 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f94dc007230 con 0x7f94e415d670 2024-04-20T10:28:46.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.667+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e414f800 con 0x7f94e415d670 2024-04-20T10:28:46.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.667+0000 7f94da000640 1 -- 172.21.15.47:0/1517848221 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc007660 con 0x7f94e415d670 2024-04-20T10:28:46.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.667+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e414fdc0 con 0x7f94e415d670 2024-04-20T10:28:46.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.668+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e4068980 con 0x7f94e415d670 2024-04-20T10:28:46.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.669+0000 7f94da000640 1 -- 172.21.15.47:0/1517848221 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f94dc01a070 con 0x7f94e415d670 2024-04-20T10:28:46.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.670+0000 7f94da000640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f94c006c8f0 0x7f94c006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:46.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.670+0000 7f94da000640 1 -- 172.21.15.47:0/1517848221 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f94dc08b230 con 0x7f94e415d670 2024-04-20T10:28:46.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.671+0000 7f94e1e00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f94c006c8f0 0x7f94c006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:46.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.673+0000 7f94e1e00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f94c006c8f0 0x7f94c006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f94d4035000 tx=0x7f94d4030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:46.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.675+0000 7f94da000640 1 -- 172.21.15.47:0/1517848221 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f94dc059210 con 0x7f94e415d670 2024-04-20T10:28:46.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.881+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f94e4073510 con 0x7f94e415d670 2024-04-20T10:28:46.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.882+0000 7f94da000640 1 -- 172.21.15.47:0/1517848221 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f94dc058b10 con 0x7f94e415d670 2024-04-20T10:28:46.882 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.884+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f94c006c8f0 msgr2=0x7f94c006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.884+0000 7f94e8c00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f94c006c8f0 0x7f94c006edb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f94d4035000 tx=0x7f94d4030000 comp rx=0 tx=0).stop 2024-04-20T10:28:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.884+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415d670 msgr2=0x7f94e409d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.884+0000 7f94e8c00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415d670 0x7f94e409d5e0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f94dc00c410 tx=0x7f94dc00c8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 shutdown_connections 2024-04-20T10:28:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f94c006c8f0 0x7f94c006edb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94e415d670 0x7f94e409d5e0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94e415b3b0 0x7f94e415c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 --2- 172.21.15.47:0/1517848221 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94e406dcb0 0x7f94e41508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 >> 172.21.15.47:0/1517848221 conn(0x7f94e40b3350 msgr2=0x7f94e40725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 shutdown_connections 2024-04-20T10:28:46.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:46.885+0000 7f94e8c00640 1 -- 172.21.15.47:0/1517848221 wait complete. 2024-04-20T10:28:46.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.122+0000 7f1d2de00640 1 Processor -- start 2024-04-20T10:28:47.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.122+0000 7f1d2de00640 1 -- start start 2024-04-20T10:28:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.124+0000 7f1d2de00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1d2806db90 0x7f1d280d0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.124+0000 7f1d2de00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d280afb40 con 0x7f1d28099900 2024-04-20T10:28:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.124+0000 7f1d2de00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d2806baa0 con 0x7f1d280ad160 2024-04-20T10:28:47.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.124+0000 7f1d2de00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1d2806bd80 con 0x7f1d2806db90 2024-04-20T10:28:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.125+0000 7f1d27e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1d28099900 0x7f1d2806df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44582/0 (socket says 172.21.15.47:44582) 2024-04-20T10:28:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.125+0000 7f1d27e00640 1 -- 172.21.15.47:0/1952831701 learned_addr learned my addr 172.21.15.47:0/1952831701 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.125+0000 7f1d2ca00640 1 --2- 172.21.15.47:0/1952831701 >> v2:172.21.15.47:3300/0 conn(0x7f1d2806db90 0x7f1d280d0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.125+0000 7f1d2ca00640 1 -- 172.21.15.47:0/1952831701 >> v1:172.21.15.175:6789/0 conn(0x7f1d280ad160 legacy=0x7f1d280ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.125+0000 7f1d2ca00640 1 -- 172.21.15.47:0/1952831701 >> v1:172.21.15.47:6789/0 conn(0x7f1d28099900 legacy=0x7f1d2806df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.125+0000 7f1d2ca00640 1 -- 172.21.15.47:0/1952831701 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2806c060 con 0x7f1d2806db90 2024-04-20T10:28:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.125+0000 7f1d27400640 1 -- 172.21.15.47:0/1952831701 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1161105391 0 0) 0x7f1d280afb40 con 0x7f1d28099900 2024-04-20T10:28:47.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.127+0000 7f1d2ca00640 1 --2- 172.21.15.47:0/1952831701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d280d0760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0097a0 tx=0x7f1d1c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64151a68ba9eca9c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.127+0000 7f1d27400640 1 -- 172.21.15.47:0/1952831701 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c035020 con 0x7f1d2806db90 2024-04-20T10:28:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.127+0000 7f1d27400640 1 -- 172.21.15.47:0/1952831701 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1c031830 con 0x7f1d2806db90 2024-04-20T10:28:47.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.127+0000 7f1d27400640 1 -- 172.21.15.47:0/1952831701 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c031c00 con 0x7f1d2806db90 2024-04-20T10:28:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.127+0000 7f1d2de00640 1 -- 172.21.15.47:0/1952831701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 msgr2=0x7f1d280d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.127+0000 7f1d2de00640 1 --2- 172.21.15.47:0/1952831701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d280d0760 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0097a0 tx=0x7f1d1c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.128+0000 7f1d2de00640 1 -- 172.21.15.47:0/1952831701 shutdown_connections 2024-04-20T10:28:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.128+0000 7f1d2de00640 1 --2- 172.21.15.47:0/1952831701 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d280d0760 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.128+0000 7f1d2de00640 1 -- 172.21.15.47:0/1952831701 >> 172.21.15.47:0/1952831701 conn(0x7f1d280b5270 msgr2=0x7f1d280b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.128+0000 7f1d2de00640 1 -- 172.21.15.47:0/1952831701 shutdown_connections 2024-04-20T10:28:47.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.128+0000 7f1d2de00640 1 -- 172.21.15.47:0/1952831701 wait complete. 2024-04-20T10:28:47.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.129+0000 7f1d2de00640 1 Processor -- start 2024-04-20T10:28:47.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.130+0000 7f1d2de00640 1 -- start start 2024-04-20T10:28:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.130+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d28161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.130+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28099900 0x7f1d28150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d280ad160 0x7f1d281507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2de00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1d28162af0 con 0x7f1d2806db90 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d2806c620 con 0x7f1d28099900 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d28162c70 con 0x7f1d280ad160 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d28161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d28161850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35090/0 (socket says 172.21.15.47:35090) 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 -- 172.21.15.47:0/676717738 learned_addr learned my addr 172.21.15.47:0/676717738 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d27e00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28099900 0x7f1d28150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 -- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d280ad160 msgr2=0x7f1d281507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d280ad160 0x7f1d281507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 -- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28099900 msgr2=0x7f1d28150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28099900 0x7f1d28150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d2ca00640 1 -- 172.21.15.47:0/676717738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d2815fc60 con 0x7f1d2806db90 2024-04-20T10:28:47.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.131+0000 7f1d27e00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28099900 0x7f1d28150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:47.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.132+0000 7f1d2ca00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d28161850 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0097a0 tx=0x7f1d1c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:47.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.132+0000 7f1d24c00640 1 -- 172.21.15.47:0/676717738 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c004230 con 0x7f1d2806db90 2024-04-20T10:28:47.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.132+0000 7f1d24c00640 1 -- 172.21.15.47:0/676717738 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d1c0043f0 con 0x7f1d2806db90 2024-04-20T10:28:47.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.132+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d2815ff40 con 0x7f1d2806db90 2024-04-20T10:28:47.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.132+0000 7f1d24c00640 1 -- 172.21.15.47:0/676717738 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c003850 con 0x7f1d2806db90 2024-04-20T10:28:47.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.132+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d28160500 con 0x7f1d2806db90 2024-04-20T10:28:47.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.133+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d28075b30 con 0x7f1d2806db90 2024-04-20T10:28:47.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.135+0000 7f1d24c00640 1 -- 172.21.15.47:0/676717738 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d1c04c060 con 0x7f1d2806db90 2024-04-20T10:28:47.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.135+0000 7f1d24c00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d0806c920 0x7f1d0806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.136+0000 7f1d24c00640 1 -- 172.21.15.47:0/676717738 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f1d1c0adad0 con 0x7f1d2806db90 2024-04-20T10:28:47.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.136+0000 7f1d27e00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d0806c920 0x7f1d0806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.138+0000 7f1d27e00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d0806c920 0x7f1d0806ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1d28162200 tx=0x7f1d14009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:47.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.140+0000 7f1d24c00640 1 -- 172.21.15.47:0/676717738 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1d1c040070 con 0x7f1d2806db90 2024-04-20T10:28:47.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.344+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1d28073c40 con 0x7f1d2806db90 2024-04-20T10:28:47.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.345+0000 7f1d24c00640 1 -- 172.21.15.47:0/676717738 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f1d1c07bab0 con 0x7f1d2806db90 2024-04-20T10:28:47.345 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d0806c920 msgr2=0x7f1d0806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d0806c920 0x7f1d0806ede0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1d28162200 tx=0x7f1d14009040 comp rx=0 tx=0).stop 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 msgr2=0x7f1d28161850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d28161850 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c0097a0 tx=0x7f1d1c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 shutdown_connections 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d280ad160 0x7f1d281507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d28099900 0x7f1d28150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d0806c920 0x7f1d0806ede0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 --2- 172.21.15.47:0/676717738 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d2806db90 0x7f1d28161850 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.347+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 >> 172.21.15.47:0/676717738 conn(0x7f1d280b5270 msgr2=0x7f1d280ceb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.348+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 shutdown_connections 2024-04-20T10:28:47.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.348+0000 7f1d2de00640 1 -- 172.21.15.47:0/676717738 wait complete. 2024-04-20T10:28:47.368 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:47.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.580+0000 7ff2c8600640 1 Processor -- start 2024-04-20T10:28:47.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.580+0000 7ff2c8600640 1 -- start start 2024-04-20T10:28:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.582+0000 7ff2c8600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff2c00d0360 0x7ff2c00d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.582+0000 7ff2c8600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00ade40 con 0x7ff2c006dcb0 2024-04-20T10:28:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.582+0000 7ff2c8600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff2c00ae0e0 con 0x7ff2c015b3b0 2024-04-20T10:28:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.582+0000 7ff2c8600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff2c006f1b0 con 0x7ff2c00d0360 2024-04-20T10:28:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.582+0000 7ff2c5200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff2c006dcb0 0x7ff2c0071b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44594/0 (socket says 172.21.15.47:44594) 2024-04-20T10:28:47.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.582+0000 7ff2c5200640 1 -- 172.21.15.47:0/1968689628 learned_addr learned my addr 172.21.15.47:0/1968689628 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2c6600640 1 --2- 172.21.15.47:0/1968689628 >> v2:172.21.15.175:3300/0 conn(0x7ff2c00d0360 0x7ff2c00d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 975864116 0 0) 0x7ff2c00ade40 con 0x7ff2c006dcb0 2024-04-20T10:28:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff2a8005730 con 0x7ff2c006dcb0 2024-04-20T10:28:47.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3176080856 0 0) 0x7ff2a8005730 con 0x7ff2c006dcb0 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 >> v2:172.21.15.175:3300/0 conn(0x7ff2c00d0360 msgr2=0x7ff2c00d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 --2- 172.21.15.47:0/1968689628 >> v2:172.21.15.175:3300/0 conn(0x7ff2c00d0360 0x7ff2c00d0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 >> v1:172.21.15.175:6789/0 conn(0x7ff2c015b3b0 legacy=0x7ff2c006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c015b790 con 0x7ff2c006dcb0 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7ff2ac001670 con 0x7ff2c006dcb0 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 >> v1:172.21.15.47:6789/0 conn(0x7ff2c006dcb0 legacy=0x7ff2c0071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a8005bc0 0x7ff2a8005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2a8006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a80073d0 0x7ff2a800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c006f1b0 con 0x7ff2a8006a10 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a800e270 con 0x7ff2a8005bc0 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.583+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a800e470 con 0x7ff2a80073d0 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c5200640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2a8006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c5c00640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a8005bc0 0x7ff2a8005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c5200640 1 -- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a80073d0 msgr2=0x7ff2a800dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c6600640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a80073d0 0x7ff2a800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c5200640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2a80073d0 0x7ff2a800dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c5200640 1 -- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a8005bc0 msgr2=0x7ff2a8005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c5200640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a8005bc0 0x7ff2a8005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c5200640 1 -- 172.21.15.47:0/1968689628 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a8005730 con 0x7ff2a8006a10 2024-04-20T10:28:47.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.584+0000 7ff2c6600640 1 -- 172.21.15.47:0/1968689628 reap_dead start 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.585+0000 7ff2c5200640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2a8006e90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00c240 tx=0x7ff2a8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83670900b5662df9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.586+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac007d10 con 0x7ff2a8006a10 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.586+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2ac030070 con 0x7ff2a8006a10 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.586+0000 7ff2bfe00640 1 -- 172.21.15.47:0/1968689628 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac005d90 con 0x7ff2a8006a10 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.586+0000 7ff2c8600640 1 -- 172.21.15.47:0/1968689628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 msgr2=0x7ff2a8006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.586+0000 7ff2c8600640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2a8006e90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00c240 tx=0x7ff2a8007a40 comp rx=0 tx=0).stop 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.587+0000 7ff2c8600640 1 -- 172.21.15.47:0/1968689628 shutdown_connections 2024-04-20T10:28:47.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.587+0000 7ff2c8600640 1 --2- 172.21.15.47:0/1968689628 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2a8006e90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.587+0000 7ff2c8600640 1 -- 172.21.15.47:0/1968689628 >> 172.21.15.47:0/1968689628 conn(0x7ff2c00b3350 msgr2=0x7ff2c00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.587+0000 7ff2c8600640 1 -- 172.21.15.47:0/1968689628 shutdown_connections 2024-04-20T10:28:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.587+0000 7ff2c8600640 1 -- 172.21.15.47:0/1968689628 wait complete. 2024-04-20T10:28:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.588+0000 7ff2c8600640 1 Processor -- start 2024-04-20T10:28:47.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.589+0000 7ff2c8600640 1 -- start start 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.589+0000 7ff2c8600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2c01f79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c8600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c0150e00 0x7ff2c009dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c8600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c00b0310 0x7ff2c00b0790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c8600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c00ae940 con 0x7ff2a8006a10 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c8600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2c00ae660 con 0x7ff2c00b0310 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c8600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2c00aec20 con 0x7ff2c0150e00 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2c01f79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2c01f79a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35116/0 (socket says 172.21.15.47:35116) 2024-04-20T10:28:47.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 -- 172.21.15.47:0/2910487951 learned_addr learned my addr 172.21.15.47:0/2910487951 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c6600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c00b0310 0x7ff2c00b0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5200640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c0150e00 0x7ff2c009dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 -- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c0150e00 msgr2=0x7ff2c009dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c0150e00 0x7ff2c009dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 -- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c00b0310 msgr2=0x7ff2c00b0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c00b0310 0x7ff2c00b0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c5c00640 1 -- 172.21.15.47:0/2910487951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2c01f8060 con 0x7ff2a8006a10 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.590+0000 7ff2c6600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c00b0310 0x7ff2c00b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.591+0000 7ff2c5c00640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2c01f79a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e3c0 tx=0x7ff2b400e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.591+0000 7ff2bd600640 1 -- 172.21.15.47:0/2910487951 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4019070 con 0x7ff2a8006a10 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.591+0000 7ff2bd600640 1 -- 172.21.15.47:0/2910487951 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff2b40052e0 con 0x7ff2a8006a10 2024-04-20T10:28:47.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.591+0000 7ff2bd600640 1 -- 172.21.15.47:0/2910487951 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2b4007270 con 0x7ff2a8006a10 2024-04-20T10:28:47.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.591+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2c01f8340 con 0x7ff2a8006a10 2024-04-20T10:28:47.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.591+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2c015bb00 con 0x7ff2a8006a10 2024-04-20T10:28:47.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.592+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2c0068980 con 0x7ff2a8006a10 2024-04-20T10:28:47.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.594+0000 7ff2bd600640 1 -- 172.21.15.47:0/2910487951 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff2b4007410 con 0x7ff2a8006a10 2024-04-20T10:28:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.595+0000 7ff2bd600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a006c8f0 0x7ff2a006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:47.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.595+0000 7ff2bd600640 1 -- 172.21.15.47:0/2910487951 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff2b408af40 con 0x7ff2a8006a10 2024-04-20T10:28:47.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.596+0000 7ff2c5200640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a006c8f0 0x7ff2a006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:47.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.597+0000 7ff2c5200640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a006c8f0 0x7ff2a006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac034000 tx=0x7ff2ac009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:47.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.600+0000 7ff2bd600640 1 -- 172.21.15.47:0/2910487951 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff2b40588d0 con 0x7ff2a8006a10 2024-04-20T10:28:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.821+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff2c0151200 con 0x7ff2a8006a10 2024-04-20T10:28:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.822+0000 7ff2bd600640 1 -- 172.21.15.47:0/2910487951 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7ff2b405cbd0 con 0x7ff2a8006a10 2024-04-20T10:28:47.822 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:47.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a006c8f0 msgr2=0x7ff2a006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a006c8f0 0x7ff2a006edb0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac034000 tx=0x7ff2ac009030 comp rx=0 tx=0).stop 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 msgr2=0x7ff2c01f79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2c01f79a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff2b400e3c0 tx=0x7ff2b400e880 comp rx=0 tx=0).stop 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 shutdown_connections 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2c00b0310 0x7ff2c00b0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2c0150e00 0x7ff2c009dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff2a006c8f0 0x7ff2a006edb0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 --2- 172.21.15.47:0/2910487951 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff2a8006a10 0x7ff2c01f79a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.825+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 >> 172.21.15.47:0/2910487951 conn(0x7ff2c00b3350 msgr2=0x7ff2c006e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.826+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 shutdown_connections 2024-04-20T10:28:47.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:47.826+0000 7ff2c8600640 1 -- 172.21.15.47:0/2910487951 wait complete. 2024-04-20T10:28:47.845 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T10:28:47.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:47.846 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2024-04-20T10:28:47.937 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:28:47.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:28:47.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:47.937 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T10:28:48.000 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:28:48.000 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:48.061 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:28:48.061 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:48.119 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:48.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.324+0000 7ff157200640 1 Processor -- start 2024-04-20T10:28:48.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.324+0000 7ff157200640 1 -- start start 2024-04-20T10:28:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.326+0000 7ff157200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff158163b60 0x7ff158071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.326+0000 7ff157200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15806c310 con 0x7ff1581651d0 2024-04-20T10:28:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.326+0000 7ff157200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff15806c5f0 con 0x7ff158163070 2024-04-20T10:28:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.326+0000 7ff157200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff158073cf0 con 0x7ff158163b60 2024-04-20T10:28:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.327+0000 7ff14d400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff1581651d0 0x7ff158163450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44606/0 (socket says 172.21.15.47:44606) 2024-04-20T10:28:48.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.327+0000 7ff14d400640 1 -- 172.21.15.47:0/3861615249 learned_addr learned my addr 172.21.15.47:0/3861615249 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.327+0000 7ff156800640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff158163b60 0x7ff158071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.327+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4220174816 0 0) 0x7ff15806c310 con 0x7ff1581651d0 2024-04-20T10:28:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.327+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff13c003520 con 0x7ff1581651d0 2024-04-20T10:28:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1188362541 0 0) 0x7ff13c003520 con 0x7ff1581651d0 2024-04-20T10:28:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff158163b60 msgr2=0x7ff158071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:48.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff158163b60 0x7ff158071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 >> v1:172.21.15.175:6789/0 conn(0x7ff158163070 legacy=0x7ff1580735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15806c8d0 con 0x7ff1581651d0 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7ff140001670 con 0x7ff1581651d0 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 >> v1:172.21.15.47:6789/0 conn(0x7ff1581651d0 legacy=0x7ff158163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0039e0 0x7ff13c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff158163b60 0x7ff13c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c004d40 0x7ff13c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff158073cf0 con 0x7ff13c0039e0 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c00bb20 con 0x7ff13c004d40 2024-04-20T10:28:48.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff13c00bd20 con 0x7ff158163b60 2024-04-20T10:28:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.328+0000 7ff155e00640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0039e0 0x7ff13c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.329+0000 7ff156800640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c004d40 0x7ff13c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.329+0000 7ff155e00640 1 -- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff158163b60 msgr2=0x7ff13c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.329+0000 7ff155e00640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff158163b60 0x7ff13c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.329+0000 7ff155e00640 1 -- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c004d40 msgr2=0x7ff13c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.329+0000 7ff155e00640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff13c004d40 0x7ff13c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.329+0000 7ff155e00640 1 -- 172.21.15.47:0/3861615249 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c003520 con 0x7ff13c0039e0 2024-04-20T10:28:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.330+0000 7ff155e00640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0039e0 0x7ff13c003e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff148001750 tx=0x7ff13c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554a8033c725091e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.330+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148001370 con 0x7ff13c0039e0 2024-04-20T10:28:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.330+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1480052d0 con 0x7ff13c0039e0 2024-04-20T10:28:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.330+0000 7ff155400640 1 -- 172.21.15.47:0/3861615249 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff148003830 con 0x7ff13c0039e0 2024-04-20T10:28:48.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.331+0000 7ff157200640 1 -- 172.21.15.47:0/3861615249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0039e0 msgr2=0x7ff13c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.331+0000 7ff157200640 1 --2- 172.21.15.47:0/3861615249 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff13c0039e0 0x7ff13c003e00 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff148001750 tx=0x7ff13c005830 comp rx=0 tx=0).stop 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.331+0000 7ff156800640 1 -- 172.21.15.47:0/3861615249 reap_dead start 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.331+0000 7ff157200640 1 -- 172.21.15.47:0/3861615249 shutdown_connections 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.331+0000 7ff157200640 1 -- 172.21.15.47:0/3861615249 >> 172.21.15.47:0/3861615249 conn(0x7ff1580d0040 msgr2=0x7ff1580d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.331+0000 7ff157200640 1 -- 172.21.15.47:0/3861615249 shutdown_connections 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.331+0000 7ff157200640 1 -- 172.21.15.47:0/3861615249 wait complete. 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.333+0000 7ff157200640 1 Processor -- start 2024-04-20T10:28:48.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.334+0000 7ff157200640 1 -- start start 2024-04-20T10:28:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff157200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff13c0039e0 0x7ff1580c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff157200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff15809b400 0x7ff15809d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff157200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff158155da0 0x7ff158156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff157200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff1581662a0 con 0x7ff15809b400 2024-04-20T10:28:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff157200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff15806ce90 con 0x7ff158155da0 2024-04-20T10:28:48.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff157200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff158071d90 con 0x7ff13c0039e0 2024-04-20T10:28:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff14d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff15809b400 0x7ff15809d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff14d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff15809b400 0x7ff15809d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35136/0 (socket says 172.21.15.47:35136) 2024-04-20T10:28:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff14d400640 1 -- 172.21.15.47:0/171409203 learned_addr learned my addr 172.21.15.47:0/171409203 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff156800640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff158155da0 0x7ff158156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.335+0000 7ff155e00640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff13c0039e0 0x7ff1580c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff14d400640 1 -- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff13c0039e0 msgr2=0x7ff1580c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff14d400640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff13c0039e0 0x7ff1580c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff14d400640 1 -- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff158155da0 msgr2=0x7ff158156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff14d400640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff158155da0 0x7ff158156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff14d400640 1 -- 172.21.15.47:0/171409203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff15809b970 con 0x7ff15809b400 2024-04-20T10:28:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff156800640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff158155da0 0x7ff158156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:48.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff14d400640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff15809b400 0x7ff15809d1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff14000c690 tx=0x7ff14000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.336+0000 7ff14ea00640 1 -- 172.21.15.47:0/171409203 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff140018070 con 0x7ff15809b400 2024-04-20T10:28:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.337+0000 7ff14ea00640 1 -- 172.21.15.47:0/171409203 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff1400072d0 con 0x7ff15809b400 2024-04-20T10:28:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.337+0000 7ff14ea00640 1 -- 172.21.15.47:0/171409203 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1400047e0 con 0x7ff15809b400 2024-04-20T10:28:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.337+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff15809bc50 con 0x7ff15809b400 2024-04-20T10:28:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.337+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff15809c210 con 0x7ff15809b400 2024-04-20T10:28:48.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.338+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff15809c640 con 0x7ff15809b400 2024-04-20T10:28:48.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.339+0000 7ff14ea00640 1 -- 172.21.15.47:0/171409203 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff14001b070 con 0x7ff15809b400 2024-04-20T10:28:48.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.341+0000 7ff14ea00640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff13406c8f0 0x7ff13406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.341+0000 7ff14ea00640 1 -- 172.21.15.47:0/171409203 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff14008b4e0 con 0x7ff15809b400 2024-04-20T10:28:48.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.341+0000 7ff155e00640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff13406c8f0 0x7ff13406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.343+0000 7ff155e00640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff13406c8f0 0x7ff13406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff148033000 tx=0x7ff148009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:48.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.346+0000 7ff14ea00640 1 -- 172.21.15.47:0/171409203 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff140059430 con 0x7ff15809b400 2024-04-20T10:28:48.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.557+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff158156740 con 0x7ff15809b400 2024-04-20T10:28:48.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.560+0000 7ff14ea00640 1 -- 172.21.15.47:0/171409203 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7ff140058d30 con 0x7ff15809b400 2024-04-20T10:28:48.561 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:48.561 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff13406c8f0 msgr2=0x7ff13406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff13406c8f0 0x7ff13406edb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff148033000 tx=0x7ff148009030 comp rx=0 tx=0).stop 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff15809b400 msgr2=0x7ff15809d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff15809b400 0x7ff15809d1c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff14000c690 tx=0x7ff14000cb50 comp rx=0 tx=0).stop 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 shutdown_connections 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff158155da0 0x7ff158156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff13406c8f0 0x7ff13406edb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff15809b400 0x7ff15809d1c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 --2- 172.21.15.47:0/171409203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff13c0039e0 0x7ff1580c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.563+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 >> 172.21.15.47:0/171409203 conn(0x7ff1580d0040 msgr2=0x7ff158163450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.564+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 shutdown_connections 2024-04-20T10:28:48.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.564+0000 7ff157200640 1 -- 172.21.15.47:0/171409203 wait complete. 2024-04-20T10:28:48.584 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:48.585 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:48.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.798+0000 7f5bdf600640 1 Processor -- start 2024-04-20T10:28:48.799 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.799+0000 7f5bdf600640 1 -- start start 2024-04-20T10:28:48.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.800+0000 7f5bdf600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5bd80d4270 0x7f5bd80d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.800+0000 7f5bdf600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd80d3820 con 0x7f5bd80d4650 2024-04-20T10:28:48.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.800+0000 7f5bdf600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5bd80d3b00 con 0x7f5bd80d0380 2024-04-20T10:28:48.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.800+0000 7f5bdf600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5bd80d40c0 con 0x7f5bd80d4270 2024-04-20T10:28:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.801+0000 7f5bd7e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5bd80d4650 0x7f5bd80b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44614/0 (socket says 172.21.15.47:44614) 2024-04-20T10:28:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.801+0000 7f5bd7e00640 1 -- 172.21.15.47:0/3843172650 learned_addr learned my addr 172.21.15.47:0/3843172650 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.801+0000 7f5bdcc00640 1 --2- 172.21.15.47:0/3843172650 >> v2:172.21.15.175:3300/0 conn(0x7f5bd80d4270 0x7f5bd80d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.801 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.801+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3035152014 0 0) 0x7f5bd80d3820 con 0x7f5bd80d4650 2024-04-20T10:28:48.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.801+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5bbc003520 con 0x7f5bd80d4650 2024-04-20T10:28:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1906740419 0 0) 0x7f5bbc003520 con 0x7f5bd80d4650 2024-04-20T10:28:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 >> v2:172.21.15.175:3300/0 conn(0x7f5bd80d4270 msgr2=0x7f5bd80d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 --2- 172.21.15.47:0/3843172650 >> v2:172.21.15.175:3300/0 conn(0x7f5bd80d4270 0x7f5bd80d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.803 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 >> v1:172.21.15.175:6789/0 conn(0x7f5bd80d0380 legacy=0x7f5bd80d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd80d3de0 con 0x7f5bd80d4650 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f5bc4001670 con 0x7f5bd80d4650 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 >> v1:172.21.15.47:6789/0 conn(0x7f5bd80d4650 legacy=0x7f5bd80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0039b0 0x7f5bbc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bbc004800 0x7f5bbc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0051c0 0x7f5bbc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80d40c0 con 0x7f5bbc004800 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bbc00c060 con 0x7f5bbc0051c0 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bbc00c260 con 0x7f5bbc0039b0 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.802+0000 7f5bd7e00640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bbc004800 0x7f5bbc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bdcc00640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0039b0 0x7f5bbc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bdd600640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0051c0 0x7f5bbc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bd7e00640 1 -- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0039b0 msgr2=0x7f5bbc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:48.804 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bd7e00640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bbc0039b0 0x7f5bbc003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bd7e00640 1 -- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0051c0 msgr2=0x7f5bbc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bd7e00640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0051c0 0x7f5bbc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bd7e00640 1 -- 172.21.15.47:0/3843172650 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bbc003520 con 0x7f5bbc004800 2024-04-20T10:28:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bdd600640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc0051c0 0x7f5bbc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:48.805 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.803+0000 7f5bdd600640 1 -- 172.21.15.47:0/3843172650 reap_dead start 2024-04-20T10:28:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.804+0000 7f5bd7e00640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bbc004800 0x7f5bbc004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c210 tx=0x7f5bbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b94722c6f56244e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.804+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4007d30 con 0x7f5bbc004800 2024-04-20T10:28:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.804+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bc4030070 con 0x7f5bbc004800 2024-04-20T10:28:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.804+0000 7f5bd7400640 1 -- 172.21.15.47:0/3843172650 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc402cda0 con 0x7f5bbc004800 2024-04-20T10:28:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.805+0000 7f5bdf600640 1 -- 172.21.15.47:0/3843172650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bbc004800 msgr2=0x7f5bbc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.805+0000 7f5bdf600640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bbc004800 0x7f5bbc004c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c210 tx=0x7f5bbc005830 comp rx=0 tx=0).stop 2024-04-20T10:28:48.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.805+0000 7f5bdf600640 1 -- 172.21.15.47:0/3843172650 shutdown_connections 2024-04-20T10:28:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.805+0000 7f5bdf600640 1 --2- 172.21.15.47:0/3843172650 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bbc004800 0x7f5bbc004c80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.805+0000 7f5bdf600640 1 -- 172.21.15.47:0/3843172650 >> 172.21.15.47:0/3843172650 conn(0x7f5bd80b99c0 msgr2=0x7f5bd80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.805+0000 7f5bdf600640 1 -- 172.21.15.47:0/3843172650 shutdown_connections 2024-04-20T10:28:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.805+0000 7f5bdf600640 1 -- 172.21.15.47:0/3843172650 wait complete. 2024-04-20T10:28:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.807+0000 7f5bdf600640 1 Processor -- start 2024-04-20T10:28:48.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.807+0000 7f5bdf600640 1 -- start start 2024-04-20T10:28:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.808+0000 7f5bdf600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004800 0x7f5bd8151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.808+0000 7f5bdf600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd806c210 0x7f5bd806dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.808+0000 7f5bdf600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bd806c670 0x7f5bd806e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.808+0000 7f5bdf600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80d4ae0 con 0x7f5bd806c670 2024-04-20T10:28:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.808+0000 7f5bdf600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bd80d53b0 con 0x7f5bbc004800 2024-04-20T10:28:48.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.808+0000 7f5bdf600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bd80d4c60 con 0x7f5bd806c210 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.808+0000 7f5bdd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bd806c670 0x7f5bd806e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bd806c670 0x7f5bd806e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35150/0 (socket says 172.21.15.47:35150) 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 -- 172.21.15.47:0/2405847054 learned_addr learned my addr 172.21.15.47:0/2405847054 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdcc00640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004800 0x7f5bd8151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bd7e00640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd806c210 0x7f5bd806dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 -- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd806c210 msgr2=0x7f5bd806dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd806c210 0x7f5bd806dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 -- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004800 msgr2=0x7f5bd8151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004800 0x7f5bd8151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 -- 172.21.15.47:0/2405847054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bd80cdc00 con 0x7f5bd806c670 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.809+0000 7f5bdd600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bd806c670 0x7f5bd806e2d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d540 tx=0x7f5bd000da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.810+0000 7f5bd4c00640 1 -- 172.21.15.47:0/2405847054 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0018070 con 0x7f5bd806c670 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.810+0000 7f5bd4c00640 1 -- 172.21.15.47:0/2405847054 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5bd0003da0 con 0x7f5bd806c670 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.810+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bd80cdee0 con 0x7f5bd806c670 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.810+0000 7f5bd4c00640 1 -- 172.21.15.47:0/2405847054 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd0005330 con 0x7f5bd806c670 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.810+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bd815e2c0 con 0x7f5bd806c670 2024-04-20T10:28:48.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.811+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bd8068980 con 0x7f5bd806c670 2024-04-20T10:28:48.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.812+0000 7f5bd4c00640 1 -- 172.21.15.47:0/2405847054 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5bd001b070 con 0x7f5bd806c670 2024-04-20T10:28:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.813+0000 7f5bd4c00640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5bb806c8f0 0x7f5bb806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:48.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.813+0000 7f5bd4c00640 1 -- 172.21.15.47:0/2405847054 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f5bd008b570 con 0x7f5bd806c670 2024-04-20T10:28:48.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.814+0000 7f5bdcc00640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5bb806c8f0 0x7f5bb806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:48.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.815+0000 7f5bdcc00640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5bb806c8f0 0x7f5bb806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009a90 tx=0x7f5bc8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:48.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:48.817+0000 7f5bd4c00640 1 -- 172.21.15.47:0/2405847054 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5bd00595e0 con 0x7f5bd806c670 2024-04-20T10:28:49.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.022+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5bd815fc40 con 0x7f5bd806c670 2024-04-20T10:28:49.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.023+0000 7f5bd4c00640 1 -- 172.21.15.47:0/2405847054 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f5bd0058ee0 con 0x7f5bd806c670 2024-04-20T10:28:49.023 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5bb806c8f0 msgr2=0x7f5bb806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5bb806c8f0 0x7f5bb806edb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8009a90 tx=0x7f5bc8008040 comp rx=0 tx=0).stop 2024-04-20T10:28:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bd806c670 msgr2=0x7f5bd806e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bd806c670 0x7f5bd806e2d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000d540 tx=0x7f5bd000da00 comp rx=0 tx=0).stop 2024-04-20T10:28:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 shutdown_connections 2024-04-20T10:28:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5bb806c8f0 0x7f5bb806edb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5bd806c670 0x7f5bd806e2d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bd806c210 0x7f5bd806dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 --2- 172.21.15.47:0/2405847054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bbc004800 0x7f5bd8151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 >> 172.21.15.47:0/2405847054 conn(0x7f5bd80b99c0 msgr2=0x7f5bd80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 shutdown_connections 2024-04-20T10:28:49.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.025+0000 7f5bdf600640 1 -- 172.21.15.47:0/2405847054 wait complete. 2024-04-20T10:28:49.047 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:49.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.269+0000 7f1c16c00640 1 Processor -- start 2024-04-20T10:28:49.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.269+0000 7f1c16c00640 1 -- start start 2024-04-20T10:28:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.271+0000 7f1c16c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1c10145e90 0x7f1c1006f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.271+0000 7f1c16c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1c1006d020 0x7f1c100b5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.271+0000 7f1c16c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1c100b41e0 con 0x7f1c10070dc0 2024-04-20T10:28:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.271+0000 7f1c16c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1c100b44c0 con 0x7f1c10145e90 2024-04-20T10:28:49.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.271+0000 7f1c16c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f1c100b47a0 con 0x7f1c1006d020 2024-04-20T10:28:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.271+0000 7f1c15800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1c10145e90 0x7f1c1006f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.272+0000 7f1c15800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f1c10145e90 0x7f1c1006f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35160/0 (socket says 172.21.15.47:35160) 2024-04-20T10:28:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.272+0000 7f1c15800640 1 -- 172.21.15.47:0/293490913 learned_addr learned my addr 172.21.15.47:0/293490913 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.272+0000 7f1c16200640 1 --2- 172.21.15.47:0/293490913 >> v2:172.21.15.175:3300/0 conn(0x7f1c1006d020 0x7f1c100b5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.272+0000 7f1c15800640 1 -- 172.21.15.47:0/293490913 >> v2:172.21.15.175:3300/0 conn(0x7f1c1006d020 msgr2=0x7f1c100b5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:49.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.272+0000 7f1c15800640 1 --2- 172.21.15.47:0/293490913 >> v2:172.21.15.175:3300/0 conn(0x7f1c1006d020 0x7f1c100b5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.272+0000 7f1c15800640 1 -- 172.21.15.47:0/293490913 >> v1:172.21.15.175:6789/0 conn(0x7f1c10070dc0 legacy=0x7f1c1006c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.272+0000 7f1c15800640 1 -- 172.21.15.47:0/293490913 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c100b4a80 con 0x7f1c10145e90 2024-04-20T10:28:49.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.274+0000 7f1c15800640 1 --2- 172.21.15.47:0/293490913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c10145e90 0x7f1c1006f8d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c10071640 tx=0x7f1c0402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc07e0ece29e566 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.274+0000 7f1c0fe00640 1 -- 172.21.15.47:0/293490913 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0402e070 con 0x7f1c10145e90 2024-04-20T10:28:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.274+0000 7f1c0fe00640 1 -- 172.21.15.47:0/293490913 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c04037070 con 0x7f1c10145e90 2024-04-20T10:28:49.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.274+0000 7f1c0fe00640 1 -- 172.21.15.47:0/293490913 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04031a00 con 0x7f1c10145e90 2024-04-20T10:28:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 -- 172.21.15.47:0/293490913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c10145e90 msgr2=0x7f1c1006f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 --2- 172.21.15.47:0/293490913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c10145e90 0x7f1c1006f8d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f1c10071640 tx=0x7f1c0402b880 comp rx=0 tx=0).stop 2024-04-20T10:28:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 -- 172.21.15.47:0/293490913 shutdown_connections 2024-04-20T10:28:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 --2- 172.21.15.47:0/293490913 >> v2:172.21.15.175:3300/0 conn(0x7f1c1006d020 0x7f1c100b5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 --2- 172.21.15.47:0/293490913 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c10145e90 0x7f1c1006f8d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 -- 172.21.15.47:0/293490913 >> 172.21.15.47:0/293490913 conn(0x7f1c10070090 msgr2=0x7f1c100704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 -- 172.21.15.47:0/293490913 shutdown_connections 2024-04-20T10:28:49.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.275+0000 7f1c16c00640 1 -- 172.21.15.47:0/293490913 wait complete. 2024-04-20T10:28:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.277+0000 7f1c16c00640 1 Processor -- start 2024-04-20T10:28:49.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.277+0000 7f1c16c00640 1 -- start start 2024-04-20T10:28:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c16c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c1006d020 0x7f1c1014f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c16c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c10070dc0 0x7f1c10135310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c16c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c10145e90 0x7f1c10136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c16c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1c100713a0 con 0x7f1c1006d020 2024-04-20T10:28:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c16c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c10071220 con 0x7f1c10070dc0 2024-04-20T10:28:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c16c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c100b57c0 con 0x7f1c10145e90 2024-04-20T10:28:49.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c15800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c1006d020 0x7f1c1014f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c15800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c1006d020 0x7f1c1014f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35172/0 (socket says 172.21.15.47:35172) 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.278+0000 7f1c15800640 1 -- 172.21.15.47:0/740983944 learned_addr learned my addr 172.21.15.47:0/740983944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c16200640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c10145e90 0x7f1c10136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c14e00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c10070dc0 0x7f1c10135310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c15800640 1 -- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c10145e90 msgr2=0x7f1c10136f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c15800640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c10145e90 0x7f1c10136f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c15800640 1 -- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c10070dc0 msgr2=0x7f1c10135310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c15800640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c10070dc0 0x7f1c10135310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c15800640 1 -- 172.21.15.47:0/740983944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c10140f20 con 0x7f1c1006d020 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c14e00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c10070dc0 0x7f1c10135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:49.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.279+0000 7f1c15800640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c1006d020 0x7f1c1014f8c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1c0402b8c0 tx=0x7f1c040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.280+0000 7f1c0d600640 1 -- 172.21.15.47:0/740983944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04032670 con 0x7f1c1006d020 2024-04-20T10:28:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.280+0000 7f1c0d600640 1 -- 172.21.15.47:0/740983944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1c04032830 con 0x7f1c1006d020 2024-04-20T10:28:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.280+0000 7f1c0d600640 1 -- 172.21.15.47:0/740983944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c04032c20 con 0x7f1c1006d020 2024-04-20T10:28:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.280+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c1015bde0 con 0x7f1c1006d020 2024-04-20T10:28:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.280+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c10140c40 con 0x7f1c1006d020 2024-04-20T10:28:49.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.281+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c10075b30 con 0x7f1c1006d020 2024-04-20T10:28:49.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.282+0000 7f1c0d600640 1 -- 172.21.15.47:0/740983944 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1c0404c060 con 0x7f1c1006d020 2024-04-20T10:28:49.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.283+0000 7f1c0d600640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1bf006c8f0 0x7f1bf006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.283+0000 7f1c0d600640 1 -- 172.21.15.47:0/740983944 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f1c040ada10 con 0x7f1c1006d020 2024-04-20T10:28:49.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.284+0000 7f1c14e00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1bf006c8f0 0x7f1bf006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.285+0000 7f1c14e00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1bf006c8f0 0x7f1bf006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1c10138840 tx=0x7f1bfc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:49.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.288+0000 7f1c0d600640 1 -- 172.21.15.47:0/740983944 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1c1015bde0 con 0x7f1c1006d020 2024-04-20T10:28:49.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.490+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1c100b6de0 con 0x7f1c1006d020 2024-04-20T10:28:49.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.491+0000 7f1c0d600640 1 -- 172.21.15.47:0/740983944 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f1c04040070 con 0x7f1c1006d020 2024-04-20T10:28:49.491 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:49.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1bf006c8f0 msgr2=0x7f1bf006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1bf006c8f0 0x7f1bf006edb0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f1c10138840 tx=0x7f1bfc009040 comp rx=0 tx=0).stop 2024-04-20T10:28:49.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c1006d020 msgr2=0x7f1c1014f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c1006d020 0x7f1c1014f8c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1c0402b8c0 tx=0x7f1c040046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 shutdown_connections 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c10145e90 0x7f1c10136f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c10070dc0 0x7f1c10135310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1bf006c8f0 0x7f1bf006edb0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 --2- 172.21.15.47:0/740983944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1c1006d020 0x7f1c1014f8c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 >> 172.21.15.47:0/740983944 conn(0x7f1c10070090 msgr2=0x7f1c100ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 shutdown_connections 2024-04-20T10:28:49.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.494+0000 7f1c16c00640 1 -- 172.21.15.47:0/740983944 wait complete. 2024-04-20T10:28:49.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.723+0000 7fef4b200640 1 Processor -- start 2024-04-20T10:28:49.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.723+0000 7fef4b200640 1 -- start start 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.724+0000 7fef4b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef4415af80 0x7fef44071a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef4b200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef4415b360 0x7fef4406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef4b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef4406f3b0 con 0x7fef440ca910 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef4b200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fef4409e150 con 0x7fef4415b360 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef4b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fef4406f180 con 0x7fef4415af80 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef49400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef4415b360 0x7fef4406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef49400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef4415b360 0x7fef4406ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35188/0 (socket says 172.21.15.47:35188) 2024-04-20T10:28:49.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef49400640 1 -- 172.21.15.47:0/3610561237 learned_addr learned my addr 172.21.15.47:0/3610561237 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.725+0000 7fef49e00640 1 --2- 172.21.15.47:0/3610561237 >> v2:172.21.15.175:3300/0 conn(0x7fef4415af80 0x7fef44071a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.726+0000 7fef49400640 1 -- 172.21.15.47:0/3610561237 >> v2:172.21.15.175:3300/0 conn(0x7fef4415af80 msgr2=0x7fef44071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.726+0000 7fef49400640 1 --2- 172.21.15.47:0/3610561237 >> v2:172.21.15.175:3300/0 conn(0x7fef4415af80 0x7fef44071a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.726+0000 7fef49400640 1 -- 172.21.15.47:0/3610561237 >> v1:172.21.15.175:6789/0 conn(0x7fef440ca910 legacy=0x7fef440cae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:49.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.726+0000 7fef49400640 1 -- 172.21.15.47:0/3610561237 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef4406f690 con 0x7fef4415b360 2024-04-20T10:28:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.727+0000 7fef49400640 1 --2- 172.21.15.47:0/3610561237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef4415b360 0x7fef4406ec40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef4406d680 tx=0x7fef3402e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02f3fd0fb53dbdc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.727+0000 7fef48a00640 1 -- 172.21.15.47:0/3610561237 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3403b070 con 0x7fef4415b360 2024-04-20T10:28:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.727+0000 7fef48a00640 1 -- 172.21.15.47:0/3610561237 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef3402ee00 con 0x7fef4415b360 2024-04-20T10:28:49.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.727+0000 7fef48a00640 1 -- 172.21.15.47:0/3610561237 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef34036660 con 0x7fef4415b360 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 -- 172.21.15.47:0/3610561237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef4415b360 msgr2=0x7fef4406ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 --2- 172.21.15.47:0/3610561237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef4415b360 0x7fef4406ec40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fef4406d680 tx=0x7fef3402e050 comp rx=0 tx=0).stop 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 -- 172.21.15.47:0/3610561237 shutdown_connections 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 --2- 172.21.15.47:0/3610561237 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef4415b360 0x7fef4406ec40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 --2- 172.21.15.47:0/3610561237 >> v2:172.21.15.175:3300/0 conn(0x7fef4415af80 0x7fef44071a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 -- 172.21.15.47:0/3610561237 >> 172.21.15.47:0/3610561237 conn(0x7fef440cffa0 msgr2=0x7fef440d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 -- 172.21.15.47:0/3610561237 shutdown_connections 2024-04-20T10:28:49.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.728+0000 7fef4b200640 1 -- 172.21.15.47:0/3610561237 wait complete. 2024-04-20T10:28:49.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.730+0000 7fef4b200640 1 Processor -- start 2024-04-20T10:28:49.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.730+0000 7fef4b200640 1 -- start start 2024-04-20T10:28:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef4b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef440ca910 0x7fef4409ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef4b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4415af80 0x7fef44153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef4b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef4415b360 0x7fef440ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef4b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef440ad480 con 0x7fef440ca910 2024-04-20T10:28:49.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef4b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef440acf50 con 0x7fef4415af80 2024-04-20T10:28:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef4b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef440cb570 con 0x7fef4415b360 2024-04-20T10:28:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef49e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef440ca910 0x7fef4409ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef49e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef440ca910 0x7fef4409ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35190/0 (socket says 172.21.15.47:35190) 2024-04-20T10:28:49.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.731+0000 7fef49e00640 1 -- 172.21.15.47:0/1071691985 learned_addr learned my addr 172.21.15.47:0/1071691985 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef4a800640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef4415b360 0x7fef440ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef49e00640 1 -- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef4415b360 msgr2=0x7fef440ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef49e00640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef4415b360 0x7fef440ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef49e00640 1 -- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4415af80 msgr2=0x7fef44153950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef49e00640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4415af80 0x7fef44153950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef49e00640 1 -- 172.21.15.47:0/1071691985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef44153e90 con 0x7fef440ca910 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef4a800640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef4415b360 0x7fef440ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef49e00640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef440ca910 0x7fef4409ada0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d800 tx=0x7fef3800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef3e000640 1 -- 172.21.15.47:0/1071691985 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3800b6c0 con 0x7fef440ca910 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef3e000640 1 -- 172.21.15.47:0/1071691985 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef38019070 con 0x7fef440ca910 2024-04-20T10:28:49.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.732+0000 7fef3e000640 1 -- 172.21.15.47:0/1071691985 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3800ba80 con 0x7fef440ca910 2024-04-20T10:28:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.733+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef44154140 con 0x7fef440ca910 2024-04-20T10:28:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.733+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef440ce0b0 con 0x7fef440ca910 2024-04-20T10:28:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.734+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef44070370 con 0x7fef440ca910 2024-04-20T10:28:49.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.735+0000 7fef3e000640 1 -- 172.21.15.47:0/1071691985 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef3800bc20 con 0x7fef440ca910 2024-04-20T10:28:49.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.736+0000 7fef3e000640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef2806c920 0x7fef2806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:49.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.736+0000 7fef3e000640 1 -- 172.21.15.47:0/1071691985 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fef3808c3a0 con 0x7fef440ca910 2024-04-20T10:28:49.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.736+0000 7fef49400640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef2806c920 0x7fef2806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:49.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.738+0000 7fef49400640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef2806c920 0x7fef2806ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef34038000 tx=0x7fef34030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:49.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.740+0000 7fef3e000640 1 -- 172.21.15.47:0/1071691985 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef44070370 con 0x7fef440ca910 2024-04-20T10:28:49.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.943+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fef44075b30 con 0x7fef440ca910 2024-04-20T10:28:49.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.944+0000 7fef3e000640 1 -- 172.21.15.47:0/1071691985 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fef3801c360 con 0x7fef440ca910 2024-04-20T10:28:49.944 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef2806c920 msgr2=0x7fef2806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef2806c920 0x7fef2806ede0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fef34038000 tx=0x7fef34030000 comp rx=0 tx=0).stop 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef440ca910 msgr2=0x7fef4409ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef440ca910 0x7fef4409ada0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fef3800d800 tx=0x7fef3800dcc0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 shutdown_connections 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef4415b360 0x7fef440ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef4415af80 0x7fef44153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef2806c920 0x7fef2806ede0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 --2- 172.21.15.47:0/1071691985 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef440ca910 0x7fef4409ada0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 >> 172.21.15.47:0/1071691985 conn(0x7fef440cffa0 msgr2=0x7fef44072b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 shutdown_connections 2024-04-20T10:28:49.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:49.947+0000 7fef4b200640 1 -- 172.21.15.47:0/1071691985 wait complete. 2024-04-20T10:28:49.965 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:50.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.179+0000 7f6406a00640 1 Processor -- start 2024-04-20T10:28:50.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.179+0000 7f6406a00640 1 -- start start 2024-04-20T10:28:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6406a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64000aee60 0x7f64000af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6406a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f64000ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6406a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f64000af780 con 0x7f640015fe10 2024-04-20T10:28:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6406a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f640006c510 con 0x7f64000aee60 2024-04-20T10:28:50.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6406a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f640006b900 con 0x7f640015d4f0 2024-04-20T10:28:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6405600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64000aee60 0x7f64000af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6405600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f64000aee60 0x7f64000af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35192/0 (socket says 172.21.15.47:35192) 2024-04-20T10:28:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.181+0000 7f6405600640 1 -- 172.21.15.47:0/3891840681 learned_addr learned my addr 172.21.15.47:0/3891840681 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.182+0000 7f6404c00640 1 --2- 172.21.15.47:0/3891840681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f64000ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.182+0000 7f6405600640 1 -- 172.21.15.47:0/3891840681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 msgr2=0x7f64000ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:50.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.182+0000 7f6405600640 1 --2- 172.21.15.47:0/3891840681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f64000ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.182+0000 7f6405600640 1 -- 172.21.15.47:0/3891840681 >> v1:172.21.15.175:6789/0 conn(0x7f640015fe10 legacy=0x7f6400160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.182+0000 7f6405600640 1 -- 172.21.15.47:0/3891840681 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640006bbe0 con 0x7f64000aee60 2024-04-20T10:28:50.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.184+0000 7f6405600640 1 --2- 172.21.15.47:0/3891840681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64000aee60 0x7f64000af240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009870 tx=0x7f63f402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f79a2abc7672a01a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.184+0000 7f63ffe00640 1 -- 172.21.15.47:0/3891840681 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4034070 con 0x7f64000aee60 2024-04-20T10:28:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.184+0000 7f63ffe00640 1 -- 172.21.15.47:0/3891840681 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f4030840 con 0x7f64000aee60 2024-04-20T10:28:50.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.184+0000 7f63ffe00640 1 -- 172.21.15.47:0/3891840681 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f4030c10 con 0x7f64000aee60 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 -- 172.21.15.47:0/3891840681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64000aee60 msgr2=0x7f64000af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 --2- 172.21.15.47:0/3891840681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64000aee60 0x7f64000af240 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009870 tx=0x7f63f402b980 comp rx=0 tx=0).stop 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 -- 172.21.15.47:0/3891840681 shutdown_connections 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 --2- 172.21.15.47:0/3891840681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f64000ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 --2- 172.21.15.47:0/3891840681 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64000aee60 0x7f64000af240 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 -- 172.21.15.47:0/3891840681 >> 172.21.15.47:0/3891840681 conn(0x7f64000b5270 msgr2=0x7f64000b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 -- 172.21.15.47:0/3891840681 shutdown_connections 2024-04-20T10:28:50.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.185+0000 7f6406a00640 1 -- 172.21.15.47:0/3891840681 wait complete. 2024-04-20T10:28:50.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.187+0000 7f6406a00640 1 Processor -- start 2024-04-20T10:28:50.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.187+0000 7f6406a00640 1 -- start start 2024-04-20T10:28:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64000aee60 0x7f640015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f640015b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f640015fe10 0x7f64001f4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f640006f070 con 0x7f640015fe10 2024-04-20T10:28:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f640006c1a0 con 0x7f64000aee60 2024-04-20T10:28:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f640006c9a0 con 0x7f640015d4f0 2024-04-20T10:28:50.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f640015fe10 0x7f64001f4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f640015fe10 0x7f64001f4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35202/0 (socket says 172.21.15.47:35202) 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 -- 172.21.15.47:0/3075038889 learned_addr learned my addr 172.21.15.47:0/3075038889 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6405600640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64000aee60 0x7f640015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 -- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 msgr2=0x7f640015b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6404c00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f640015b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f640015b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 -- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64000aee60 msgr2=0x7f640015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64000aee60 0x7f640015b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.188+0000 7f6406000640 1 -- 172.21.15.47:0/3075038889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6400151050 con 0x7f640015fe10 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f6405600640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64000aee60 0x7f640015b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f6404c00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f640015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f6406000640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f640015fe10 0x7f64001f4d70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c7d0 tx=0x7f63f800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f63fd600640 1 -- 172.21.15.47:0/3075038889 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8003b70 con 0x7f640015fe10 2024-04-20T10:28:50.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f63fd600640 1 -- 172.21.15.47:0/3075038889 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63f8003d30 con 0x7f640015fe10 2024-04-20T10:28:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f63fd600640 1 -- 172.21.15.47:0/3075038889 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800aa90 con 0x7f640015fe10 2024-04-20T10:28:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6400151330 con 0x7f640015fe10 2024-04-20T10:28:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.189+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6400151e10 con 0x7f640015fe10 2024-04-20T10:28:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.190+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6400075b30 con 0x7f640015fe10 2024-04-20T10:28:50.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.192+0000 7f63fd600640 1 -- 172.21.15.47:0/3075038889 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63f800ac30 con 0x7f640015fe10 2024-04-20T10:28:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.192+0000 7f63fd600640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63e006c8f0 0x7f63e006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.193+0000 7f63fd600640 1 -- 172.21.15.47:0/3075038889 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f63f808be80 con 0x7f640015fe10 2024-04-20T10:28:50.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.193+0000 7f6405600640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63e006c8f0 0x7f63e006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.194+0000 7f6405600640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63e006c8f0 0x7f63e006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63f4036000 tx=0x7f63f403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:50.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.197+0000 7f63fd600640 1 -- 172.21.15.47:0/3075038889 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f63f8059e60 con 0x7f640015fe10 2024-04-20T10:28:50.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.399+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6400153120 con 0x7f640015fe10 2024-04-20T10:28:50.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.400+0000 7f63fd600640 1 -- 172.21.15.47:0/3075038889 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f63f8059760 con 0x7f640015fe10 2024-04-20T10:28:50.400 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.402+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63e006c8f0 msgr2=0x7f63e006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.402+0000 7f6406a00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63e006c8f0 0x7f63e006edb0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f63f4036000 tx=0x7f63f403b040 comp rx=0 tx=0).stop 2024-04-20T10:28:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.402+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f640015fe10 msgr2=0x7f64001f4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.402+0000 7f6406a00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f640015fe10 0x7f64001f4d70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f63f800c7d0 tx=0x7f63f800cc90 comp rx=0 tx=0).stop 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 shutdown_connections 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63e006c8f0 0x7f63e006edb0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f640015fe10 0x7f64001f4d70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f640015d4f0 0x7f640015b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 --2- 172.21.15.47:0/3075038889 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f64000aee60 0x7f640015b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 >> 172.21.15.47:0/3075038889 conn(0x7f64000b5270 msgr2=0x7f6400160270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 shutdown_connections 2024-04-20T10:28:50.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.403+0000 7f6406a00640 1 -- 172.21.15.47:0/3075038889 wait complete. 2024-04-20T10:28:50.423 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:50.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.640+0000 7f6e5d800640 1 Processor -- start 2024-04-20T10:28:50.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.641+0000 7f6e5d800640 1 -- start start 2024-04-20T10:28:50.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.642+0000 7f6e5d800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e58159e10 0x7f6e58071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.642+0000 7f6e5d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e5815a1f0 0x7f6e58071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.642+0000 7f6e5d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5806da90 0x7f6e58160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.642+0000 7f6e5d800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6e58072900 con 0x7f6e5815a1f0 2024-04-20T10:28:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.642+0000 7f6e5d800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e5815ca40 con 0x7f6e58159e10 2024-04-20T10:28:50.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.642+0000 7f6e5d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5815cc90 con 0x7f6e5806da90 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.643+0000 7f6e4fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e5815a1f0 0x7f6e58071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.643+0000 7f6e4fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6e5815a1f0 0x7f6e58071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35216/0 (socket says 172.21.15.47:35216) 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.643+0000 7f6e4fe00640 1 -- 172.21.15.47:0/1559957801 learned_addr learned my addr 172.21.15.47:0/1559957801 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.643+0000 7f6e57e00640 1 --2- 172.21.15.47:0/1559957801 >> v2:172.21.15.175:3300/0 conn(0x7f6e58159e10 0x7f6e58071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.643+0000 7f6e5ce00640 1 --2- 172.21.15.47:0/1559957801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5806da90 0x7f6e58160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.643+0000 7f6e4fe00640 1 -- 172.21.15.47:0/1559957801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5806da90 msgr2=0x7f6e58160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.644+0000 7f6e4fe00640 1 --2- 172.21.15.47:0/1559957801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5806da90 0x7f6e58160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.644+0000 7f6e4fe00640 1 -- 172.21.15.47:0/1559957801 >> v2:172.21.15.175:3300/0 conn(0x7f6e58159e10 msgr2=0x7f6e58071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:50.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.644+0000 7f6e4fe00640 1 --2- 172.21.15.47:0/1559957801 >> v2:172.21.15.175:3300/0 conn(0x7f6e58159e10 0x7f6e58071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.644+0000 7f6e4fe00640 1 -- 172.21.15.47:0/1559957801 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5815cf70 con 0x7f6e5815a1f0 2024-04-20T10:28:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.645+0000 7f6e4fe00640 1 --2- 172.21.15.47:0/1559957801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5815a1f0 0x7f6e58071fc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6e58073f30 tx=0x7f6e4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f316b575e36fa198 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.645+0000 7f6e57400640 1 -- 172.21.15.47:0/1559957801 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40035020 con 0x7f6e5815a1f0 2024-04-20T10:28:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.645+0000 7f6e57400640 1 -- 172.21.15.47:0/1559957801 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e40031870 con 0x7f6e5815a1f0 2024-04-20T10:28:50.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.645+0000 7f6e57400640 1 -- 172.21.15.47:0/1559957801 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40031c40 con 0x7f6e5815a1f0 2024-04-20T10:28:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 -- 172.21.15.47:0/1559957801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5815a1f0 msgr2=0x7f6e58071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1559957801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5815a1f0 0x7f6e58071fc0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6e58073f30 tx=0x7f6e4002b910 comp rx=0 tx=0).stop 2024-04-20T10:28:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 -- 172.21.15.47:0/1559957801 shutdown_connections 2024-04-20T10:28:50.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1559957801 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e5806da90 0x7f6e58160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1559957801 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5815a1f0 0x7f6e58071fc0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1559957801 >> v2:172.21.15.175:3300/0 conn(0x7f6e58159e10 0x7f6e58071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 -- 172.21.15.47:0/1559957801 >> 172.21.15.47:0/1559957801 conn(0x7f6e580d0040 msgr2=0x7f6e580d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 -- 172.21.15.47:0/1559957801 shutdown_connections 2024-04-20T10:28:50.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.646+0000 7f6e5d800640 1 -- 172.21.15.47:0/1559957801 wait complete. 2024-04-20T10:28:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.648+0000 7f6e5d800640 1 Processor -- start 2024-04-20T10:28:50.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.648+0000 7f6e5d800640 1 -- start start 2024-04-20T10:28:50.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e5d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5806da90 0x7f6e5814bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e5d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e58159e10 0x7f6e5815c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e5d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5815a1f0 0x7f6e5814b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e5d800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5815d810 con 0x7f6e5806da90 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e5d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5815d530 con 0x7f6e5815a1f0 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e5d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e5806b900 con 0x7f6e58159e10 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e57e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5806da90 0x7f6e5814bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e57e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5806da90 0x7f6e5814bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35226/0 (socket says 172.21.15.47:35226) 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.649+0000 7f6e57e00640 1 -- 172.21.15.47:0/1124426708 learned_addr learned my addr 172.21.15.47:0/1124426708 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e57e00640 1 -- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e58159e10 msgr2=0x7f6e5815c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e57e00640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e58159e10 0x7f6e5815c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e57e00640 1 -- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5815a1f0 msgr2=0x7f6e5814b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e57e00640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5815a1f0 0x7f6e5814b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e57e00640 1 -- 172.21.15.47:0/1124426708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5809cf80 con 0x7f6e5806da90 2024-04-20T10:28:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e57e00640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5806da90 0x7f6e5814bb50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d800 tx=0x7f6e4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e54c00640 1 -- 172.21.15.47:0/1124426708 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800b6c0 con 0x7f6e5806da90 2024-04-20T10:28:50.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e54c00640 1 -- 172.21.15.47:0/1124426708 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e48019070 con 0x7f6e5806da90 2024-04-20T10:28:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5809d260 con 0x7f6e5806da90 2024-04-20T10:28:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.650+0000 7f6e54c00640 1 -- 172.21.15.47:0/1124426708 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4800ba80 con 0x7f6e5806da90 2024-04-20T10:28:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.651+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e5809d820 con 0x7f6e5806da90 2024-04-20T10:28:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.652+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e58075b60 con 0x7f6e5806da90 2024-04-20T10:28:50.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.653+0000 7f6e54c00640 1 -- 172.21.15.47:0/1124426708 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e4800bc20 con 0x7f6e5806da90 2024-04-20T10:28:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.653+0000 7f6e54c00640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e3406c8f0 0x7f6e3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:50.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.654+0000 7f6e54c00640 1 -- 172.21.15.47:0/1124426708 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f6e4808c450 con 0x7f6e5806da90 2024-04-20T10:28:50.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.654+0000 7f6e4fe00640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e3406c8f0 0x7f6e3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.655+0000 7f6e4fe00640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e3406c8f0 0x7f6e3406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036000 tx=0x7f6e40030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:50.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.658+0000 7f6e54c00640 1 -- 172.21.15.47:0/1124426708 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e4805a4c0 con 0x7f6e5806da90 2024-04-20T10:28:50.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.877+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f6e58155990 con 0x7f6e5806da90 2024-04-20T10:28:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.878+0000 7f6e54c00640 1 -- 172.21.15.47:0/1124426708 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f6e48059dc0 con 0x7f6e5806da90 2024-04-20T10:28:50.878 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.880+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e3406c8f0 msgr2=0x7f6e3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.880+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e3406c8f0 0x7f6e3406edb0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e40036000 tx=0x7f6e40030000 comp rx=0 tx=0).stop 2024-04-20T10:28:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.880+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5806da90 msgr2=0x7f6e5814bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.880+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5806da90 0x7f6e5814bb50 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800d800 tx=0x7f6e4800dcc0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 shutdown_connections 2024-04-20T10:28:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e5815a1f0 0x7f6e5814b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e58159e10 0x7f6e5815c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e3406c8f0 0x7f6e3406edb0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 --2- 172.21.15.47:0/1124426708 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6e5806da90 0x7f6e5814bb50 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 >> 172.21.15.47:0/1124426708 conn(0x7f6e580d0040 msgr2=0x7f6e580b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 shutdown_connections 2024-04-20T10:28:50.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:50.881+0000 7f6e5d800640 1 -- 172.21.15.47:0/1124426708 wait complete. 2024-04-20T10:28:50.902 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:28:50.902 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:50.959 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:51.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.170+0000 7fa787200640 1 Processor -- start 2024-04-20T10:28:51.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.171+0000 7fa787200640 1 -- start start 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.172+0000 7fa787200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa78806cde0 0x7fa7880d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.172+0000 7fa787200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7880b3330 0x7fa7880b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.172+0000 7fa787200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa78806c190 con 0x7fa788161d50 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.172+0000 7fa787200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa788162130 con 0x7fa7880b3330 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.172+0000 7fa787200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa78806c470 con 0x7fa78806cde0 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa786800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7880b3330 0x7fa7880b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa786800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa7880b3330 0x7fa7880b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35228/0 (socket says 172.21.15.47:35228) 2024-04-20T10:28:51.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa786800640 1 -- 172.21.15.47:0/2485394287 learned_addr learned my addr 172.21.15.47:0/2485394287 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:51.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa785400640 1 --2- 172.21.15.47:0/2485394287 >> v2:172.21.15.175:3300/0 conn(0x7fa78806cde0 0x7fa7880d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa786800640 1 -- 172.21.15.47:0/2485394287 >> v2:172.21.15.175:3300/0 conn(0x7fa78806cde0 msgr2=0x7fa7880d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:51.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa786800640 1 --2- 172.21.15.47:0/2485394287 >> v2:172.21.15.175:3300/0 conn(0x7fa78806cde0 0x7fa7880d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa786800640 1 -- 172.21.15.47:0/2485394287 >> v1:172.21.15.175:6789/0 conn(0x7fa788161d50 legacy=0x7fa78806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:51.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.173+0000 7fa786800640 1 -- 172.21.15.47:0/2485394287 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7880d2e00 con 0x7fa7880b3330 2024-04-20T10:28:51.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.174+0000 7fa786800640 1 --2- 172.21.15.47:0/2485394287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7880b3330 0x7fa7880b3880 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa7880d1f00 tx=0x7fa78002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606841df37d9d298 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:51.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa784a00640 1 -- 172.21.15.47:0/2485394287 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78002ca80 con 0x7fa7880b3330 2024-04-20T10:28:51.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa784a00640 1 -- 172.21.15.47:0/2485394287 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78002cc20 con 0x7fa7880b3330 2024-04-20T10:28:51.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa784a00640 1 -- 172.21.15.47:0/2485394287 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7800343f0 con 0x7fa7880b3330 2024-04-20T10:28:51.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 -- 172.21.15.47:0/2485394287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7880b3330 msgr2=0x7fa7880b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 --2- 172.21.15.47:0/2485394287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7880b3330 0x7fa7880b3880 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa7880d1f00 tx=0x7fa78002bee0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 -- 172.21.15.47:0/2485394287 shutdown_connections 2024-04-20T10:28:51.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 --2- 172.21.15.47:0/2485394287 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7880b3330 0x7fa7880b3880 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 --2- 172.21.15.47:0/2485394287 >> v2:172.21.15.175:3300/0 conn(0x7fa78806cde0 0x7fa7880d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 -- 172.21.15.47:0/2485394287 >> 172.21.15.47:0/2485394287 conn(0x7fa7880d4450 msgr2=0x7fa7880d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:51.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 -- 172.21.15.47:0/2485394287 shutdown_connections 2024-04-20T10:28:51.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.175+0000 7fa787200640 1 -- 172.21.15.47:0/2485394287 wait complete. 2024-04-20T10:28:51.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.177+0000 7fa787200640 1 Processor -- start 2024-04-20T10:28:51.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.177+0000 7fa787200640 1 -- start start 2024-04-20T10:28:51.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.178+0000 7fa787200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa78806cde0 0x7fa7881f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.178+0000 7fa787200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7880b3330 0x7fa78815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.178+0000 7fa787200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa788161d50 0x7fa78815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.178+0000 7fa787200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa7880b7660 con 0x7fa788161d50 2024-04-20T10:28:51.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.178+0000 7fa787200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7880d33c0 con 0x7fa7880b3330 2024-04-20T10:28:51.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.178+0000 7fa787200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7880b77e0 con 0x7fa78806cde0 2024-04-20T10:28:51.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa788161d50 0x7fa78815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa788161d50 0x7fa78815b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35230/0 (socket says 172.21.15.47:35230) 2024-04-20T10:28:51.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 -- 172.21.15.47:0/1452235937 learned_addr learned my addr 172.21.15.47:0/1452235937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa785400640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7880b3330 0x7fa78815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa785e00640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa78806cde0 0x7fa7881f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 -- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa78806cde0 msgr2=0x7fa7881f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa78806cde0 0x7fa7881f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 -- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7880b3330 msgr2=0x7fa78815b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7880b3330 0x7fa78815b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 -- 172.21.15.47:0/1452235937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7881f52d0 con 0x7fa788161d50 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.179+0000 7fa786800640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa788161d50 0x7fa78815b570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa780036040 tx=0x7fa780004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.180+0000 7fa77e000640 1 -- 172.21.15.47:0/1452235937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780043020 con 0x7fa788161d50 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.180+0000 7fa77e000640 1 -- 172.21.15.47:0/1452235937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa78003e450 con 0x7fa788161d50 2024-04-20T10:28:51.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.180+0000 7fa77e000640 1 -- 172.21.15.47:0/1452235937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa780009510 con 0x7fa788161d50 2024-04-20T10:28:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.180+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7881f4510 con 0x7fa788161d50 2024-04-20T10:28:51.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.180+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7881f4ad0 con 0x7fa788161d50 2024-04-20T10:28:51.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.181+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa788075b60 con 0x7fa788161d50 2024-04-20T10:28:51.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.182+0000 7fa77e000640 1 -- 172.21.15.47:0/1452235937 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa780046020 con 0x7fa788161d50 2024-04-20T10:28:51.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.183+0000 7fa77e000640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76806c8f0 0x7fa76806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.183+0000 7fa77e000640 1 -- 172.21.15.47:0/1452235937 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fa7800ae780 con 0x7fa788161d50 2024-04-20T10:28:51.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.184+0000 7fa785e00640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76806c8f0 0x7fa76806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.185+0000 7fa785e00640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76806c8f0 0x7fa76806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa778006fd0 tx=0x7fa778008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:51.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.188+0000 7fa77e000640 1 -- 172.21.15.47:0/1452235937 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa78004d030 con 0x7fa788161d50 2024-04-20T10:28:51.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.394+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa7880cb2e0 con 0x7fa788161d50 2024-04-20T10:28:51.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.397+0000 7fa77e000640 1 -- 172.21.15.47:0/1452235937 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7fa78007c800 con 0x7fa788161d50 2024-04-20T10:28:51.398 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:51.399 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:51.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76806c8f0 msgr2=0x7fa76806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76806c8f0 0x7fa76806edb0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa778006fd0 tx=0x7fa778008040 comp rx=0 tx=0).stop 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa788161d50 msgr2=0x7fa78815b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa788161d50 0x7fa78815b570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa780036040 tx=0x7fa780004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 shutdown_connections 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa76806c8f0 0x7fa76806edb0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa788161d50 0x7fa78815b570 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7880b3330 0x7fa78815b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 --2- 172.21.15.47:0/1452235937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa78806cde0 0x7fa7881f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 >> 172.21.15.47:0/1452235937 conn(0x7fa7880d4450 msgr2=0x7fa7880676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 shutdown_connections 2024-04-20T10:28:51.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.400+0000 7fa787200640 1 -- 172.21.15.47:0/1452235937 wait complete. 2024-04-20T10:28:51.420 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:51.420 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:51.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.627+0000 7f6955c00640 1 Processor -- start 2024-04-20T10:28:51.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.628+0000 7f6955c00640 1 -- start start 2024-04-20T10:28:51.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.629+0000 7f6955c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f695015a130 0x7f6950073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.629+0000 7f6955c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f695006f870 0x7f69500739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.629+0000 7f6955c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f695015cc20 con 0x7f6950071a60 2024-04-20T10:28:51.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.629+0000 7f6955c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f695015d7f0 con 0x7f695015a130 2024-04-20T10:28:51.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.629+0000 7f6955c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f695015cec0 con 0x7f695006f870 2024-04-20T10:28:51.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f694fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f695015a130 0x7f6950073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f694fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f695015a130 0x7f6950073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35244/0 (socket says 172.21.15.47:35244) 2024-04-20T10:28:51.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f694fe00640 1 -- 172.21.15.47:0/701907347 learned_addr learned my addr 172.21.15.47:0/701907347 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f694fe00640 1 -- 172.21.15.47:0/701907347 >> v2:172.21.15.175:3300/0 conn(0x7f695006f870 msgr2=0x7f69500739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f6955200640 1 --2- 172.21.15.47:0/701907347 >> v2:172.21.15.175:3300/0 conn(0x7f695006f870 0x7f69500739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f694fe00640 1 --2- 172.21.15.47:0/701907347 >> v2:172.21.15.175:3300/0 conn(0x7f695006f870 0x7f69500739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f694fe00640 1 -- 172.21.15.47:0/701907347 >> v1:172.21.15.175:6789/0 conn(0x7f6950071a60 legacy=0x7f695006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.630+0000 7f694fe00640 1 -- 172.21.15.47:0/701907347 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695015d1a0 con 0x7f695015a130 2024-04-20T10:28:51.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.631+0000 7f694fe00640 1 --2- 172.21.15.47:0/701907347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f695015a130 0x7f6950073470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69500d0f90 tx=0x7f694002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45de1862d8aec23c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.631+0000 7f694ea00640 1 -- 172.21.15.47:0/701907347 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694002c960 con 0x7f695015a130 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f694ea00640 1 -- 172.21.15.47:0/701907347 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f694002cb00 con 0x7f695015a130 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f694ea00640 1 -- 172.21.15.47:0/701907347 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69400343f0 con 0x7f695015a130 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 -- 172.21.15.47:0/701907347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f695015a130 msgr2=0x7f6950073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 --2- 172.21.15.47:0/701907347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f695015a130 0x7f6950073470 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f69500d0f90 tx=0x7f694002bdc0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 -- 172.21.15.47:0/701907347 shutdown_connections 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 --2- 172.21.15.47:0/701907347 >> v2:172.21.15.175:3300/0 conn(0x7f695006f870 0x7f69500739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 --2- 172.21.15.47:0/701907347 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f695015a130 0x7f6950073470 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 -- 172.21.15.47:0/701907347 >> 172.21.15.47:0/701907347 conn(0x7f69500cffa0 msgr2=0x7f69500d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 -- 172.21.15.47:0/701907347 shutdown_connections 2024-04-20T10:28:51.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.632+0000 7f6955c00640 1 -- 172.21.15.47:0/701907347 wait complete. 2024-04-20T10:28:51.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.634+0000 7f6955c00640 1 Processor -- start 2024-04-20T10:28:51.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.634+0000 7f6955c00640 1 -- start start 2024-04-20T10:28:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.635+0000 7f6955c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695006f870 0x7f6950155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.635+0000 7f6955c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6950071a60 0x7f6950157ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.635+0000 7f6955c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695015a130 0x7f6950156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.635+0000 7f6955c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f695006ba80 con 0x7f6950071a60 2024-04-20T10:28:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.635+0000 7f6955c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f695006b900 con 0x7f695006f870 2024-04-20T10:28:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.635+0000 7f6955c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f695006bc00 con 0x7f695015a130 2024-04-20T10:28:51.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.635+0000 7f694f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6950071a60 0x7f6950157ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6950071a60 0x7f6950157ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35260/0 (socket says 172.21.15.47:35260) 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 -- 172.21.15.47:0/3274789661 learned_addr learned my addr 172.21.15.47:0/3274789661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694fe00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695006f870 0x7f6950155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 -- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695015a130 msgr2=0x7f6950156560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695015a130 0x7f6950156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 -- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695006f870 msgr2=0x7f6950155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695006f870 0x7f6950155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 -- 172.21.15.47:0/3274789661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69501560c0 con 0x7f6950071a60 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694fe00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695006f870 0x7f6950155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:51.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.636+0000 7f694f400640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6950071a60 0x7f6950157ee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f693800c410 tx=0x7f693800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.637+0000 7f6947e00640 1 -- 172.21.15.47:0/3274789661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938017070 con 0x7f6950071a60 2024-04-20T10:28:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.637+0000 7f6947e00640 1 -- 172.21.15.47:0/3274789661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69380035f0 con 0x7f6950071a60 2024-04-20T10:28:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.637+0000 7f6947e00640 1 -- 172.21.15.47:0/3274789661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6938003a20 con 0x7f6950071a60 2024-04-20T10:28:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.637+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6950156e40 con 0x7f6950071a60 2024-04-20T10:28:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.637+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6950157400 con 0x7f6950071a60 2024-04-20T10:28:51.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.638+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6950075b30 con 0x7f6950071a60 2024-04-20T10:28:51.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.639+0000 7f6947e00640 1 -- 172.21.15.47:0/3274789661 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6938003bc0 con 0x7f6950071a60 2024-04-20T10:28:51.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.640+0000 7f6947e00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f692c06c8f0 0x7f692c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:51.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.640+0000 7f6947e00640 1 -- 172.21.15.47:0/3274789661 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f693808b2c0 con 0x7f6950071a60 2024-04-20T10:28:51.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.641+0000 7f694fe00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f692c06c8f0 0x7f692c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:51.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.642+0000 7f694fe00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f692c06c8f0 0x7f692c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6940035000 tx=0x7f6940030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:51.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.645+0000 7f6947e00640 1 -- 172.21.15.47:0/3274789661 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f69501560c0 con 0x7f6950071a60 2024-04-20T10:28:51.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.851+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f695006d010 con 0x7f6950071a60 2024-04-20T10:28:51.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.852+0000 7f6947e00640 1 -- 172.21.15.47:0/3274789661 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f69380592a0 con 0x7f6950071a60 2024-04-20T10:28:51.852 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f692c06c8f0 msgr2=0x7f692c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f692c06c8f0 0x7f692c06edb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6940035000 tx=0x7f6940030000 comp rx=0 tx=0).stop 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6950071a60 msgr2=0x7f6950157ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6950071a60 0x7f6950157ee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f693800c410 tx=0x7f693800c8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 shutdown_connections 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695015a130 0x7f6950156560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f692c06c8f0 0x7f692c06edb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6950071a60 0x7f6950157ee0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 --2- 172.21.15.47:0/3274789661 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695006f870 0x7f6950155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:51.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 >> 172.21.15.47:0/3274789661 conn(0x7f69500cffa0 msgr2=0x7f695015f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:51.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 shutdown_connections 2024-04-20T10:28:51.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:51.855+0000 7f6955c00640 1 -- 172.21.15.47:0/3274789661 wait complete. 2024-04-20T10:28:51.875 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:52.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.092+0000 7fe07be00640 1 Processor -- start 2024-04-20T10:28:52.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.092+0000 7fe07be00640 1 -- start start 2024-04-20T10:28:52.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.093+0000 7fe07be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c15b1f0 0x7fe07c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe07c0d0010 0x7fe07c0adc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07be00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe07c15b5d0 con 0x7fe07c0ae140 2024-04-20T10:28:52.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe07c06e730 con 0x7fe07c0d0010 2024-04-20T10:28:52.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe07c0d05d0 con 0x7fe07c15b1f0 2024-04-20T10:28:52.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07b400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe07c0ae140 0x7fe07c06ca10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44624/0 (socket says 172.21.15.47:44624) 2024-04-20T10:28:52.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07b400640 1 -- 172.21.15.47:0/1019097881 learned_addr learned my addr 172.21.15.47:0/1019097881 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07aa00640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c15b1f0 0x7fe07c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe07a000640 1 --2- 172.21.15.47:0/1019097881 >> v2:172.21.15.175:3300/0 conn(0x7fe07c0d0010 0x7fe07c0adc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2565810482 0 0) 0x7fe07c15b5d0 con 0x7fe07c0ae140 2024-04-20T10:28:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.094+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe05c005730 con 0x7fe07c0ae140 2024-04-20T10:28:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3601601348 0 0) 0x7fe05c005730 con 0x7fe07c0ae140 2024-04-20T10:28:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c15b1f0 msgr2=0x7fe07c15f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:52.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c15b1f0 0x7fe07c15f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 >> v2:172.21.15.175:3300/0 conn(0x7fe07c0d0010 msgr2=0x7fe07c0adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 --2- 172.21.15.47:0/1019097881 >> v2:172.21.15.175:3300/0 conn(0x7fe07c0d0010 0x7fe07c0adc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c15fc70 con 0x7fe07c0ae140 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fe074001670 con 0x7fe07c0ae140 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 >> v1:172.21.15.47:6789/0 conn(0x7fe07c0ae140 legacy=0x7fe07c06ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c15b1f0 0x7fe05c005bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe05c006190 0x7fe05c006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe05c006f50 0x7fe05c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c06e730 con 0x7fe05c006f50 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0d05d0 con 0x7fe05c006190 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c00dbd0 con 0x7fe07c15b1f0 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.095+0000 7fe07b400640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe05c006f50 0x7fe05c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.096+0000 7fe07b400640 1 -- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c15b1f0 msgr2=0x7fe05c005bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.096+0000 7fe07b400640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c15b1f0 0x7fe05c005bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.096+0000 7fe07a000640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe05c006190 0x7fe05c006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.096+0000 7fe07b400640 1 -- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe05c006190 msgr2=0x7fe05c006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.096+0000 7fe07b400640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe05c006190 0x7fe05c006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.096+0000 7fe07b400640 1 -- 172.21.15.47:0/1019097881 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c005730 con 0x7fe05c006f50 2024-04-20T10:28:52.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.096+0000 7fe07a000640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe05c006190 0x7fe05c006a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:52.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.097+0000 7fe07b400640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe05c006f50 0x7fe05c0073d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c160 tx=0x7fe05c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85add40e17c4cab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.097+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074007c80 con 0x7fe05c006f50 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.097+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe074030070 con 0x7fe05c006f50 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.097+0000 7fe079600640 1 -- 172.21.15.47:0/1019097881 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe074005d70 con 0x7fe05c006f50 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.098+0000 7fe07be00640 1 -- 172.21.15.47:0/1019097881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe05c006f50 msgr2=0x7fe05c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.098+0000 7fe07be00640 1 --2- 172.21.15.47:0/1019097881 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe05c006f50 0x7fe05c0073d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe07400c160 tx=0x7fe05c007a40 comp rx=0 tx=0).stop 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.098+0000 7fe07b400640 1 -- 172.21.15.47:0/1019097881 reap_dead start 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.098+0000 7fe07be00640 1 -- 172.21.15.47:0/1019097881 shutdown_connections 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.098+0000 7fe07be00640 1 -- 172.21.15.47:0/1019097881 >> 172.21.15.47:0/1019097881 conn(0x7fe07c0b5270 msgr2=0x7fe07c0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.098+0000 7fe07be00640 1 -- 172.21.15.47:0/1019097881 shutdown_connections 2024-04-20T10:28:52.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.098+0000 7fe07be00640 1 -- 172.21.15.47:0/1019097881 wait complete. 2024-04-20T10:28:52.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.100+0000 7fe07be00640 1 Processor -- start 2024-04-20T10:28:52.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.101+0000 7fe07be00640 1 -- start start 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c1f6fc0 0x7fe07c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c1521f0 0x7fe07c154c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe07c1551a0 0x7fe07c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0d0e70 con 0x7fe07c1521f0 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c0d0b90 con 0x7fe07c1551a0 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe07c06d120 con 0x7fe07c1f6fc0 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c1521f0 0x7fe07c154c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c1521f0 0x7fe07c154c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35288/0 (socket says 172.21.15.47:35288) 2024-04-20T10:28:52.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 -- 172.21.15.47:0/4076807543 learned_addr learned my addr 172.21.15.47:0/4076807543 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07b400640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe07c1551a0 0x7fe07c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 -- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c1f6fc0 msgr2=0x7fe07c157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c1f6fc0 0x7fe07c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 -- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe07c1551a0 msgr2=0x7fe07c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe07c1551a0 0x7fe07c150110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.102+0000 7fe07a000640 1 -- 172.21.15.47:0/4076807543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c1581d0 con 0x7fe07c1521f0 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.103+0000 7fe07a000640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c1521f0 0x7fe07c154c60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe06400f400 tx=0x7fe06400f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.103+0000 7fe073400640 1 -- 172.21.15.47:0/4076807543 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064003990 con 0x7fe07c1521f0 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.103+0000 7fe073400640 1 -- 172.21.15.47:0/4076807543 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe064003b30 con 0x7fe07c1521f0 2024-04-20T10:28:52.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.103+0000 7fe073400640 1 -- 172.21.15.47:0/4076807543 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe064004020 con 0x7fe07c1521f0 2024-04-20T10:28:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.103+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c1584b0 con 0x7fe07c1521f0 2024-04-20T10:28:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.103+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c158a70 con 0x7fe07c1521f0 2024-04-20T10:28:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.104+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c075b30 con 0x7fe07c1521f0 2024-04-20T10:28:52.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.105+0000 7fe073400640 1 -- 172.21.15.47:0/4076807543 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe06401d020 con 0x7fe07c1521f0 2024-04-20T10:28:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.106+0000 7fe073400640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe05806c920 0x7fe05806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.106+0000 7fe073400640 1 -- 172.21.15.47:0/4076807543 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fe06408c940 con 0x7fe07c1521f0 2024-04-20T10:28:52.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.107+0000 7fe07aa00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe05806c920 0x7fe05806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.108+0000 7fe07aa00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe05806c920 0x7fe05806ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009a40 tx=0x7fe06c009500 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:52.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.111+0000 7fe073400640 1 -- 172.21.15.47:0/4076807543 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe06405a920 con 0x7fe07c1521f0 2024-04-20T10:28:52.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.316+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fe07c0ade10 con 0x7fe07c1521f0 2024-04-20T10:28:52.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.317+0000 7fe073400640 1 -- 172.21.15.47:0/4076807543 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7fe06405a220 con 0x7fe07c1521f0 2024-04-20T10:28:52.317 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe05806c920 msgr2=0x7fe05806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe05806c920 0x7fe05806ede0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009a40 tx=0x7fe06c009500 comp rx=0 tx=0).stop 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c1521f0 msgr2=0x7fe07c154c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c1521f0 0x7fe07c154c60 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe06400f400 tx=0x7fe06400f8c0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 shutdown_connections 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe07c1551a0 0x7fe07c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe05806c920 0x7fe05806ede0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.319+0000 7fe07be00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe07c1521f0 0x7fe07c154c60 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.320+0000 7fe07be00640 1 --2- 172.21.15.47:0/4076807543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe07c1f6fc0 0x7fe07c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.320+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 >> 172.21.15.47:0/4076807543 conn(0x7fe07c0b5270 msgr2=0x7fe07c0ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:52.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.320+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 shutdown_connections 2024-04-20T10:28:52.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.320+0000 7fe07be00640 1 -- 172.21.15.47:0/4076807543 wait complete. 2024-04-20T10:28:52.340 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:52.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.554+0000 7f33bca00640 1 Processor -- start 2024-04-20T10:28:52.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.554+0000 7f33bca00640 1 -- start start 2024-04-20T10:28:52.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.556+0000 7f33bca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f33b815a130 0x7f33b8073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.556+0000 7f33bca00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f33b806f870 0x7f33b80739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.556+0000 7f33bca00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f33b815cc20 con 0x7f33b8071a60 2024-04-20T10:28:52.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.556+0000 7f33bca00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f33b815d7f0 con 0x7f33b815a130 2024-04-20T10:28:52.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.556+0000 7f33bca00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f33b815cec0 con 0x7f33b806f870 2024-04-20T10:28:52.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33afe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f33b8071a60 0x7f33b806f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:44636/0 (socket says 172.21.15.47:44636) 2024-04-20T10:28:52.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33b6800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f33b815a130 0x7f33b8073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33afe00640 1 -- 172.21.15.47:0/1335322277 learned_addr learned my addr 172.21.15.47:0/1335322277 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:52.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33b7200640 1 --2- 172.21.15.47:0/1335322277 >> v2:172.21.15.175:3300/0 conn(0x7f33b806f870 0x7f33b80739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33b6800640 1 -- 172.21.15.47:0/1335322277 >> v2:172.21.15.175:3300/0 conn(0x7f33b806f870 msgr2=0x7f33b80739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:52.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33b6800640 1 --2- 172.21.15.47:0/1335322277 >> v2:172.21.15.175:3300/0 conn(0x7f33b806f870 0x7f33b80739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33b6800640 1 -- 172.21.15.47:0/1335322277 >> v1:172.21.15.47:6789/0 conn(0x7f33b8071a60 legacy=0x7f33b806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.557+0000 7f33b6800640 1 -- 172.21.15.47:0/1335322277 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b815d1a0 con 0x7f33b815a130 2024-04-20T10:28:52.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.558+0000 7f33b6800640 1 --2- 172.21.15.47:0/1335322277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b815a130 0x7f33b8073470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f33b80d0f90 tx=0x7f33a802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6163deb6c1d77065 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:52.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.559+0000 7f33b5e00640 1 -- 172.21.15.47:0/1335322277 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8038070 con 0x7f33b815a130 2024-04-20T10:28:52.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.559+0000 7f33b5e00640 1 -- 172.21.15.47:0/1335322277 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a8031520 con 0x7f33b815a130 2024-04-20T10:28:52.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.559+0000 7f33b5e00640 1 -- 172.21.15.47:0/1335322277 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a80318f0 con 0x7f33b815a130 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.559+0000 7f33bca00640 1 -- 172.21.15.47:0/1335322277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b815a130 msgr2=0x7f33b8073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.559+0000 7f33bca00640 1 --2- 172.21.15.47:0/1335322277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b815a130 0x7f33b8073470 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f33b80d0f90 tx=0x7f33a802b7c0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.560+0000 7f33bca00640 1 -- 172.21.15.47:0/1335322277 shutdown_connections 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.560+0000 7f33bca00640 1 --2- 172.21.15.47:0/1335322277 >> v2:172.21.15.175:3300/0 conn(0x7f33b806f870 0x7f33b80739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.560+0000 7f33bca00640 1 --2- 172.21.15.47:0/1335322277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b815a130 0x7f33b8073470 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.560+0000 7f33bca00640 1 -- 172.21.15.47:0/1335322277 >> 172.21.15.47:0/1335322277 conn(0x7f33b80cffa0 msgr2=0x7f33b80d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.560+0000 7f33bca00640 1 -- 172.21.15.47:0/1335322277 shutdown_connections 2024-04-20T10:28:52.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.560+0000 7f33bca00640 1 -- 172.21.15.47:0/1335322277 wait complete. 2024-04-20T10:28:52.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.562+0000 7f33bca00640 1 Processor -- start 2024-04-20T10:28:52.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.562+0000 7f33bca00640 1 -- start start 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33bca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b806f870 0x7f33b809a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33bca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b8071a60 0x7f33b8155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33bca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b815a130 0x7f33b80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33bca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f33b806ba80 con 0x7f33b8071a60 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33bca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33b806b900 con 0x7f33b815a130 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33bca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33b806bc00 con 0x7f33b806f870 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b8071a60 0x7f33b8155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33afe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b8071a60 0x7f33b8155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35312/0 (socket says 172.21.15.47:35312) 2024-04-20T10:28:52.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33afe00640 1 -- 172.21.15.47:0/3797429478 learned_addr learned my addr 172.21.15.47:0/3797429478 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:52.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.563+0000 7f33b6800640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b806f870 0x7f33b809a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33afe00640 1 -- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b806f870 msgr2=0x7f33b809a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33afe00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b806f870 0x7f33b809a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33afe00640 1 -- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b815a130 msgr2=0x7f33b80b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33afe00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b815a130 0x7f33b80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33afe00640 1 -- 172.21.15.47:0/3797429478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b80b42d0 con 0x7f33b8071a60 2024-04-20T10:28:52.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33b6800640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b806f870 0x7f33b809a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33afe00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b8071a60 0x7f33b8155bf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c430 tx=0x7f33a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33aea00640 1 -- 172.21.15.47:0/3797429478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a0017070 con 0x7f33b8071a60 2024-04-20T10:28:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33aea00640 1 -- 172.21.15.47:0/3797429478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f33a0005ce0 con 0x7f33b8071a60 2024-04-20T10:28:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b80cade0 con 0x7f33b8071a60 2024-04-20T10:28:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.564+0000 7f33aea00640 1 -- 172.21.15.47:0/3797429478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a00072f0 con 0x7f33b8071a60 2024-04-20T10:28:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.565+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b80cb3a0 con 0x7f33b8071a60 2024-04-20T10:28:52.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.566+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8075b30 con 0x7f33b8071a60 2024-04-20T10:28:52.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.567+0000 7f33aea00640 1 -- 172.21.15.47:0/3797429478 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f33a001a070 con 0x7f33b8071a60 2024-04-20T10:28:52.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.568+0000 7f33aea00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f339406c8f0 0x7f339406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:52.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.568+0000 7f33aea00640 1 -- 172.21.15.47:0/3797429478 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f33a008b3c0 con 0x7f33b8071a60 2024-04-20T10:28:52.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.569+0000 7f33b6800640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f339406c8f0 0x7f339406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:52.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.570+0000 7f33b6800640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f339406c8f0 0x7f339406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33a8035000 tx=0x7f33a8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:52.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.574+0000 7f33aea00640 1 -- 172.21.15.47:0/3797429478 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f33a00593a0 con 0x7f33b8071a60 2024-04-20T10:28:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.776+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f33b80cabc0 con 0x7f33b8071a60 2024-04-20T10:28:52.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.777+0000 7f33aea00640 1 -- 172.21.15.47:0/3797429478 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f33a0058ca0 con 0x7f33b8071a60 2024-04-20T10:28:52.777 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f339406c8f0 msgr2=0x7f339406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f339406c8f0 0x7f339406edb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f33a8035000 tx=0x7f33a8030000 comp rx=0 tx=0).stop 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b8071a60 msgr2=0x7f33b8155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b8071a60 0x7f33b8155bf0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c430 tx=0x7f33a000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 shutdown_connections 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b815a130 0x7f33b80b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f339406c8f0 0x7f339406edb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f33b8071a60 0x7f33b8155bf0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 --2- 172.21.15.47:0/3797429478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b806f870 0x7f33b809a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.779+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 >> 172.21.15.47:0/3797429478 conn(0x7f33b80cffa0 msgr2=0x7f33b815e3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.780+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 shutdown_connections 2024-04-20T10:28:52.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:52.780+0000 7f33bca00640 1 -- 172.21.15.47:0/3797429478 wait complete. 2024-04-20T10:28:52.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:53.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.024+0000 7f934a800640 1 Processor -- start 2024-04-20T10:28:53.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.024+0000 7f934a800640 1 -- start start 2024-04-20T10:28:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.026+0000 7f934a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93440cf510 0x7f934415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.026+0000 7f934a800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f934415fe20 0x7f93440cfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.026+0000 7f934a800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f934415fc20 con 0x7f934406ed60 2024-04-20T10:28:53.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.026+0000 7f934a800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f934406d800 con 0x7f934415fe20 2024-04-20T10:28:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.026+0000 7f934a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f934406dae0 con 0x7f93440cf510 2024-04-20T10:28:53.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.027+0000 7f9343400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93440cf510 0x7f934415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.027+0000 7f9343400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93440cf510 0x7f934415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:59398/0 (socket says 172.21.15.47:59398) 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.027+0000 7f9343400640 1 -- 172.21.15.47:0/3657740807 learned_addr learned my addr 172.21.15.47:0/3657740807 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.027+0000 7f9342a00640 1 --2- 172.21.15.47:0/3657740807 >> v2:172.21.15.175:3300/0 conn(0x7f934415fe20 0x7f93440cfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.028+0000 7f9343400640 1 -- 172.21.15.47:0/3657740807 >> v2:172.21.15.175:3300/0 conn(0x7f934415fe20 msgr2=0x7f93440cfff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.028+0000 7f9343400640 1 --2- 172.21.15.47:0/3657740807 >> v2:172.21.15.175:3300/0 conn(0x7f934415fe20 0x7f93440cfff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.028+0000 7f9343400640 1 -- 172.21.15.47:0/3657740807 >> v1:172.21.15.175:6789/0 conn(0x7f934406ed60 legacy=0x7f9344161240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.028+0000 7f9343400640 1 -- 172.21.15.47:0/3657740807 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f934406ddc0 con 0x7f93440cf510 2024-04-20T10:28:53.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.028+0000 7f9342a00640 1 --2- 172.21.15.47:0/3657740807 >> v2:172.21.15.175:3300/0 conn(0x7f934415fe20 0x7f93440cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:28:53.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.029+0000 7f9343400640 1 --2- 172.21.15.47:0/3657740807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93440cf510 0x7f934415f6e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9334009e50 tx=0x7f933402bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d00fc979b8accf87 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.030+0000 7f9342000640 1 -- 172.21.15.47:0/3657740807 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f933402cb00 con 0x7f93440cf510 2024-04-20T10:28:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.030+0000 7f9342000640 1 -- 172.21.15.47:0/3657740807 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f933402cca0 con 0x7f93440cf510 2024-04-20T10:28:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.030+0000 7f9342000640 1 -- 172.21.15.47:0/3657740807 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9334034610 con 0x7f93440cf510 2024-04-20T10:28:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.030+0000 7f934a800640 1 -- 172.21.15.47:0/3657740807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93440cf510 msgr2=0x7f934415f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.030+0000 7f934a800640 1 --2- 172.21.15.47:0/3657740807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93440cf510 0x7f934415f6e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9334009e50 tx=0x7f933402bf60 comp rx=0 tx=0).stop 2024-04-20T10:28:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.031+0000 7f934a800640 1 -- 172.21.15.47:0/3657740807 shutdown_connections 2024-04-20T10:28:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.031+0000 7f934a800640 1 --2- 172.21.15.47:0/3657740807 >> v2:172.21.15.175:3300/0 conn(0x7f934415fe20 0x7f93440cfff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.031+0000 7f934a800640 1 --2- 172.21.15.47:0/3657740807 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93440cf510 0x7f934415f6e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.031+0000 7f934a800640 1 -- 172.21.15.47:0/3657740807 >> 172.21.15.47:0/3657740807 conn(0x7f93440b3fb0 msgr2=0x7f93440b4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.031+0000 7f934a800640 1 -- 172.21.15.47:0/3657740807 shutdown_connections 2024-04-20T10:28:53.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.031+0000 7f934a800640 1 -- 172.21.15.47:0/3657740807 wait complete. 2024-04-20T10:28:53.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.033+0000 7f934a800640 1 Processor -- start 2024-04-20T10:28:53.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.033+0000 7f934a800640 1 -- start start 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.034+0000 7f934a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934406ed60 0x7f9344152500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f934a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93440cf510 0x7f9344162400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f934a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934415fe20 0x7f9344162940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f934a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93440d0530 con 0x7f93440cf510 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f934a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f934406e380 con 0x7f934406ed60 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f934a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93440d06b0 con 0x7f934415fe20 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9342a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93440cf510 0x7f9344162400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9342a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93440cf510 0x7f9344162400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35318/0 (socket says 172.21.15.47:35318) 2024-04-20T10:28:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9342a00640 1 -- 172.21.15.47:0/3532773846 learned_addr learned my addr 172.21.15.47:0/3532773846 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9343400640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934406ed60 0x7f9344152500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9343e00640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934415fe20 0x7f9344162940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9342a00640 1 -- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934415fe20 msgr2=0x7f9344162940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9342a00640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934415fe20 0x7f9344162940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.035+0000 7f9342a00640 1 -- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934406ed60 msgr2=0x7f9344152500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f9342a00640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934406ed60 0x7f9344152500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f9342a00640 1 -- 172.21.15.47:0/3532773846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93440c5b70 con 0x7f93440cf510 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f9342a00640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93440cf510 0x7f9344162400 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f932c00d800 tx=0x7f932c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f933b400640 1 -- 172.21.15.47:0/3532773846 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c004030 con 0x7f93440cf510 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f933b400640 1 -- 172.21.15.47:0/3532773846 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f932c0041d0 con 0x7f93440cf510 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f933b400640 1 -- 172.21.15.47:0/3532773846 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f932c0096a0 con 0x7f93440cf510 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93440c5e50 con 0x7f93440cf510 2024-04-20T10:28:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.036+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93440c6410 con 0x7f93440cf510 2024-04-20T10:28:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.038+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93440c9a40 con 0x7f93440cf510 2024-04-20T10:28:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.039+0000 7f933b400640 1 -- 172.21.15.47:0/3532773846 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f932c009840 con 0x7f93440cf510 2024-04-20T10:28:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.040+0000 7f933b400640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f932006c8f0 0x7f932006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.040+0000 7f933b400640 1 -- 172.21.15.47:0/3532773846 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f932c08c2a0 con 0x7f93440cf510 2024-04-20T10:28:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.040+0000 7f9343400640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f932006c8f0 0x7f932006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.042+0000 7f9343400640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f932006c8f0 0x7f932006edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9334035000 tx=0x7f9334030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.046+0000 7f933b400640 1 -- 172.21.15.47:0/3532773846 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93440c9a40 con 0x7f93440cf510 2024-04-20T10:28:53.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.260+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f9344069090 con 0x7f93440cf510 2024-04-20T10:28:53.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.261+0000 7f933b400640 1 -- 172.21.15.47:0/3532773846 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f932c05a280 con 0x7f93440cf510 2024-04-20T10:28:53.261 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:53.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.263+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f932006c8f0 msgr2=0x7f932006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.263+0000 7f934a800640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f932006c8f0 0x7f932006edb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9334035000 tx=0x7f9334030000 comp rx=0 tx=0).stop 2024-04-20T10:28:53.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.263+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93440cf510 msgr2=0x7f9344162400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.263+0000 7f934a800640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93440cf510 0x7f9344162400 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f932c00d800 tx=0x7f932c00dcc0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 shutdown_connections 2024-04-20T10:28:53.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f934415fe20 0x7f9344162940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f932006c8f0 0x7f932006edb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93440cf510 0x7f9344162400 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 --2- 172.21.15.47:0/3532773846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f934406ed60 0x7f9344152500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 >> 172.21.15.47:0/3532773846 conn(0x7f93440b3fb0 msgr2=0x7f93440cd0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:53.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 shutdown_connections 2024-04-20T10:28:53.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.264+0000 7f934a800640 1 -- 172.21.15.47:0/3532773846 wait complete. 2024-04-20T10:28:53.283 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.489+0000 7f4d21e00640 1 Processor -- start 2024-04-20T10:28:53.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.489+0000 7f4d21e00640 1 -- start start 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c159e10 0x7f4d1c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d1c06da90 0x7f4d1c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4d1c15cc90 con 0x7f4d1c15a1f0 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4d1c072900 con 0x7f4d1c06da90 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c15ca40 con 0x7f4d1c159e10 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d1c06da90 0x7f4d1c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d1c06da90 0x7f4d1c071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35332/0 (socket says 172.21.15.47:35332) 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.491+0000 7f4d21400640 1 -- 172.21.15.47:0/1871917374 learned_addr learned my addr 172.21.15.47:0/1871917374 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:53.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.492+0000 7f4d20a00640 1 --2- 172.21.15.47:0/1871917374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c159e10 0x7f4d1c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.492+0000 7f4d21400640 1 -- 172.21.15.47:0/1871917374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c159e10 msgr2=0x7f4d1c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.492+0000 7f4d21400640 1 --2- 172.21.15.47:0/1871917374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c159e10 0x7f4d1c071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.492+0000 7f4d21400640 1 -- 172.21.15.47:0/1871917374 >> v1:172.21.15.175:6789/0 conn(0x7f4d1c15a1f0 legacy=0x7f4d1c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.492+0000 7f4d21400640 1 -- 172.21.15.47:0/1871917374 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c15cf70 con 0x7f4d1c06da90 2024-04-20T10:28:53.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.493+0000 7f4d21400640 1 --2- 172.21.15.47:0/1871917374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c071fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d1402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf4dd90022c7fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d1b400640 1 -- 172.21.15.47:0/1871917374 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d14035020 con 0x7f4d1c06da90 2024-04-20T10:28:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d1b400640 1 -- 172.21.15.47:0/1871917374 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d14031640 con 0x7f4d1c06da90 2024-04-20T10:28:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d1b400640 1 -- 172.21.15.47:0/1871917374 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d1402d040 con 0x7f4d1c06da90 2024-04-20T10:28:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d21e00640 1 -- 172.21.15.47:0/1871917374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 msgr2=0x7f4d1c071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d21e00640 1 --2- 172.21.15.47:0/1871917374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c071fc0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009870 tx=0x7f4d1402b8f0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d21e00640 1 -- 172.21.15.47:0/1871917374 shutdown_connections 2024-04-20T10:28:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d21e00640 1 --2- 172.21.15.47:0/1871917374 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c071fc0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d21e00640 1 --2- 172.21.15.47:0/1871917374 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c159e10 0x7f4d1c071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d21e00640 1 -- 172.21.15.47:0/1871917374 >> 172.21.15.47:0/1871917374 conn(0x7f4d1c0d0040 msgr2=0x7f4d1c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.494+0000 7f4d21e00640 1 -- 172.21.15.47:0/1871917374 shutdown_connections 2024-04-20T10:28:53.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.495+0000 7f4d21e00640 1 -- 172.21.15.47:0/1871917374 wait complete. 2024-04-20T10:28:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.496+0000 7f4d21e00640 1 Processor -- start 2024-04-20T10:28:53.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.496+0000 7f4d21e00640 1 -- start start 2024-04-20T10:28:53.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.497+0000 7f4d21e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.497+0000 7f4d21e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d1c159e10 0x7f4d1c09cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.497+0000 7f4d21e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c15a1f0 0x7f4d1c1f4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.497+0000 7f4d21e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c15d810 con 0x7f4d1c06da90 2024-04-20T10:28:53.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.497+0000 7f4d21e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d1c15d530 con 0x7f4d1c159e10 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.497+0000 7f4d21e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d1c06b900 con 0x7f4d1c15a1f0 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c156cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:35334/0 (socket says 172.21.15.47:35334) 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 -- 172.21.15.47:0/4175326461 learned_addr learned my addr 172.21.15.47:0/4175326461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d21400640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c15a1f0 0x7f4d1c1f4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d1be00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d1c159e10 0x7f4d1c09cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 -- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c15a1f0 msgr2=0x7f4d1c1f4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c15a1f0 0x7f4d1c1f4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 -- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d1c159e10 msgr2=0x7f4d1c09cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d1c159e10 0x7f4d1c09cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 -- 172.21.15.47:0/4175326461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d1c1f4570 con 0x7f4d1c06da90 2024-04-20T10:28:53.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.498+0000 7f4d20a00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c156cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c007ef0 tx=0x7f4d0c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.499+0000 7f4d18c00640 1 -- 172.21.15.47:0/4175326461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c014620 con 0x7f4d1c06da90 2024-04-20T10:28:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.499+0000 7f4d18c00640 1 -- 172.21.15.47:0/4175326461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d0c019070 con 0x7f4d1c06da90 2024-04-20T10:28:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.499+0000 7f4d18c00640 1 -- 172.21.15.47:0/4175326461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d0c014a00 con 0x7f4d1c06da90 2024-04-20T10:28:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.499+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d1c1f4840 con 0x7f4d1c06da90 2024-04-20T10:28:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.499+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d1c1f4e00 con 0x7f4d1c06da90 2024-04-20T10:28:53.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.500+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d1c075b60 con 0x7f4d1c06da90 2024-04-20T10:28:53.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.501+0000 7f4d18c00640 1 -- 172.21.15.47:0/4175326461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d0c014ba0 con 0x7f4d1c06da90 2024-04-20T10:28:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.502+0000 7f4d18c00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4cfc06c8f0 0x7f4cfc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.502+0000 7f4d18c00640 1 -- 172.21.15.47:0/4175326461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f4d0c08c880 con 0x7f4d1c06da90 2024-04-20T10:28:53.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.503+0000 7f4d1be00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4cfc06c8f0 0x7f4cfc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:53.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.504+0000 7f4d1be00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4cfc06c8f0 0x7f4cfc06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006fd0 tx=0x7f4d08008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:53.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.506+0000 7f4d18c00640 1 -- 172.21.15.47:0/4175326461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d0c05a8f0 con 0x7f4d1c06da90 2024-04-20T10:28:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.729+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4d1c09c3a0 con 0x7f4d1c06da90 2024-04-20T10:28:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.729+0000 7f4d18c00640 1 -- 172.21.15.47:0/4175326461 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f4d0c05a1f0 con 0x7f4d1c06da90 2024-04-20T10:28:53.729 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4cfc06c8f0 msgr2=0x7f4cfc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4cfc06c8f0 0x7f4cfc06edb0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f4d08006fd0 tx=0x7f4d08008040 comp rx=0 tx=0).stop 2024-04-20T10:28:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 msgr2=0x7f4d1c156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c156cc0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c007ef0 tx=0x7f4d0c00e4b0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 shutdown_connections 2024-04-20T10:28:53.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d1c15a1f0 0x7f4d1c1f4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d1c159e10 0x7f4d1c09cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4cfc06c8f0 0x7f4cfc06edb0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 --2- 172.21.15.47:0/4175326461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d1c06da90 0x7f4d1c156cc0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 >> 172.21.15.47:0/4175326461 conn(0x7f4d1c0d0040 msgr2=0x7f4d1c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 shutdown_connections 2024-04-20T10:28:53.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:53.732+0000 7f4d21e00640 1 -- 172.21.15.47:0/4175326461 wait complete. 2024-04-20T10:28:53.751 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-04-20T10:28:53.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:53.751 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2024-04-20T10:28:53.835 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:28:53.835 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:28:53.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:53.835 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T10:28:53.887 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:28:53.887 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:53.945 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:28:53.945 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:53.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:54.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.219+0000 7fb519e00640 1 Processor -- start 2024-04-20T10:28:54.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.220+0000 7fb519e00640 1 -- start start 2024-04-20T10:28:54.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.221+0000 7fb519e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb514159e10 0x7fb514071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.221+0000 7fb519e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb514071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.221+0000 7fb519e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb51406da90 0x7fb514160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.221+0000 7fb519e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb514072900 con 0x7fb51406da90 2024-04-20T10:28:54.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.221+0000 7fb519e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb51415ca40 con 0x7fb514159e10 2024-04-20T10:28:54.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.221+0000 7fb519e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb51415cc90 con 0x7fb51415a1f0 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb51406da90 0x7fb514160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb51406da90 0x7fb514160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59564/0 (socket says 172.21.15.47:59564) 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 -- 172.21.15.47:0/2825070637 learned_addr learned my addr 172.21.15.47:0/2825070637 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb518a00640 1 --2- 172.21.15.47:0/2825070637 >> v2:172.21.15.175:3300/0 conn(0x7fb514159e10 0x7fb514071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb513e00640 1 --2- 172.21.15.47:0/2825070637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb514071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 -- 172.21.15.47:0/2825070637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 msgr2=0x7fb514071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 --2- 172.21.15.47:0/2825070637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb514071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 -- 172.21.15.47:0/2825070637 >> v2:172.21.15.175:3300/0 conn(0x7fb514159e10 msgr2=0x7fb514071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 --2- 172.21.15.47:0/2825070637 >> v2:172.21.15.175:3300/0 conn(0x7fb514159e10 0x7fb514071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.222+0000 7fb519400640 1 -- 172.21.15.47:0/2825070637 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb51415cf70 con 0x7fb51406da90 2024-04-20T10:28:54.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.224+0000 7fb519400640 1 --2- 172.21.15.47:0/2825070637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb514160030 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb514073f30 tx=0x7fb50c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5d5ba8c2e666d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:54.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.224+0000 7fb513400640 1 -- 172.21.15.47:0/2825070637 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c02ca70 con 0x7fb51406da90 2024-04-20T10:28:54.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.224+0000 7fb513400640 1 -- 172.21.15.47:0/2825070637 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb50c02cc10 con 0x7fb51406da90 2024-04-20T10:28:54.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.224+0000 7fb513400640 1 -- 172.21.15.47:0/2825070637 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb50c0343f0 con 0x7fb51406da90 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 -- 172.21.15.47:0/2825070637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 msgr2=0x7fb514160030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 --2- 172.21.15.47:0/2825070637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb514160030 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb514073f30 tx=0x7fb50c02bed0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 -- 172.21.15.47:0/2825070637 shutdown_connections 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 --2- 172.21.15.47:0/2825070637 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb514160030 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 --2- 172.21.15.47:0/2825070637 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb514071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 --2- 172.21.15.47:0/2825070637 >> v2:172.21.15.175:3300/0 conn(0x7fb514159e10 0x7fb514071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 -- 172.21.15.47:0/2825070637 >> 172.21.15.47:0/2825070637 conn(0x7fb5140d0040 msgr2=0x7fb5140d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:54.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 -- 172.21.15.47:0/2825070637 shutdown_connections 2024-04-20T10:28:54.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.225+0000 7fb519e00640 1 -- 172.21.15.47:0/2825070637 wait complete. 2024-04-20T10:28:54.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.227+0000 7fb519e00640 1 Processor -- start 2024-04-20T10:28:54.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.227+0000 7fb519e00640 1 -- start start 2024-04-20T10:28:54.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb519e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb5141f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb519e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb514159e10 0x7fb5140b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb519e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb5140cd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb519e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb51415d810 con 0x7fb51406da90 2024-04-20T10:28:54.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb519e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb51415d530 con 0x7fb514159e10 2024-04-20T10:28:54.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb519e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb51406b900 con 0x7fb51415a1f0 2024-04-20T10:28:54.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb518a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb5141f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb518a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb5141f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59580/0 (socket says 172.21.15.47:59580) 2024-04-20T10:28:54.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.228+0000 7fb518a00640 1 -- 172.21.15.47:0/3657578394 learned_addr learned my addr 172.21.15.47:0/3657578394 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb519400640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb5140cd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb513e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb514159e10 0x7fb5140b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb518a00640 1 -- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 msgr2=0x7fb5140cd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb518a00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb5140cd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb518a00640 1 -- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb514159e10 msgr2=0x7fb5140b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb518a00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb514159e10 0x7fb5140b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb518a00640 1 -- 172.21.15.47:0/3657578394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5141f4f60 con 0x7fb51406da90 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb518a00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb5141f4a20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb50800d800 tx=0x7fb50800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb510c00640 1 -- 172.21.15.47:0/3657578394 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508018020 con 0x7fb51406da90 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb510c00640 1 -- 172.21.15.47:0/3657578394 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb5080096a0 con 0x7fb51406da90 2024-04-20T10:28:54.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb510c00640 1 -- 172.21.15.47:0/3657578394 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb508009a70 con 0x7fb51406da90 2024-04-20T10:28:54.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.229+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5140b33f0 con 0x7fb51406da90 2024-04-20T10:28:54.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.230+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5140b39b0 con 0x7fb51406da90 2024-04-20T10:28:54.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.231+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb514075b60 con 0x7fb51406da90 2024-04-20T10:28:54.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.232+0000 7fb510c00640 1 -- 172.21.15.47:0/3657578394 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb50801b020 con 0x7fb51406da90 2024-04-20T10:28:54.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.233+0000 7fb510c00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.233+0000 7fb510c00640 1 -- 172.21.15.47:0/3657578394 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fb50808c060 con 0x7fb51406da90 2024-04-20T10:28:54.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.233+0000 7fb513e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.235+0000 7fb513e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb4f406c8f0 0x7fb4f406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb500006fd0 tx=0x7fb500008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:54.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.238+0000 7fb510c00640 1 -- 172.21.15.47:0/3657578394 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb508022030 con 0x7fb51406da90 2024-04-20T10:28:54.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.442+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb51406d140 con 0x7fb51406da90 2024-04-20T10:28:54.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.444+0000 7fb510c00640 1 -- 172.21.15.47:0/3657578394 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7fb5141f4f60 con 0x7fb51406da90 2024-04-20T10:28:54.445 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:54.445 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:54.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb4f406c8f0 msgr2=0x7fb4f406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb4f406c8f0 0x7fb4f406edb0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb500006fd0 tx=0x7fb500008040 comp rx=0 tx=0).stop 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 msgr2=0x7fb5141f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb5141f4a20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb50800d800 tx=0x7fb50800dcc0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 shutdown_connections 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb51415a1f0 0x7fb5140cd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb514159e10 0x7fb5140b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb4f406c8f0 0x7fb4f406edb0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 --2- 172.21.15.47:0/3657578394 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb51406da90 0x7fb5141f4a20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.447+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 >> 172.21.15.47:0/3657578394 conn(0x7fb5140d0040 msgr2=0x7fb514156df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.448+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 shutdown_connections 2024-04-20T10:28:54.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.448+0000 7fb519e00640 1 -- 172.21.15.47:0/3657578394 wait complete. 2024-04-20T10:28:54.468 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:54.469 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.677+0000 7f7814c00640 1 Processor -- start 2024-04-20T10:28:54.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.677+0000 7f7814c00640 1 -- start start 2024-04-20T10:28:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.679+0000 7f7814c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 0x7f78100c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.679+0000 7f7814c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f78100ae3e0 0x7f78100c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.679+0000 7f7814c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f78100aded0 con 0x7f78100cdea0 2024-04-20T10:28:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.679+0000 7f7814c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f78100ad2c0 con 0x7f78100ae3e0 2024-04-20T10:28:54.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.679+0000 7f7814c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78100ad5a0 con 0x7f7810161b40 2024-04-20T10:28:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.679+0000 7f780fe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f78100cdea0 0x7f78101620d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34722/0 (socket says 172.21.15.47:34722) 2024-04-20T10:28:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.679+0000 7f780fe00640 1 -- 172.21.15.47:0/4087037414 learned_addr learned my addr 172.21.15.47:0/4087037414 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f7807e00640 1 --2- 172.21.15.47:0/4087037414 >> v2:172.21.15.175:3300/0 conn(0x7f78100ae3e0 0x7f78100c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780f400640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 0x7f78100c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2025121722 0 0) 0x7f78100aded0 con 0x7f78100cdea0 2024-04-20T10:28:54.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77f4003520 con 0x7f78100cdea0 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2961943673 0 0) 0x7f77f4003520 con 0x7f78100cdea0 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 msgr2=0x7f78100c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 0x7f78100c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 >> v2:172.21.15.175:3300/0 conn(0x7f78100ae3e0 msgr2=0x7f78100c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 --2- 172.21.15.47:0/4087037414 >> v2:172.21.15.175:3300/0 conn(0x7f78100ae3e0 0x7f78100c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78100ad880 con 0x7f78100cdea0 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f78080019f0 con 0x7f78100cdea0 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f7807e00640 1 --2- 172.21.15.47:0/4087037414 >> v2:172.21.15.175:3300/0 conn(0x7f78100ae3e0 0x7f78100c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:28:54.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.680+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 >> v1:172.21.15.47:6789/0 conn(0x7f78100cdea0 legacy=0x7f78101620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780ea00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40039b0 0x7f77f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780ea00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 0x7f77f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780ea00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4004d40 0x7f77f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f78100ad2c0 con 0x7f77f40039b0 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78100ad5a0 con 0x7f77f4004d40 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77f400b9c0 con 0x7f7810161b40 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780f400640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40039b0 0x7f77f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f7807e00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 0x7f77f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780f400640 1 -- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 msgr2=0x7f77f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780fe00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4004d40 0x7f77f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780f400640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 0x7f77f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780f400640 1 -- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4004d40 msgr2=0x7f77f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:54.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.681+0000 7f780f400640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f4004d40 0x7f77f40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.682+0000 7f780f400640 1 -- 172.21.15.47:0/4087037414 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f4003520 con 0x7f77f40039b0 2024-04-20T10:28:54.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.682+0000 7f7807e00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7810161b40 0x7f77f4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.683+0000 7f780f400640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40039b0 0x7f77f4003dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7800009ac0 tx=0x7f77f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95099bf37f97134e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.683+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800039070 con 0x7f77f40039b0 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.683+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f780002c6b0 con 0x7f77f40039b0 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.683+0000 7f780ea00640 1 -- 172.21.15.47:0/4087037414 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800009690 con 0x7f77f40039b0 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.684+0000 7f7814c00640 1 -- 172.21.15.47:0/4087037414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40039b0 msgr2=0x7f77f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.684+0000 7f7814c00640 1 --2- 172.21.15.47:0/4087037414 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77f40039b0 0x7f77f4003dd0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7800009ac0 tx=0x7f77f4005830 comp rx=0 tx=0).stop 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.684+0000 7f780fe00640 1 -- 172.21.15.47:0/4087037414 reap_dead start 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.684+0000 7f7814c00640 1 -- 172.21.15.47:0/4087037414 shutdown_connections 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.684+0000 7f7814c00640 1 -- 172.21.15.47:0/4087037414 >> 172.21.15.47:0/4087037414 conn(0x7f78100cffa0 msgr2=0x7f78100d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.684+0000 7f7814c00640 1 -- 172.21.15.47:0/4087037414 shutdown_connections 2024-04-20T10:28:54.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.684+0000 7f7814c00640 1 -- 172.21.15.47:0/4087037414 wait complete. 2024-04-20T10:28:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.686+0000 7f7814c00640 1 Processor -- start 2024-04-20T10:28:54.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.686+0000 7f7814c00640 1 -- start start 2024-04-20T10:28:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.687+0000 7f7814c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40039b0 0x7f78100b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.687+0000 7f7814c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78100b2d60 0x7f781009c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.687+0000 7f7814c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78101f4430 0x7f78100b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.687+0000 7f7814c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f781006d0c0 con 0x7f78100b2d60 2024-04-20T10:28:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.687+0000 7f7814c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78100acf50 con 0x7f78101f4430 2024-04-20T10:28:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.687+0000 7f7814c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f781006d240 con 0x7f77f40039b0 2024-04-20T10:28:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78100b2d60 0x7f781009c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78100b2d60 0x7f781009c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59610/0 (socket says 172.21.15.47:59610) 2024-04-20T10:28:54.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 -- 172.21.15.47:0/2112657548 learned_addr learned my addr 172.21.15.47:0/2112657548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f780fe00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78101f4430 0x7f78100b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f780f400640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40039b0 0x7f78100b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 -- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40039b0 msgr2=0x7f78100b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40039b0 0x7f78100b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 -- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78101f4430 msgr2=0x7f78100b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78101f4430 0x7f78100b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.688+0000 7f7807e00640 1 -- 172.21.15.47:0/2112657548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78100b4eb0 con 0x7f78100b2d60 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.689+0000 7f7807e00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78100b2d60 0x7f781009c160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f77f80109d0 tx=0x7f77f8010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.689+0000 7f7807400640 1 -- 172.21.15.47:0/2112657548 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f800f040 con 0x7f78100b2d60 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.689+0000 7f7807400640 1 -- 172.21.15.47:0/2112657548 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77f800ec80 con 0x7f78100b2d60 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.689+0000 7f7807400640 1 -- 172.21.15.47:0/2112657548 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f801c400 con 0x7f78100b2d60 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.689+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78100b5190 con 0x7f78100b2d60 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.689+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78100b5750 con 0x7f78100b2d60 2024-04-20T10:28:54.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.690+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78100b5b30 con 0x7f78100b2d60 2024-04-20T10:28:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.691+0000 7f7807400640 1 -- 172.21.15.47:0/2112657548 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77f801c5a0 con 0x7f78100b2d60 2024-04-20T10:28:54.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.692+0000 7f7807400640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f77ec06c920 0x7f77ec06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.692+0000 7f7807400640 1 -- 172.21.15.47:0/2112657548 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f77f808c7e0 con 0x7f78100b2d60 2024-04-20T10:28:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.693+0000 7f780f400640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f77ec06c920 0x7f77ec06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:54.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.694+0000 7f780f400640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f77ec06c920 0x7f77ec06ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7800036000 tx=0x7f7800030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:54.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.697+0000 7f7807400640 1 -- 172.21.15.47:0/2112657548 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f78100b5b30 con 0x7f78100b2d60 2024-04-20T10:28:54.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.903+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f7810075b30 con 0x7f78100b2d60 2024-04-20T10:28:54.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.904+0000 7f7807400640 1 -- 172.21.15.47:0/2112657548 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f77f8018030 con 0x7f78100b2d60 2024-04-20T10:28:54.905 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f77ec06c920 msgr2=0x7f77ec06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f77ec06c920 0x7f77ec06ede0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f7800036000 tx=0x7f7800030000 comp rx=0 tx=0).stop 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78100b2d60 msgr2=0x7f781009c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78100b2d60 0x7f781009c160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f77f80109d0 tx=0x7f77f8010e90 comp rx=0 tx=0).stop 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 shutdown_connections 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78101f4430 0x7f78100b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f77ec06c920 0x7f77ec06ede0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f78100b2d60 0x7f781009c160 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 --2- 172.21.15.47:0/2112657548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f40039b0 0x7f78100b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.907+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 >> 172.21.15.47:0/2112657548 conn(0x7f78100cffa0 msgr2=0x7f781015cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:54.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.908+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 shutdown_connections 2024-04-20T10:28:54.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:54.908+0000 7f7814c00640 1 -- 172.21.15.47:0/2112657548 wait complete. 2024-04-20T10:28:54.926 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:55.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.146+0000 7f589e400640 1 Processor -- start 2024-04-20T10:28:55.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.146+0000 7f589e400640 1 -- start start 2024-04-20T10:28:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.148+0000 7f589e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5898161a50 0x7f58980b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.148+0000 7f589e400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f589806c010 0x7f58980ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.148+0000 7f589e400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f58980d0980 con 0x7f58980d3b20 2024-04-20T10:28:55.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.148+0000 7f589e400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f58980d0c60 con 0x7f589806c010 2024-04-20T10:28:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.148+0000 7f589e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5898162ae0 con 0x7f5898161a50 2024-04-20T10:28:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.149+0000 7f589d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5898161a50 0x7f58980b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.149+0000 7f589da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f589806c010 0x7f58980ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.149+0000 7f589da00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f589806c010 0x7f58980ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:53844/0 (socket says 172.21.15.47:53844) 2024-04-20T10:28:55.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.149+0000 7f589da00640 1 -- 172.21.15.47:0/4028201756 learned_addr learned my addr 172.21.15.47:0/4028201756 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.150+0000 7f589d000640 1 -- 172.21.15.47:0/4028201756 >> v2:172.21.15.175:3300/0 conn(0x7f589806c010 msgr2=0x7f58980ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.150+0000 7f589d000640 1 --2- 172.21.15.47:0/4028201756 >> v2:172.21.15.175:3300/0 conn(0x7f589806c010 0x7f58980ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.150+0000 7f589d000640 1 -- 172.21.15.47:0/4028201756 >> v1:172.21.15.175:6789/0 conn(0x7f58980d3b20 legacy=0x7f589806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.150+0000 7f589d000640 1 -- 172.21.15.47:0/4028201756 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5898162490 con 0x7f5898161a50 2024-04-20T10:28:55.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.151+0000 7f589d000640 1 --2- 172.21.15.47:0/4028201756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5898161a50 0x7f58980b3180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f588c0097a0 tx=0x7f588c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e069f00d14942ce server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.152+0000 7f5897400640 1 -- 172.21.15.47:0/4028201756 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c035020 con 0x7f5898161a50 2024-04-20T10:28:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.152+0000 7f5897400640 1 -- 172.21.15.47:0/4028201756 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f588c031830 con 0x7f5898161a50 2024-04-20T10:28:55.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.152+0000 7f5897400640 1 -- 172.21.15.47:0/4028201756 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588c031c00 con 0x7f5898161a50 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.152+0000 7f589e400640 1 -- 172.21.15.47:0/4028201756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5898161a50 msgr2=0x7f58980b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.152+0000 7f589e400640 1 --2- 172.21.15.47:0/4028201756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5898161a50 0x7f58980b3180 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f588c0097a0 tx=0x7f588c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.153+0000 7f589e400640 1 -- 172.21.15.47:0/4028201756 shutdown_connections 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.153+0000 7f589e400640 1 --2- 172.21.15.47:0/4028201756 >> v2:172.21.15.175:3300/0 conn(0x7f589806c010 0x7f58980ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.153+0000 7f589e400640 1 --2- 172.21.15.47:0/4028201756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5898161a50 0x7f58980b3180 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.153+0000 7f589e400640 1 -- 172.21.15.47:0/4028201756 >> 172.21.15.47:0/4028201756 conn(0x7f58980d0ed0 msgr2=0x7f58980d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.153+0000 7f589e400640 1 -- 172.21.15.47:0/4028201756 shutdown_connections 2024-04-20T10:28:55.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.153+0000 7f589e400640 1 -- 172.21.15.47:0/4028201756 wait complete. 2024-04-20T10:28:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.155+0000 7f589e400640 1 Processor -- start 2024-04-20T10:28:55.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.155+0000 7f589e400640 1 -- start start 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f589806c010 0x7f589815f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589e400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5898161a50 0x7f58980cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f589815bf20 0x7f58980cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589e400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f58980d0600 con 0x7f5898161a50 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58980d0480 con 0x7f589815bf20 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5898161e30 con 0x7f589806c010 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f5897e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5898161a50 0x7f58980cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f5897e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5898161a50 0x7f58980cce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59616/0 (socket says 172.21.15.47:59616) 2024-04-20T10:28:55.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f5897e00640 1 -- 172.21.15.47:0/949537078 learned_addr learned my addr 172.21.15.47:0/949537078 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589d000640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f589806c010 0x7f589815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.156+0000 7f589da00640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f589815bf20 0x7f58980cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5897e00640 1 -- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f589806c010 msgr2=0x7f589815f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5897e00640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f589806c010 0x7f589815f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5897e00640 1 -- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f589815bf20 msgr2=0x7f58980cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5897e00640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f589815bf20 0x7f58980cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5897e00640 1 -- 172.21.15.47:0/949537078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589815fef0 con 0x7f5898161a50 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f589d000640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f589806c010 0x7f589815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5897e00640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5898161a50 0x7f58980cce00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f588400c400 tx=0x7f588400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5894c00640 1 -- 172.21.15.47:0/949537078 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58840052d0 con 0x7f5898161a50 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5894c00640 1 -- 172.21.15.47:0/949537078 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5884005490 con 0x7f5898161a50 2024-04-20T10:28:55.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f5894c00640 1 -- 172.21.15.47:0/949537078 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5884007260 con 0x7f5898161a50 2024-04-20T10:28:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.157+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589815e410 con 0x7f5898161a50 2024-04-20T10:28:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.158+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f589815c390 con 0x7f5898161a50 2024-04-20T10:28:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.159+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5898075b30 con 0x7f5898161a50 2024-04-20T10:28:55.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.160+0000 7f5894c00640 1 -- 172.21.15.47:0/949537078 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5884007400 con 0x7f5898161a50 2024-04-20T10:28:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.161+0000 7f5894c00640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f587806c920 0x7f587806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.161+0000 7f5894c00640 1 -- 172.21.15.47:0/949537078 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f588408b420 con 0x7f5898161a50 2024-04-20T10:28:55.161 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.162+0000 7f589d000640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f587806c920 0x7f587806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.163+0000 7f589d000640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f587806c920 0x7f587806ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f588c036000 tx=0x7f588c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:55.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.166+0000 7f5894c00640 1 -- 172.21.15.47:0/949537078 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5884059490 con 0x7f5898161a50 2024-04-20T10:28:55.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.379+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f589806e8a0 con 0x7f5898161a50 2024-04-20T10:28:55.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.380+0000 7f5894c00640 1 -- 172.21.15.47:0/949537078 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f5884058d90 con 0x7f5898161a50 2024-04-20T10:28:55.380 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.382+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f587806c920 msgr2=0x7f587806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.382+0000 7f589e400640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f587806c920 0x7f587806ede0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f588c036000 tx=0x7f588c008030 comp rx=0 tx=0).stop 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5898161a50 msgr2=0x7f58980cce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5898161a50 0x7f58980cce00 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f588400c400 tx=0x7f588400c8c0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 shutdown_connections 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f589815bf20 0x7f58980cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f587806c920 0x7f587806ede0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5898161a50 0x7f58980cce00 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 --2- 172.21.15.47:0/949537078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f589806c010 0x7f589815f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 >> 172.21.15.47:0/949537078 conn(0x7f58980d0ed0 msgr2=0x7f58980b5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 shutdown_connections 2024-04-20T10:28:55.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.383+0000 7f589e400640 1 -- 172.21.15.47:0/949537078 wait complete. 2024-04-20T10:28:55.402 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.612+0000 7fd17e200640 1 Processor -- start 2024-04-20T10:28:55.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.612+0000 7fd17e200640 1 -- start start 2024-04-20T10:28:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.614+0000 7fd17e200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd178073b80 0x7fd17806f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.614+0000 7fd17e200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd178071a60 0x7fd17806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.614+0000 7fd17e200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd17815cc20 con 0x7fd17815a130 2024-04-20T10:28:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.614+0000 7fd17e200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd17815d7f0 con 0x7fd178071a60 2024-04-20T10:28:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.614+0000 7fd17e200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd17806fbe0 con 0x7fd178073b80 2024-04-20T10:28:55.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.615+0000 7fd17d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd178071a60 0x7fd17806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.615+0000 7fd17d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd178071a60 0x7fd17806f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59618/0 (socket says 172.21.15.47:59618) 2024-04-20T10:28:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.615+0000 7fd17d800640 1 -- 172.21.15.47:0/2552660029 learned_addr learned my addr 172.21.15.47:0/2552660029 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.615+0000 7fd17d800640 1 -- 172.21.15.47:0/2552660029 >> v2:172.21.15.175:3300/0 conn(0x7fd178073b80 msgr2=0x7fd17806f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.615+0000 7fd17d800640 1 --2- 172.21.15.47:0/2552660029 >> v2:172.21.15.175:3300/0 conn(0x7fd178073b80 0x7fd17806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.615+0000 7fd17d800640 1 -- 172.21.15.47:0/2552660029 >> v1:172.21.15.175:6789/0 conn(0x7fd17815a130 legacy=0x7fd178073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:55.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.615+0000 7fd17d800640 1 -- 172.21.15.47:0/2552660029 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17815cec0 con 0x7fd178071a60 2024-04-20T10:28:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.616+0000 7fd17d800640 1 --2- 172.21.15.47:0/2552660029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd178071a60 0x7fd17806f6a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd17806cfe0 tx=0x7fd17002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668956c16b19c9db server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd177400640 1 -- 172.21.15.47:0/2552660029 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17002c9f0 con 0x7fd178071a60 2024-04-20T10:28:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd177400640 1 -- 172.21.15.47:0/2552660029 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17002cb90 con 0x7fd178071a60 2024-04-20T10:28:55.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd177400640 1 -- 172.21.15.47:0/2552660029 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1700343f0 con 0x7fd178071a60 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd17e200640 1 -- 172.21.15.47:0/2552660029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd178071a60 msgr2=0x7fd17806f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd17e200640 1 --2- 172.21.15.47:0/2552660029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd178071a60 0x7fd17806f6a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd17806cfe0 tx=0x7fd17002be50 comp rx=0 tx=0).stop 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd17e200640 1 -- 172.21.15.47:0/2552660029 shutdown_connections 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd17e200640 1 --2- 172.21.15.47:0/2552660029 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd178071a60 0x7fd17806f6a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd17e200640 1 --2- 172.21.15.47:0/2552660029 >> v2:172.21.15.175:3300/0 conn(0x7fd178073b80 0x7fd17806f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.617+0000 7fd17e200640 1 -- 172.21.15.47:0/2552660029 >> 172.21.15.47:0/2552660029 conn(0x7fd1780cffa0 msgr2=0x7fd1780d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.618+0000 7fd17e200640 1 -- 172.21.15.47:0/2552660029 shutdown_connections 2024-04-20T10:28:55.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.618+0000 7fd17e200640 1 -- 172.21.15.47:0/2552660029 wait complete. 2024-04-20T10:28:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.619+0000 7fd17e200640 1 Processor -- start 2024-04-20T10:28:55.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.619+0000 7fd17e200640 1 -- start start 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.620+0000 7fd17e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178071a60 0x7fd1780b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.620+0000 7fd17e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178073b80 0x7fd17814d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.620+0000 7fd17e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd17815a130 0x7fd17814daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.620+0000 7fd17e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd17806b900 con 0x7fd17815a130 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.620+0000 7fd17e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd17815d480 con 0x7fd178071a60 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.620+0000 7fd17e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd17806ba80 con 0x7fd178073b80 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd17815a130 0x7fd17814daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd17815a130 0x7fd17814daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59628/0 (socket says 172.21.15.47:59628) 2024-04-20T10:28:55.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 -- 172.21.15.47:0/3483401629 learned_addr learned my addr 172.21.15.47:0/3483401629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd177e00640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178073b80 0x7fd17814d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17ce00640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178071a60 0x7fd1780b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 -- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178073b80 msgr2=0x7fd17814d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178073b80 0x7fd17814d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 -- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178071a60 msgr2=0x7fd1780b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178071a60 0x7fd1780b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17d800640 1 -- 172.21.15.47:0/3483401629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17814dfe0 con 0x7fd17815a130 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd17ce00640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178071a60 0x7fd1780b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.621+0000 7fd177e00640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178073b80 0x7fd17814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.622+0000 7fd17d800640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd17815a130 0x7fd17814daa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd170036040 tx=0x7fd170004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.622+0000 7fd174c00640 1 -- 172.21.15.47:0/3483401629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170043020 con 0x7fd17815a130 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.622+0000 7fd174c00640 1 -- 172.21.15.47:0/3483401629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd17003e450 con 0x7fd17815a130 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.622+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1780cb820 con 0x7fd17815a130 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.622+0000 7fd174c00640 1 -- 172.21.15.47:0/3483401629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17003e860 con 0x7fd17815a130 2024-04-20T10:28:55.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.622+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1780c97e0 con 0x7fd17815a130 2024-04-20T10:28:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.623+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd178075b30 con 0x7fd17815a130 2024-04-20T10:28:55.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.624+0000 7fd174c00640 1 -- 172.21.15.47:0/3483401629 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd170046020 con 0x7fd17815a130 2024-04-20T10:28:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.625+0000 7fd174c00640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:55.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.625+0000 7fd174c00640 1 -- 172.21.15.47:0/3483401629 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fd1700ae910 con 0x7fd17815a130 2024-04-20T10:28:55.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.626+0000 7fd17ce00640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:55.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.627+0000 7fd17ce00640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd15806c8f0 0x7fd15806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00a830 tx=0x7fd16c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:55.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.630+0000 7fd174c00640 1 -- 172.21.15.47:0/3483401629 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd17004d030 con 0x7fd17815a130 2024-04-20T10:28:55.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.835+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd17806d010 con 0x7fd17815a130 2024-04-20T10:28:55.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.836+0000 7fd174c00640 1 -- 172.21.15.47:0/3483401629 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7fd17007c8f0 con 0x7fd17815a130 2024-04-20T10:28:55.837 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd15806c8f0 msgr2=0x7fd15806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd15806c8f0 0x7fd15806edb0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd16c00a830 tx=0x7fd16c008040 comp rx=0 tx=0).stop 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd17815a130 msgr2=0x7fd17814daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd17815a130 0x7fd17814daa0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fd170036040 tx=0x7fd170004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 shutdown_connections 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd15806c8f0 0x7fd15806edb0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd17815a130 0x7fd17814daa0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd178073b80 0x7fd17814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 --2- 172.21.15.47:0/3483401629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd178071a60 0x7fd1780b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 >> 172.21.15.47:0/3483401629 conn(0x7fd1780cffa0 msgr2=0x7fd17815e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 shutdown_connections 2024-04-20T10:28:55.840 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:55.839+0000 7fd17e200640 1 -- 172.21.15.47:0/3483401629 wait complete. 2024-04-20T10:28:55.859 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.075+0000 7feb68a00640 1 Processor -- start 2024-04-20T10:28:56.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.075+0000 7feb68a00640 1 -- start start 2024-04-20T10:28:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.076+0000 7feb68a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb64161a50 0x7feb640b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.076+0000 7feb68a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb640d3b20 0x7feb6406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.077+0000 7feb68a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb640d0980 con 0x7feb640d0480 2024-04-20T10:28:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.077+0000 7feb68a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7feb640d0c60 con 0x7feb64161a50 2024-04-20T10:28:56.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.077+0000 7feb68a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb64162ae0 con 0x7feb640d3b20 2024-04-20T10:28:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.077+0000 7feb63400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb64161a50 0x7feb640b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.077+0000 7feb63400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb64161a50 0x7feb640b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59640/0 (socket says 172.21.15.47:59640) 2024-04-20T10:28:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.077+0000 7feb63400640 1 -- 172.21.15.47:0/3706924698 learned_addr learned my addr 172.21.15.47:0/3706924698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.078+0000 7feb63400640 1 -- 172.21.15.47:0/3706924698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb640d3b20 msgr2=0x7feb6406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.078+0000 7feb63400640 1 --2- 172.21.15.47:0/3706924698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb640d3b20 0x7feb6406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.078+0000 7feb63400640 1 -- 172.21.15.47:0/3706924698 >> v1:172.21.15.175:6789/0 conn(0x7feb640d0480 legacy=0x7feb6406c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:56.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.078+0000 7feb63400640 1 -- 172.21.15.47:0/3706924698 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb64162490 con 0x7feb64161a50 2024-04-20T10:28:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.079+0000 7feb63400640 1 --2- 172.21.15.47:0/3706924698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb64161a50 0x7feb640b3180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb5402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8aa3721fb6e2708 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.079+0000 7feb62000640 1 -- 172.21.15.47:0/3706924698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5402ca70 con 0x7feb64161a50 2024-04-20T10:28:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.079+0000 7feb62000640 1 -- 172.21.15.47:0/3706924698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5402cc10 con 0x7feb64161a50 2024-04-20T10:28:56.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.079+0000 7feb62000640 1 -- 172.21.15.47:0/3706924698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb540343f0 con 0x7feb64161a50 2024-04-20T10:28:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 -- 172.21.15.47:0/3706924698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb64161a50 msgr2=0x7feb640b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 --2- 172.21.15.47:0/3706924698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb64161a50 0x7feb640b3180 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7feb54009e50 tx=0x7feb5402bed0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 -- 172.21.15.47:0/3706924698 shutdown_connections 2024-04-20T10:28:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 --2- 172.21.15.47:0/3706924698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb640d3b20 0x7feb6406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 --2- 172.21.15.47:0/3706924698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb64161a50 0x7feb640b3180 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 -- 172.21.15.47:0/3706924698 >> 172.21.15.47:0/3706924698 conn(0x7feb640d0ed0 msgr2=0x7feb640d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 -- 172.21.15.47:0/3706924698 shutdown_connections 2024-04-20T10:28:56.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.080+0000 7feb68a00640 1 -- 172.21.15.47:0/3706924698 wait complete. 2024-04-20T10:28:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.082+0000 7feb68a00640 1 Processor -- start 2024-04-20T10:28:56.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.082+0000 7feb68a00640 1 -- start start 2024-04-20T10:28:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.083+0000 7feb68a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb640d0480 0x7feb640cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.083+0000 7feb68a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb64161a50 0x7feb640ce330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.083+0000 7feb68a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6414ee90 0x7feb6414f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.083+0000 7feb68a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb64161fb0 con 0x7feb640d0480 2024-04-20T10:28:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.083+0000 7feb68a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb64161e30 con 0x7feb64161a50 2024-04-20T10:28:56.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.083+0000 7feb68a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb64162130 con 0x7feb6414ee90 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb640d0480 0x7feb640cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb640d0480 0x7feb640cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59642/0 (socket says 172.21.15.47:59642) 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 -- 172.21.15.47:0/968378568 learned_addr learned my addr 172.21.15.47:0/968378568 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb62a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb64161a50 0x7feb640ce330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 -- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6414ee90 msgr2=0x7feb6414f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63e00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6414ee90 0x7feb6414f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6414ee90 0x7feb6414f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 -- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb64161a50 msgr2=0x7feb640ce330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb64161a50 0x7feb640ce330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 -- 172.21.15.47:0/968378568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb640cb520 con 0x7feb640d0480 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb62a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb64161a50 0x7feb640ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63e00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6414ee90 0x7feb6414f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:56.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.084+0000 7feb63400640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb640d0480 0x7feb640cddf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb54036040 tx=0x7feb54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.085+0000 7feb5b400640 1 -- 172.21.15.47:0/968378568 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb54043020 con 0x7feb640d0480 2024-04-20T10:28:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.085+0000 7feb5b400640 1 -- 172.21.15.47:0/968378568 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb5403e450 con 0x7feb640d0480 2024-04-20T10:28:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.085+0000 7feb5b400640 1 -- 172.21.15.47:0/968378568 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb5403e820 con 0x7feb640d0480 2024-04-20T10:28:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.085+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb641e9ae0 con 0x7feb640d0480 2024-04-20T10:28:56.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.085+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb64149eb0 con 0x7feb640d0480 2024-04-20T10:28:56.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.086+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb64163090 con 0x7feb640d0480 2024-04-20T10:28:56.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.087+0000 7feb5b400640 1 -- 172.21.15.47:0/968378568 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb54046020 con 0x7feb640d0480 2024-04-20T10:28:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.088+0000 7feb5b400640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb4406c8f0 0x7feb4406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.088+0000 7feb5b400640 1 -- 172.21.15.47:0/968378568 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7feb540ae880 con 0x7feb640d0480 2024-04-20T10:28:56.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.089+0000 7feb62a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb4406c8f0 0x7feb4406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.090+0000 7feb62a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb4406c8f0 0x7feb4406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb50006fd0 tx=0x7feb50008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:56.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.093+0000 7feb5b400640 1 -- 172.21.15.47:0/968378568 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feb64163090 con 0x7feb640d0480 2024-04-20T10:28:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.295+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7feb64075b30 con 0x7feb640d0480 2024-04-20T10:28:56.296 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.296+0000 7feb5b400640 1 -- 172.21.15.47:0/968378568 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7feb5404d030 con 0x7feb640d0480 2024-04-20T10:28:56.297 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb4406c8f0 msgr2=0x7feb4406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb4406c8f0 0x7feb4406edb0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7feb50006fd0 tx=0x7feb50008040 comp rx=0 tx=0).stop 2024-04-20T10:28:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb640d0480 msgr2=0x7feb640cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb640d0480 0x7feb640cddf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7feb54036040 tx=0x7feb54004a40 comp rx=0 tx=0).stop 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 shutdown_connections 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb6414ee90 0x7feb6414f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb64161a50 0x7feb640ce330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb4406c8f0 0x7feb4406edb0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 --2- 172.21.15.47:0/968378568 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb640d0480 0x7feb640cddf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 >> 172.21.15.47:0/968378568 conn(0x7feb640d0ed0 msgr2=0x7feb640b4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 shutdown_connections 2024-04-20T10:28:56.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.299+0000 7feb68a00640 1 -- 172.21.15.47:0/968378568 wait complete. 2024-04-20T10:28:56.319 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.532+0000 7f5574c00640 1 Processor -- start 2024-04-20T10:28:56.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.532+0000 7f5574c00640 1 -- start start 2024-04-20T10:28:56.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.534+0000 7f5574c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015b7e0 0x7f557015bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.534+0000 7f5574c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f557015c190 0x7f557006ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.534+0000 7f5574c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f557006ccc0 con 0x7f557015b400 2024-04-20T10:28:56.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.534+0000 7f5574c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f557006d280 con 0x7f557015c190 2024-04-20T10:28:56.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.534+0000 7f5574c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f557006cfa0 con 0x7f557015b7e0 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.534+0000 7f556fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f557015c190 0x7f557006ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.535+0000 7f556fe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f557015c190 0x7f557006ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:53880/0 (socket says 172.21.15.47:53880) 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.535+0000 7f556fe00640 1 -- 172.21.15.47:0/1275899227 learned_addr learned my addr 172.21.15.47:0/1275899227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.535+0000 7f5567e00640 1 --2- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015b7e0 0x7f557015bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.535+0000 7f556fe00640 1 -- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015b7e0 msgr2=0x7f557015bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.535+0000 7f556fe00640 1 --2- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015b7e0 0x7f557015bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.535+0000 7f556fe00640 1 -- 172.21.15.47:0/1275899227 >> v1:172.21.15.175:6789/0 conn(0x7f557015b400 legacy=0x7f5570072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.535+0000 7f556fe00640 1 -- 172.21.15.47:0/1275899227 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557006e440 con 0x7f557015c190 2024-04-20T10:28:56.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.537+0000 7f556fe00640 1 --2- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015c190 0x7f557006ec40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5568009e50 tx=0x7f556802bf50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf7471476da326be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:56.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.537+0000 7f556ea00640 1 -- 172.21.15.47:0/1275899227 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556802caf0 con 0x7f557015c190 2024-04-20T10:28:56.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.537+0000 7f556ea00640 1 -- 172.21.15.47:0/1275899227 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f556802cc90 con 0x7f557015c190 2024-04-20T10:28:56.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.537+0000 7f556ea00640 1 -- 172.21.15.47:0/1275899227 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568034610 con 0x7f557015c190 2024-04-20T10:28:56.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 -- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015c190 msgr2=0x7f557006ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 --2- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015c190 0x7f557006ec40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5568009e50 tx=0x7f556802bf50 comp rx=0 tx=0).stop 2024-04-20T10:28:56.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 -- 172.21.15.47:0/1275899227 shutdown_connections 2024-04-20T10:28:56.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 --2- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015c190 0x7f557006ec40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 --2- 172.21.15.47:0/1275899227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015b7e0 0x7f557015bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 -- 172.21.15.47:0/1275899227 >> 172.21.15.47:0/1275899227 conn(0x7f55700d0040 msgr2=0x7f55700d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:56.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 -- 172.21.15.47:0/1275899227 shutdown_connections 2024-04-20T10:28:56.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.538+0000 7f5574c00640 1 -- 172.21.15.47:0/1275899227 wait complete. 2024-04-20T10:28:56.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.540+0000 7f5574c00640 1 Processor -- start 2024-04-20T10:28:56.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.540+0000 7f5574c00640 1 -- start start 2024-04-20T10:28:56.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5574c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015b400 0x7f55700cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5574c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557015b7e0 0x7f557014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5574c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015c190 0x7f5570151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5574c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f557015c730 con 0x7f557015b7e0 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5574c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f557015c5b0 con 0x7f557015b400 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5574c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f557006f180 con 0x7f557015c190 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5567e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557015b7e0 0x7f557014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5567e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557015b7e0 0x7f557014f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59646/0 (socket says 172.21.15.47:59646) 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.541+0000 7f5567e00640 1 -- 172.21.15.47:0/3613913588 learned_addr learned my addr 172.21.15.47:0/3613913588 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f5567e00640 1 -- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015c190 msgr2=0x7f5570151640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f5567e00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015c190 0x7f5570151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f5567e00640 1 -- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015b400 msgr2=0x7f55700cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f556f400640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015b400 0x7f55700cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f5567e00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015b400 0x7f55700cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f5567e00640 1 -- 172.21.15.47:0/3613913588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557013b810 con 0x7f557015b7e0 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f556f400640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015b400 0x7f55700cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f5567e00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557015b7e0 0x7f557014f880 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f555800e9b0 tx=0x7f555800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.542+0000 7f5567400640 1 -- 172.21.15.47:0/3613913588 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558017070 con 0x7f557015b7e0 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.543+0000 7f5567400640 1 -- 172.21.15.47:0/3613913588 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f555800c9a0 con 0x7f557015b7e0 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.543+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557013ba80 con 0x7f557015b7e0 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.543+0000 7f5567400640 1 -- 172.21.15.47:0/3613913588 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800cd90 con 0x7f557015b7e0 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.543+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557013c040 con 0x7f557015b7e0 2024-04-20T10:28:56.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.544+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5570075b60 con 0x7f557015b7e0 2024-04-20T10:28:56.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.545+0000 7f5567400640 1 -- 172.21.15.47:0/3613913588 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f555801a070 con 0x7f557015b7e0 2024-04-20T10:28:56.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.546+0000 7f5567400640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f554c06c8f0 0x7f554c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:56.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.546+0000 7f5567400640 1 -- 172.21.15.47:0/3613913588 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f555808bfd0 con 0x7f557015b7e0 2024-04-20T10:28:56.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.547+0000 7f556f400640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f554c06c8f0 0x7f554c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:56.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.548+0000 7f556f400640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f554c06c8f0 0x7f554c06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5560000f60 tx=0x7f5560009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:56.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.550+0000 7f5567400640 1 -- 172.21.15.47:0/3613913588 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5558059fb0 con 0x7f557015b7e0 2024-04-20T10:28:56.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.768+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f55700cfdb0 con 0x7f557015b7e0 2024-04-20T10:28:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.769+0000 7f5567400640 1 -- 172.21.15.47:0/3613913588 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7f55580598b0 con 0x7f557015b7e0 2024-04-20T10:28:56.769 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f554c06c8f0 msgr2=0x7f554c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f554c06c8f0 0x7f554c06edb0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5560000f60 tx=0x7f5560009040 comp rx=0 tx=0).stop 2024-04-20T10:28:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557015b7e0 msgr2=0x7f557014f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557015b7e0 0x7f557014f880 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f555800e9b0 tx=0x7f555800ee70 comp rx=0 tx=0).stop 2024-04-20T10:28:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 shutdown_connections 2024-04-20T10:28:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015c190 0x7f5570151640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f554c06c8f0 0x7f554c06edb0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f557015b7e0 0x7f557014f880 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 --2- 172.21.15.47:0/3613913588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f557015b400 0x7f55700cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 >> 172.21.15.47:0/3613913588 conn(0x7f55700d0040 msgr2=0x7f55700b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 shutdown_connections 2024-04-20T10:28:56.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:56.772+0000 7f5574c00640 1 -- 172.21.15.47:0/3613913588 wait complete. 2024-04-20T10:28:56.791 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:28:56.791 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:28:56.848 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:28:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.067+0000 7fbdd5c00640 1 Processor -- start 2024-04-20T10:28:57.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.067+0000 7fbdd5c00640 1 -- start start 2024-04-20T10:28:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdd5c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd00ccd50 0x7fbdd00711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00ce790 0x7fbdd00cec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdd5c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbdd00cf180 0x7fbdd006f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdd5c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd0160dd0 con 0x7fbdd00ccd50 2024-04-20T10:28:57.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdd5c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbdd00cf5d0 con 0x7fbdd00cf180 2024-04-20T10:28:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdd5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd0160250 con 0x7fbdd00ce790 2024-04-20T10:28:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdcfe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd00ccd50 0x7fbdd00711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdcfe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbdd00ccd50 0x7fbdd00711b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59654/0 (socket says 172.21.15.47:59654) 2024-04-20T10:28:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.069+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1336085470 learned_addr learned my addr 172.21.15.47:0/1336085470 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.070+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1336085470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00ce790 msgr2=0x7fbdd00cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:57.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.070+0000 7fbdcfe00640 1 --2- 172.21.15.47:0/1336085470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00ce790 0x7fbdd00cec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.070+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1336085470 >> v2:172.21.15.175:3300/0 conn(0x7fbdd00cf180 msgr2=0x7fbdd006f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.070+0000 7fbdcfe00640 1 --2- 172.21.15.47:0/1336085470 >> v2:172.21.15.175:3300/0 conn(0x7fbdd00cf180 0x7fbdd006f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.070+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1336085470 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd0160530 con 0x7fbdd00ccd50 2024-04-20T10:28:57.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.071+0000 7fbdcfe00640 1 --2- 172.21.15.47:0/1336085470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00711b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00b19c0 tx=0x7fbdc002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e763d8bafad83e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:57.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.071+0000 7fbdcea00640 1 -- 172.21.15.47:0/1336085470 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0035020 con 0x7fbdd00ccd50 2024-04-20T10:28:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.071+0000 7fbdcea00640 1 -- 172.21.15.47:0/1336085470 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc0031870 con 0x7fbdd00ccd50 2024-04-20T10:28:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.071+0000 7fbdcea00640 1 -- 172.21.15.47:0/1336085470 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0031c40 con 0x7fbdd00ccd50 2024-04-20T10:28:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1336085470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 msgr2=0x7fbdd00711b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1336085470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00711b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00b19c0 tx=0x7fbdc002b910 comp rx=0 tx=0).stop 2024-04-20T10:28:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1336085470 shutdown_connections 2024-04-20T10:28:57.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1336085470 >> v2:172.21.15.175:3300/0 conn(0x7fbdd00cf180 0x7fbdd006f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1336085470 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00ce790 0x7fbdd00cec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1336085470 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00711b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1336085470 >> 172.21.15.47:0/1336085470 conn(0x7fbdd00b3df0 msgr2=0x7fbdd00b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1336085470 shutdown_connections 2024-04-20T10:28:57.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.072+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1336085470 wait complete. 2024-04-20T10:28:57.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.074+0000 7fbdd5c00640 1 Processor -- start 2024-04-20T10:28:57.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.074+0000 7fbdd5c00640 1 -- start start 2024-04-20T10:28:57.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00b7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd00ce790 0x7fbdd00c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdd5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00cf180 0x7fbdd00c74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdd5c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd00cd130 con 0x7fbdd00ccd50 2024-04-20T10:28:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdd5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbdd0160af0 con 0x7fbdd00ce790 2024-04-20T10:28:57.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdd5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbdd015feb0 con 0x7fbdd00cf180 2024-04-20T10:28:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00b7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdcfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00b7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59668/0 (socket says 172.21.15.47:59668) 2024-04-20T10:28:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1038091841 learned_addr learned my addr 172.21.15.47:0/1038091841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.076+0000 7fbdcf400640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd00ce790 0x7fbdd00c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdd5200640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00cf180 0x7fbdd00c74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00cf180 msgr2=0x7fbdd00c74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdcfe00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00cf180 0x7fbdd00c74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd00ce790 msgr2=0x7fbdd00c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdcfe00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd00ce790 0x7fbdd00c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdcfe00640 1 -- 172.21.15.47:0/1038091841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdd00c7cd0 con 0x7fbdd00ccd50 2024-04-20T10:28:57.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdd5200640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00cf180 0x7fbdd00c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdcfe00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00b7500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdc7e00640 1 -- 172.21.15.47:0/1038091841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00443f0 con 0x7fbdd00ccd50 2024-04-20T10:28:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdc7e00640 1 -- 172.21.15.47:0/1038091841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdc0044590 con 0x7fbdd00ccd50 2024-04-20T10:28:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdc7e00640 1 -- 172.21.15.47:0/1038091841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0044980 con 0x7fbdd00ccd50 2024-04-20T10:28:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.077+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdd00c7fb0 con 0x7fbdd00ccd50 2024-04-20T10:28:57.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.078+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdd00c8570 con 0x7fbdd00ccd50 2024-04-20T10:28:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.079+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdd00cdee0 con 0x7fbdd00ccd50 2024-04-20T10:28:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.080+0000 7fbdc7e00640 1 -- 172.21.15.47:0/1038091841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbdc004b060 con 0x7fbdd00ccd50 2024-04-20T10:28:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.081+0000 7fbdc7e00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbdb006c8f0 0x7fbdb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.081+0000 7fbdc7e00640 1 -- 172.21.15.47:0/1038091841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fbdc00ae210 con 0x7fbdd00ccd50 2024-04-20T10:28:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.082+0000 7fbdcf400640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbdb006c8f0 0x7fbdb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.083+0000 7fbdcf400640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbdb006c8f0 0x7fbdb006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00d1770 tx=0x7fbdbc0094f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:57.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.085+0000 7fbdc7e00640 1 -- 172.21.15.47:0/1038091841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbdd00cdee0 con 0x7fbdd00ccd50 2024-04-20T10:28:57.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.302+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbdd00d19a0 con 0x7fbdd00ccd50 2024-04-20T10:28:57.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.305+0000 7fbdc7e00640 1 -- 172.21.15.47:0/1038091841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7fbdc003f070 con 0x7fbdd00ccd50 2024-04-20T10:28:57.306 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:28:57.306 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbdb006c8f0 msgr2=0x7fbdb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbdb006c8f0 0x7fbdb006edb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00d1770 tx=0x7fbdbc0094f0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 msgr2=0x7fbdd00b7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00b7500 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0000c00 tx=0x7fbdc00046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 shutdown_connections 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbdd00cf180 0x7fbdd00c74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbdd00ce790 0x7fbdd00c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbdb006c8f0 0x7fbdb006edb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 --2- 172.21.15.47:0/1038091841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbdd00ccd50 0x7fbdd00b7500 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 >> 172.21.15.47:0/1038091841 conn(0x7fbdd00b3df0 msgr2=0x7fbdd015e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 shutdown_connections 2024-04-20T10:28:57.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.308+0000 7fbdd5c00640 1 -- 172.21.15.47:0/1038091841 wait complete. 2024-04-20T10:28:57.328 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:28:57.329 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:28:57.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.544+0000 7f8baaa00640 1 Processor -- start 2024-04-20T10:28:57.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.544+0000 7f8baaa00640 1 -- start start 2024-04-20T10:28:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.545+0000 7f8baaa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8ba4156c20 0x7f8ba406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8baaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40d0360 0x7f8ba40d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8baaa00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ba4141c60 con 0x7f8ba415ca40 2024-04-20T10:28:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8baaa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8ba40adab0 con 0x7f8ba4156c20 2024-04-20T10:28:57.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8baaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba415cf90 con 0x7f8ba40d0360 2024-04-20T10:28:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8b9be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ba415ca40 0x7f8ba40ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34730/0 (socket says 172.21.15.47:34730) 2024-04-20T10:28:57.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8b9be00640 1 -- 172.21.15.47:0/1485373211 learned_addr learned my addr 172.21.15.47:0/1485373211 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:28:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/1485373211 >> v2:172.21.15.175:3300/0 conn(0x7f8ba4156c20 0x7f8ba406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.546+0000 7f8ba8a00640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40d0360 0x7f8ba40d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 317481255 0 0) 0x7f8ba4141c60 con 0x7f8ba415ca40 2024-04-20T10:28:57.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8b88003520 con 0x7f8ba415ca40 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1722139249 0 0) 0x7f8b88003520 con 0x7f8ba415ca40 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40d0360 msgr2=0x7f8ba40d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40d0360 0x7f8ba40d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 >> v2:172.21.15.175:3300/0 conn(0x7f8ba4156c20 msgr2=0x7f8ba406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 --2- 172.21.15.47:0/1485373211 >> v2:172.21.15.175:3300/0 conn(0x7f8ba4156c20 0x7f8ba406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba415d270 con 0x7f8ba415ca40 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f8b8c001670 con 0x7f8ba415ca40 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 >> v1:172.21.15.47:6789/0 conn(0x7f8ba415ca40 legacy=0x7f8ba40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8b88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b88004800 0x7f8b88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40d0360 0x7f8b880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba40adab0 con 0x7f8b880039b0 2024-04-20T10:28:57.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba415cf90 con 0x7f8b88004800 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.547+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b8800b9c0 con 0x7f8ba40d0360 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.548+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8b88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.548+0000 7f8ba3e00640 1 -- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40d0360 msgr2=0x7f8b880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.548+0000 7f8b9be00640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b88004800 0x7f8b88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.548+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40d0360 0x7f8b880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.548+0000 7f8ba3e00640 1 -- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b88004800 msgr2=0x7f8b88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.548+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b88004800 0x7f8b88004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.548+0000 7f8ba3e00640 1 -- 172.21.15.47:0/1485373211 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b88003520 con 0x7f8b880039b0 2024-04-20T10:28:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.549+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8b88003dd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009ac0 tx=0x7f8b88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7836441484b48dd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.549+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94039070 con 0x7f8b880039b0 2024-04-20T10:28:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.549+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9402c710 con 0x7f8b880039b0 2024-04-20T10:28:57.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.549+0000 7f8ba3400640 1 -- 172.21.15.47:0/1485373211 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94009690 con 0x7f8b880039b0 2024-04-20T10:28:57.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.550+0000 7f8baaa00640 1 -- 172.21.15.47:0/1485373211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 msgr2=0x7f8b88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.550+0000 7f8baaa00640 1 --2- 172.21.15.47:0/1485373211 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8b88003dd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f8b94009ac0 tx=0x7f8b88005830 comp rx=0 tx=0).stop 2024-04-20T10:28:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.550+0000 7f8ba8a00640 1 -- 172.21.15.47:0/1485373211 reap_dead start 2024-04-20T10:28:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.550+0000 7f8baaa00640 1 -- 172.21.15.47:0/1485373211 shutdown_connections 2024-04-20T10:28:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.550+0000 7f8baaa00640 1 -- 172.21.15.47:0/1485373211 >> 172.21.15.47:0/1485373211 conn(0x7f8ba40b3350 msgr2=0x7f8ba40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.550+0000 7f8baaa00640 1 -- 172.21.15.47:0/1485373211 shutdown_connections 2024-04-20T10:28:57.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.550+0000 7f8baaa00640 1 -- 172.21.15.47:0/1485373211 wait complete. 2024-04-20T10:28:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.552+0000 7f8baaa00640 1 Processor -- start 2024-04-20T10:28:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.552+0000 7f8baaa00640 1 -- start start 2024-04-20T10:28:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.553+0000 7f8baaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8ba41f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.553+0000 7f8baaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40ceb80 0x7f8ba40cf000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.553+0000 7f8baaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba4073250 0x7f8ba40736d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8baaa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba406be40 con 0x7f8b880039b0 2024-04-20T10:28:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8baaa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba415d830 con 0x7f8ba4073250 2024-04-20T10:28:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8baaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba406bfc0 con 0x7f8ba40ceb80 2024-04-20T10:28:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8ba41f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8ba41f4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59678/0 (socket says 172.21.15.47:59678) 2024-04-20T10:28:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 -- 172.21.15.47:0/212711623 learned_addr learned my addr 172.21.15.47:0/212711623 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 -- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40ceb80 msgr2=0x7f8ba40cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40ceb80 0x7f8ba40cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 -- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba4073250 msgr2=0x7f8ba40736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba4073250 0x7f8ba40736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.554+0000 7f8ba3e00640 1 -- 172.21.15.47:0/212711623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba40cf6f0 con 0x7f8b880039b0 2024-04-20T10:28:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.555+0000 7f8ba3e00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8ba41f4a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b94037040 tx=0x7f8b94004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.555+0000 7f8ba0c00640 1 -- 172.21.15.47:0/212711623 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94042020 con 0x7f8b880039b0 2024-04-20T10:28:57.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.555+0000 7f8ba0c00640 1 -- 172.21.15.47:0/212711623 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8b9403a040 con 0x7f8b880039b0 2024-04-20T10:28:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.555+0000 7f8ba0c00640 1 -- 172.21.15.47:0/212711623 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94039070 con 0x7f8b880039b0 2024-04-20T10:28:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.555+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba414d4b0 con 0x7f8b880039b0 2024-04-20T10:28:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.555+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba414da70 con 0x7f8b880039b0 2024-04-20T10:28:57.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.556+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba4068980 con 0x7f8b880039b0 2024-04-20T10:28:57.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.557+0000 7f8ba0c00640 1 -- 172.21.15.47:0/212711623 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8b94045020 con 0x7f8b880039b0 2024-04-20T10:28:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.559+0000 7f8ba0c00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b8006c920 0x7f8b8006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.559+0000 7f8ba0c00640 1 -- 172.21.15.47:0/212711623 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f8b940ad280 con 0x7f8b880039b0 2024-04-20T10:28:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.559+0000 7f8b9be00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b8006c920 0x7f8b8006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:57.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.561+0000 7f8b9be00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b8006c920 0x7f8b8006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000f60 tx=0x7f8b8c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:57.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.563+0000 7f8ba0c00640 1 -- 172.21.15.47:0/212711623 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8b9404c070 con 0x7f8b880039b0 2024-04-20T10:28:57.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.770+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ba415df60 con 0x7f8b880039b0 2024-04-20T10:28:57.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.771+0000 7f8ba0c00640 1 -- 172.21.15.47:0/212711623 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f8b9407b260 con 0x7f8b880039b0 2024-04-20T10:28:57.771 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:28:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.774+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b8006c920 msgr2=0x7f8b8006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.774+0000 7f8baaa00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b8006c920 0x7f8b8006ede0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c000f60 tx=0x7f8b8c009040 comp rx=0 tx=0).stop 2024-04-20T10:28:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.774+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 msgr2=0x7f8ba41f4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:57.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.774+0000 7f8baaa00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8ba41f4a20 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f8b94037040 tx=0x7f8b94004730 comp rx=0 tx=0).stop 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 shutdown_connections 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba4073250 0x7f8ba40736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba40ceb80 0x7f8ba40cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8b8006c920 0x7f8b8006ede0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 --2- 172.21.15.47:0/212711623 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8b880039b0 0x7f8ba41f4a20 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 >> 172.21.15.47:0/212711623 conn(0x7f8ba40b3350 msgr2=0x7f8ba415e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 shutdown_connections 2024-04-20T10:28:57.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:57.775+0000 7f8baaa00640 1 -- 172.21.15.47:0/212711623 wait complete. 2024-04-20T10:28:57.795 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:28:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.008+0000 7f63f2800640 1 Processor -- start 2024-04-20T10:28:58.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.008+0000 7f63f2800640 1 -- start start 2024-04-20T10:28:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.009+0000 7f63f2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec0aee60 0x7f63ec0af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.009+0000 7f63f2800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f63ec06e060 0x7f63ec06e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.009+0000 7f63f2800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f63ec0af780 con 0x7f63ec15d4f0 2024-04-20T10:28:58.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.009+0000 7f63f2800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f63ec06c510 con 0x7f63ec06e060 2024-04-20T10:28:58.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.009+0000 7f63f2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec06b900 con 0x7f63ec0aee60 2024-04-20T10:28:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.010+0000 7f63f1e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f63ec06e060 0x7f63ec06e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.010+0000 7f63f1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec0aee60 0x7f63ec0af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.010+0000 7f63f1e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f63ec06e060 0x7f63ec06e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:53944/0 (socket says 172.21.15.47:53944) 2024-04-20T10:28:58.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.010+0000 7f63f1e00640 1 -- 172.21.15.47:0/3739482198 learned_addr learned my addr 172.21.15.47:0/3739482198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.011+0000 7f63f1400640 1 -- 172.21.15.47:0/3739482198 >> v2:172.21.15.175:3300/0 conn(0x7f63ec06e060 msgr2=0x7f63ec06e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.011+0000 7f63f1400640 1 --2- 172.21.15.47:0/3739482198 >> v2:172.21.15.175:3300/0 conn(0x7f63ec06e060 0x7f63ec06e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.011+0000 7f63f1400640 1 -- 172.21.15.47:0/3739482198 >> v1:172.21.15.175:6789/0 conn(0x7f63ec15d4f0 legacy=0x7f63ec0ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.011+0000 7f63f1400640 1 -- 172.21.15.47:0/3739482198 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec06bbe0 con 0x7f63ec0aee60 2024-04-20T10:28:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.013+0000 7f63f1400640 1 --2- 172.21.15.47:0/3739482198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec0aee60 0x7f63ec0af240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097a0 tx=0x7f63dc02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6325914fe2395d6c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.013+0000 7f63ebe00640 1 -- 172.21.15.47:0/3739482198 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc035020 con 0x7f63ec0aee60 2024-04-20T10:28:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.013+0000 7f63ebe00640 1 -- 172.21.15.47:0/3739482198 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63dc031830 con 0x7f63ec0aee60 2024-04-20T10:28:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.013+0000 7f63ebe00640 1 -- 172.21.15.47:0/3739482198 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63dc031c00 con 0x7f63ec0aee60 2024-04-20T10:28:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 -- 172.21.15.47:0/3739482198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec0aee60 msgr2=0x7f63ec0af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 --2- 172.21.15.47:0/3739482198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec0aee60 0x7f63ec0af240 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097a0 tx=0x7f63dc02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 -- 172.21.15.47:0/3739482198 shutdown_connections 2024-04-20T10:28:58.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 --2- 172.21.15.47:0/3739482198 >> v2:172.21.15.175:3300/0 conn(0x7f63ec06e060 0x7f63ec06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 --2- 172.21.15.47:0/3739482198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec0aee60 0x7f63ec0af240 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 -- 172.21.15.47:0/3739482198 >> 172.21.15.47:0/3739482198 conn(0x7f63ec0b5270 msgr2=0x7f63ec0b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 -- 172.21.15.47:0/3739482198 shutdown_connections 2024-04-20T10:28:58.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.014+0000 7f63f2800640 1 -- 172.21.15.47:0/3739482198 wait complete. 2024-04-20T10:28:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.016+0000 7f63f2800640 1 Processor -- start 2024-04-20T10:28:58.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.016+0000 7f63f2800640 1 -- start start 2024-04-20T10:28:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec06e060 0x7f63ec1f3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ec0aee60 0x7f63ec1f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f2800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec15d4f0 0x7f63ec15ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f2800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec06c9a0 con 0x7f63ec15d4f0 2024-04-20T10:28:58.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63ec06c1a0 con 0x7f63ec0aee60 2024-04-20T10:28:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63ec06ea00 con 0x7f63ec06e060 2024-04-20T10:28:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec15d4f0 0x7f63ec15ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec15d4f0 0x7f63ec15ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59690/0 (socket says 172.21.15.47:59690) 2024-04-20T10:28:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.017+0000 7f63f1e00640 1 -- 172.21.15.47:0/3807797285 learned_addr learned my addr 172.21.15.47:0/3807797285 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f1400640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec06e060 0x7f63ec1f3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f0a00640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ec0aee60 0x7f63ec1f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f1e00640 1 -- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec06e060 msgr2=0x7f63ec1f3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f1e00640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec06e060 0x7f63ec1f3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f1e00640 1 -- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ec0aee60 msgr2=0x7f63ec1f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f1e00640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ec0aee60 0x7f63ec1f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f1e00640 1 -- 172.21.15.47:0/3807797285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63ec152590 con 0x7f63ec15d4f0 2024-04-20T10:28:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.018+0000 7f63f1e00640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec15d4f0 0x7f63ec15ea10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009cf0 tx=0x7f63e400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.019+0000 7f63e9600640 1 -- 172.21.15.47:0/3807797285 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e400cd20 con 0x7f63ec15d4f0 2024-04-20T10:28:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.019+0000 7f63e9600640 1 -- 172.21.15.47:0/3807797285 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f63e4003990 con 0x7f63ec15d4f0 2024-04-20T10:28:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.019+0000 7f63e9600640 1 -- 172.21.15.47:0/3807797285 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63e4003d80 con 0x7f63ec15d4f0 2024-04-20T10:28:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.019+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63ec1e8070 con 0x7f63ec15d4f0 2024-04-20T10:28:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.019+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63ec1e8630 con 0x7f63ec15d4f0 2024-04-20T10:28:58.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.020+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63ec1e8830 con 0x7f63ec15d4f0 2024-04-20T10:28:58.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.021+0000 7f63e9600640 1 -- 172.21.15.47:0/3807797285 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f63e4012050 con 0x7f63ec15d4f0 2024-04-20T10:28:58.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.022+0000 7f63e9600640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63cc06c8f0 0x7f63cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.023+0000 7f63e9600640 1 -- 172.21.15.47:0/3807797285 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f63e408c3c0 con 0x7f63ec15d4f0 2024-04-20T10:28:58.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.023+0000 7f63f1400640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63cc06c8f0 0x7f63cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.024+0000 7f63f1400640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63cc06c8f0 0x7f63cc06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63dc036000 tx=0x7f63dc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.027+0000 7f63e9600640 1 -- 172.21.15.47:0/3807797285 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f63ec1e8830 con 0x7f63ec15d4f0 2024-04-20T10:28:58.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.236+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f63ec075b30 con 0x7f63ec15d4f0 2024-04-20T10:28:58.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.237+0000 7f63e9600640 1 -- 172.21.15.47:0/3807797285 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f63e4010040 con 0x7f63ec15d4f0 2024-04-20T10:28:58.237 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63cc06c8f0 msgr2=0x7f63cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63cc06c8f0 0x7f63cc06edb0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f63dc036000 tx=0x7f63dc008030 comp rx=0 tx=0).stop 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec15d4f0 msgr2=0x7f63ec15ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec15d4f0 0x7f63ec15ea10 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f63e4009cf0 tx=0x7f63e400c490 comp rx=0 tx=0).stop 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 shutdown_connections 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f63cc06c8f0 0x7f63cc06edb0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f63ec15d4f0 0x7f63ec15ea10 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63ec0aee60 0x7f63ec1f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 --2- 172.21.15.47:0/3807797285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63ec06e060 0x7f63ec1f3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.240+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 >> 172.21.15.47:0/3807797285 conn(0x7f63ec0b5270 msgr2=0x7f63ec0afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.241+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 shutdown_connections 2024-04-20T10:28:58.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.241+0000 7f63f2800640 1 -- 172.21.15.47:0/3807797285 wait complete. 2024-04-20T10:28:58.262 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:28:58.377 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:58.386 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:28:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.474+0000 7f5fd3e00640 1 Processor -- start 2024-04-20T10:28:58.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.474+0000 7f5fd3e00640 1 -- start start 2024-04-20T10:28:58.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.476+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0d4270 0x7f5fcc0d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.476+0000 7f5fd3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fcc0d4650 0x7f5fcc0b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.476+0000 7f5fd3e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5fcc0d3820 con 0x7f5fcc0b17d0 2024-04-20T10:28:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.476+0000 7f5fd3e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5fcc0d40c0 con 0x7f5fcc0d4650 2024-04-20T10:28:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.476+0000 7f5fd3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fcc0d3b00 con 0x7f5fcc0d4270 2024-04-20T10:28:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd0a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fcc0d4650 0x7f5fcc0b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd0a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5fcc0d4650 0x7f5fcc0b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:53962/0 (socket says 172.21.15.47:53962) 2024-04-20T10:28:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd0a00640 1 -- 172.21.15.47:0/279911018 learned_addr learned my addr 172.21.15.47:0/279911018 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:58.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd1400640 1 --2- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0d4270 0x7f5fcc0d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd0a00640 1 -- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0d4270 msgr2=0x7f5fcc0d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd0a00640 1 --2- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0d4270 0x7f5fcc0d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd0a00640 1 -- 172.21.15.47:0/279911018 >> v1:172.21.15.175:6789/0 conn(0x7f5fcc0b17d0 legacy=0x7f5fcc0d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.477+0000 7f5fd0a00640 1 -- 172.21.15.47:0/279911018 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc0d3de0 con 0x7f5fcc0d4650 2024-04-20T10:28:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.478+0000 7f5fd0a00640 1 --2- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc0b1290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009870 tx=0x7f5fb802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca50f924eae953b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.479+0000 7f5fcbe00640 1 -- 172.21.15.47:0/279911018 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8034070 con 0x7f5fcc0d4650 2024-04-20T10:28:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.479+0000 7f5fcbe00640 1 -- 172.21.15.47:0/279911018 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fb8030840 con 0x7f5fcc0d4650 2024-04-20T10:28:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.479+0000 7f5fcbe00640 1 -- 172.21.15.47:0/279911018 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8030c10 con 0x7f5fcc0d4650 2024-04-20T10:28:58.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 -- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4650 msgr2=0x7f5fcc0b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc0b1290 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8009870 tx=0x7f5fb802b980 comp rx=0 tx=0).stop 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 -- 172.21.15.47:0/279911018 shutdown_connections 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc0b1290 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/279911018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0d4270 0x7f5fcc0d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 -- 172.21.15.47:0/279911018 >> 172.21.15.47:0/279911018 conn(0x7f5fcc0b99c0 msgr2=0x7f5fcc0b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 -- 172.21.15.47:0/279911018 shutdown_connections 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.480+0000 7f5fd3e00640 1 -- 172.21.15.47:0/279911018 wait complete. 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.482+0000 7f5fd3e00640 1 Processor -- start 2024-04-20T10:28:58.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.482+0000 7f5fd3e00640 1 -- start start 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.483+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0b17d0 0x7f5fcc1f4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.483+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4270 0x7f5fcc06dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.483+0000 7f5fd3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc15df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.483+0000 7f5fd3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc161f70 con 0x7f5fcc0d4650 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5fcc0d53b0 con 0x7f5fcc0d4270 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5fcc1620f0 con 0x7f5fcc0b17d0 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc15df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc15df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59692/0 (socket says 172.21.15.47:59692) 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 -- 172.21.15.47:0/924747483 learned_addr learned my addr 172.21.15.47:0/924747483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1400640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0b17d0 0x7f5fcc1f4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd0a00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4270 0x7f5fcc06dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 -- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0b17d0 msgr2=0x7f5fcc1f4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0b17d0 0x7f5fcc1f4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 -- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4270 msgr2=0x7f5fcc06dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4270 0x7f5fcc06dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.484+0000 7f5fd1e00640 1 -- 172.21.15.47:0/924747483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fcc06e290 con 0x7f5fcc0d4650 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.485+0000 7f5fd1e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc15df60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c730 tx=0x7f5fc400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.485+0000 7f5fc9600640 1 -- 172.21.15.47:0/924747483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4003b70 con 0x7f5fcc0d4650 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.485+0000 7f5fc9600640 1 -- 172.21.15.47:0/924747483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fc4003d10 con 0x7f5fcc0d4650 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.485+0000 7f5fc9600640 1 -- 172.21.15.47:0/924747483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc400aa90 con 0x7f5fcc0d4650 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.485+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fcc06e560 con 0x7f5fcc0d4650 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.485+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fcc0c8ea0 con 0x7f5fcc0d4650 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.486+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fcc0cf530 con 0x7f5fcc0d4650 2024-04-20T10:28:58.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.488+0000 7f5fc9600640 1 -- 172.21.15.47:0/924747483 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5fc400ac30 con 0x7f5fcc0d4650 2024-04-20T10:28:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.488+0000 7f5fc9600640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5fac06c8f0 0x7f5fac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.489+0000 7f5fc9600640 1 -- 172.21.15.47:0/924747483 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f5fc408bdc0 con 0x7f5fcc0d4650 2024-04-20T10:28:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.489+0000 7f5fd1400640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5fac06c8f0 0x7f5fac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.490+0000 7f5fd1400640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5fac06c8f0 0x7f5fac06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0b1eb0 tx=0x7f5fc0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.493+0000 7f5fc9600640 1 -- 172.21.15.47:0/924747483 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5fcc0cf530 con 0x7f5fcc0d4650 2024-04-20T10:28:58.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.695+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f5fcc068980 con 0x7f5fcc0d4650 2024-04-20T10:28:58.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.696+0000 7f5fc9600640 1 -- 172.21.15.47:0/924747483 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f5fc4059e30 con 0x7f5fcc0d4650 2024-04-20T10:28:58.696 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.698+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5fac06c8f0 msgr2=0x7f5fac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.698+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5fac06c8f0 0x7f5fac06edb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc0b1eb0 tx=0x7f5fc0008040 comp rx=0 tx=0).stop 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.698+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fcc0d4650 msgr2=0x7f5fcc15df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.698+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc15df60 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400c730 tx=0x7f5fc400cbf0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 shutdown_connections 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5fac06c8f0 0x7f5fac06edb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5fcc0d4650 0x7f5fcc15df60 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5fcc0d4270 0x7f5fcc06dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 --2- 172.21.15.47:0/924747483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5fcc0b17d0 0x7f5fcc1f4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 >> 172.21.15.47:0/924747483 conn(0x7f5fcc0b99c0 msgr2=0x7f5fcc0b30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:58.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 shutdown_connections 2024-04-20T10:28:58.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.699+0000 7f5fd3e00640 1 -- 172.21.15.47:0/924747483 wait complete. 2024-04-20T10:28:58.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:28:58.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.924+0000 7fad98c00640 1 Processor -- start 2024-04-20T10:28:58.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.924+0000 7fad98c00640 1 -- start start 2024-04-20T10:28:58.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.925+0000 7fad98c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fad9415b1f0 0x7fad9415f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.925+0000 7fad98c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fad9406ca10 0x7fad9406ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.925+0000 7fad98c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fad9415b5d0 con 0x7fad940d0010 2024-04-20T10:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.925+0000 7fad98c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fad9406d3a0 con 0x7fad9415b1f0 2024-04-20T10:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.925+0000 7fad98c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fad9406e730 con 0x7fad9406ca10 2024-04-20T10:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fad9415b1f0 0x7fad9415f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fad9415b1f0 0x7fad9415f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59696/0 (socket says 172.21.15.47:59696) 2024-04-20T10:28:58.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93400640 1 -- 172.21.15.47:0/3392745733 learned_addr learned my addr 172.21.15.47:0/3392745733 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93e00640 1 --2- 172.21.15.47:0/3392745733 >> v2:172.21.15.175:3300/0 conn(0x7fad9406ca10 0x7fad9406ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93400640 1 -- 172.21.15.47:0/3392745733 >> v2:172.21.15.175:3300/0 conn(0x7fad9406ca10 msgr2=0x7fad9406ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93400640 1 --2- 172.21.15.47:0/3392745733 >> v2:172.21.15.175:3300/0 conn(0x7fad9406ca10 0x7fad9406ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93400640 1 -- 172.21.15.47:0/3392745733 >> v1:172.21.15.175:6789/0 conn(0x7fad940d0010 legacy=0x7fad940adc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93400640 1 -- 172.21.15.47:0/3392745733 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad9415fc70 con 0x7fad9415b1f0 2024-04-20T10:28:58.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.926+0000 7fad93e00640 1 --2- 172.21.15.47:0/3392745733 >> v2:172.21.15.175:3300/0 conn(0x7fad9406ca10 0x7fad9406ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:58.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.927+0000 7fad93400640 1 --2- 172.21.15.47:0/3392745733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9415b1f0 0x7fad9415f730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fad84009870 tx=0x7fad8402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5299bff49123fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad92a00640 1 -- 172.21.15.47:0/3392745733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad840317d0 con 0x7fad9415b1f0 2024-04-20T10:28:58.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad92a00640 1 -- 172.21.15.47:0/3392745733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84037070 con 0x7fad9415b1f0 2024-04-20T10:28:58.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad92a00640 1 -- 172.21.15.47:0/3392745733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84031b90 con 0x7fad9415b1f0 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad98c00640 1 -- 172.21.15.47:0/3392745733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9415b1f0 msgr2=0x7fad9415f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad98c00640 1 --2- 172.21.15.47:0/3392745733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9415b1f0 0x7fad9415f730 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fad84009870 tx=0x7fad8402b870 comp rx=0 tx=0).stop 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad98c00640 1 -- 172.21.15.47:0/3392745733 shutdown_connections 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad98c00640 1 --2- 172.21.15.47:0/3392745733 >> v2:172.21.15.175:3300/0 conn(0x7fad9406ca10 0x7fad9406ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad98c00640 1 --2- 172.21.15.47:0/3392745733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9415b1f0 0x7fad9415f730 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.928+0000 7fad98c00640 1 -- 172.21.15.47:0/3392745733 >> 172.21.15.47:0/3392745733 conn(0x7fad940b5270 msgr2=0x7fad940b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.929+0000 7fad98c00640 1 -- 172.21.15.47:0/3392745733 shutdown_connections 2024-04-20T10:28:58.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.929+0000 7fad98c00640 1 -- 172.21.15.47:0/3392745733 wait complete. 2024-04-20T10:28:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.930+0000 7fad98c00640 1 Processor -- start 2024-04-20T10:28:58.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.930+0000 7fad98c00640 1 -- start start 2024-04-20T10:28:58.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.931+0000 7fad98c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9406ca10 0x7fad9415c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.931+0000 7fad98c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad940d0010 0x7fad9414f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.931+0000 7fad98c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9415de70 0x7fad941f7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.931+0000 7fad98c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fad940d08b0 con 0x7fad9406ca10 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.931+0000 7fad98c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fad940d05d0 con 0x7fad940d0010 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.931+0000 7fad98c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fad940d0b90 con 0x7fad9415de70 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9406ca10 0x7fad9415c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9406ca10 0x7fad9415c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59712/0 (socket says 172.21.15.47:59712) 2024-04-20T10:28:58.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 -- 172.21.15.47:0/950203994 learned_addr learned my addr 172.21.15.47:0/950203994 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:58.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad8be00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad940d0010 0x7fad9414f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93e00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9415de70 0x7fad941f7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 -- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9415de70 msgr2=0x7fad941f7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9415de70 0x7fad941f7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 -- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad940d0010 msgr2=0x7fad9414f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad940d0010 0x7fad9414f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.932+0000 7fad93400640 1 -- 172.21.15.47:0/950203994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad941500a0 con 0x7fad9406ca10 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.933+0000 7fad93400640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9406ca10 0x7fad9415c7f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad84000c00 tx=0x7fad840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.933+0000 7fad8b400640 1 -- 172.21.15.47:0/950203994 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84032870 con 0x7fad9406ca10 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.933+0000 7fad8b400640 1 -- 172.21.15.47:0/950203994 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fad84032a30 con 0x7fad9406ca10 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.933+0000 7fad8b400640 1 -- 172.21.15.47:0/950203994 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad84032e20 con 0x7fad9406ca10 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.933+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad9415cdd0 con 0x7fad9406ca10 2024-04-20T10:28:58.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.933+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad9415d390 con 0x7fad9406ca10 2024-04-20T10:28:58.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.934+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad94075b30 con 0x7fad9406ca10 2024-04-20T10:28:58.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.936+0000 7fad8b400640 1 -- 172.21.15.47:0/950203994 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fad8404c060 con 0x7fad9406ca10 2024-04-20T10:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.937+0000 7fad8b400640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fad7006c8f0 0x7fad7006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:58.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.937+0000 7fad8b400640 1 -- 172.21.15.47:0/950203994 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fad840ad7f0 con 0x7fad9406ca10 2024-04-20T10:28:58.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.938+0000 7fad8be00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fad7006c8f0 0x7fad7006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:58.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.939+0000 7fad8be00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fad7006c8f0 0x7fad7006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad7c007fd0 tx=0x7fad7c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:58.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:58.943+0000 7fad8b400640 1 -- 172.21.15.47:0/950203994 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fad941500a0 con 0x7fad9406ca10 2024-04-20T10:28:59.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.146+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fad94072860 con 0x7fad9406ca10 2024-04-20T10:28:59.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.147+0000 7fad8b400640 1 -- 172.21.15.47:0/950203994 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7fad84040070 con 0x7fad9406ca10 2024-04-20T10:28:59.147 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:28:59.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fad7006c8f0 msgr2=0x7fad7006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:59.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fad7006c8f0 0x7fad7006edb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fad7c007fd0 tx=0x7fad7c009040 comp rx=0 tx=0).stop 2024-04-20T10:28:59.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9406ca10 msgr2=0x7fad9415c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:59.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9406ca10 0x7fad9415c7f0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fad84000c00 tx=0x7fad840046c0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 shutdown_connections 2024-04-20T10:28:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fad9415de70 0x7fad941f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fad940d0010 0x7fad9414f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fad7006c8f0 0x7fad7006edb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 --2- 172.21.15.47:0/950203994 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fad9406ca10 0x7fad9415c7f0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 >> 172.21.15.47:0/950203994 conn(0x7fad940b5270 msgr2=0x7fad9406b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 shutdown_connections 2024-04-20T10:28:59.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.150+0000 7fad98c00640 1 -- 172.21.15.47:0/950203994 wait complete. 2024-04-20T10:28:59.170 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:28:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.385+0000 7fc2b1800640 1 Processor -- start 2024-04-20T10:28:59.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.385+0000 7fc2b1800640 1 -- start start 2024-04-20T10:28:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2b1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2b1800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2ac15eec0 0x7fc2ac15f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2b1800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc2ac15f840 0x7fc2ac15b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2b1800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac06d280 con 0x7fc2ac15eec0 2024-04-20T10:28:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2b1800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc2ac06ccc0 con 0x7fc2ac15f840 2024-04-20T10:28:59.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2b1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac06cfa0 con 0x7fc2ac15b400 2024-04-20T10:28:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2a3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2ac15eec0 0x7fc2ac15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2a3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc2ac15eec0 0x7fc2ac15f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59720/0 (socket says 172.21.15.47:59720) 2024-04-20T10:28:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.387+0000 7fc2a3400640 1 -- 172.21.15.47:0/1540024711 learned_addr learned my addr 172.21.15.47:0/1540024711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:59.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2b0e00640 1 --2- 172.21.15.47:0/1540024711 >> v2:172.21.15.175:3300/0 conn(0x7fc2ac15f840 0x7fc2ac15b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2abe00640 1 --2- 172.21.15.47:0/1540024711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2a3400640 1 -- 172.21.15.47:0/1540024711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 msgr2=0x7fc2ac072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2a3400640 1 --2- 172.21.15.47:0/1540024711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2a3400640 1 -- 172.21.15.47:0/1540024711 >> v2:172.21.15.175:3300/0 conn(0x7fc2ac15f840 msgr2=0x7fc2ac15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2a3400640 1 --2- 172.21.15.47:0/1540024711 >> v2:172.21.15.175:3300/0 conn(0x7fc2ac15f840 0x7fc2ac15b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2a3400640 1 -- 172.21.15.47:0/1540024711 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac06e440 con 0x7fc2ac15eec0 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2b0e00640 1 --2- 172.21.15.47:0/1540024711 >> v2:172.21.15.175:3300/0 conn(0x7fc2ac15f840 0x7fc2ac15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:59.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.388+0000 7fc2abe00640 1 --2- 172.21.15.47:0/1540024711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:28:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.389+0000 7fc2a3400640 1 --2- 172.21.15.47:0/1540024711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac15eec0 0x7fc2ac15f300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac074a40 tx=0x7fc29402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=126e2a73339172cc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.389+0000 7fc2ab400640 1 -- 172.21.15.47:0/1540024711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294035020 con 0x7fc2ac15eec0 2024-04-20T10:28:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.389+0000 7fc2ab400640 1 -- 172.21.15.47:0/1540024711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc294031870 con 0x7fc2ac15eec0 2024-04-20T10:28:59.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.389+0000 7fc2ab400640 1 -- 172.21.15.47:0/1540024711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc294031c40 con 0x7fc2ac15eec0 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 -- 172.21.15.47:0/1540024711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac15eec0 msgr2=0x7fc2ac15f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 --2- 172.21.15.47:0/1540024711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac15eec0 0x7fc2ac15f300 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac074a40 tx=0x7fc29402b910 comp rx=0 tx=0).stop 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 -- 172.21.15.47:0/1540024711 shutdown_connections 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 --2- 172.21.15.47:0/1540024711 >> v2:172.21.15.175:3300/0 conn(0x7fc2ac15f840 0x7fc2ac15b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 --2- 172.21.15.47:0/1540024711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac15eec0 0x7fc2ac15f300 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 --2- 172.21.15.47:0/1540024711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 -- 172.21.15.47:0/1540024711 >> 172.21.15.47:0/1540024711 conn(0x7fc2ac0d0040 msgr2=0x7fc2ac0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 -- 172.21.15.47:0/1540024711 shutdown_connections 2024-04-20T10:28:59.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.390+0000 7fc2b1800640 1 -- 172.21.15.47:0/1540024711 wait complete. 2024-04-20T10:28:59.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.392+0000 7fc2b1800640 1 Processor -- start 2024-04-20T10:28:59.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.392+0000 7fc2b1800640 1 -- start start 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.393+0000 7fc2b1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac14db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.393+0000 7fc2b1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2ac15f840 0x7fc2ac0b47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.393+0000 7fc2b1800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac0b3040 0x7fc2ac09a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.393+0000 7fc2b1800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac073240 con 0x7fc2ac0b3040 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.393+0000 7fc2b1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac15fc90 con 0x7fc2ac15f840 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.393+0000 7fc2b1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2ac15bd20 con 0x7fc2ac15b400 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac0b3040 0x7fc2ac09a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac0b3040 0x7fc2ac09a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59736/0 (socket says 172.21.15.47:59736) 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 -- 172.21.15.47:0/2454616840 learned_addr learned my addr 172.21.15.47:0/2454616840 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2a3400640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2ac15f840 0x7fc2ac0b47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:59.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2abe00640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac14db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:59.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 -- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 msgr2=0x7fc2ac14db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac14db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 -- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2ac15f840 msgr2=0x7fc2ac0b47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2ac15f840 0x7fc2ac0b47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.394+0000 7fc2b0e00640 1 -- 172.21.15.47:0/2454616840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac0b39f0 con 0x7fc2ac0b3040 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.395+0000 7fc2b0e00640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac0b3040 0x7fc2ac09a9f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400cd50 tx=0x7fc2a400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.395+0000 7fc2a8c00640 1 -- 172.21.15.47:0/2454616840 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400f680 con 0x7fc2ac0b3040 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.395+0000 7fc2a8c00640 1 -- 172.21.15.47:0/2454616840 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2a400f840 con 0x7fc2ac0b3040 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.395+0000 7fc2a8c00640 1 -- 172.21.15.47:0/2454616840 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400fc30 con 0x7fc2ac0b3040 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.395+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac0b3cd0 con 0x7fc2ac0b3040 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.395+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac0b4290 con 0x7fc2ac0b3040 2024-04-20T10:28:59.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.396+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac075b60 con 0x7fc2ac0b3040 2024-04-20T10:28:59.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.397+0000 7fc2a8c00640 1 -- 172.21.15.47:0/2454616840 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc2a400e3e0 con 0x7fc2ac0b3040 2024-04-20T10:28:59.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.398+0000 7fc2a8c00640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc28806c8f0 0x7fc28806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:28:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.398+0000 7fc2a8c00640 1 -- 172.21.15.47:0/2454616840 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fc2a408cc40 con 0x7fc2ac0b3040 2024-04-20T10:28:59.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.399+0000 7fc2abe00640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc28806c8f0 0x7fc28806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:28:59.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.400+0000 7fc2abe00640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc28806c8f0 0x7fc28806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc29c006b10 tx=0x7fc29c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:28:59.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.403+0000 7fc2a8c00640 1 -- 172.21.15.47:0/2454616840 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc2a4014030 con 0x7fc2ac0b3040 2024-04-20T10:28:59.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.625+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc2ac0b5220 con 0x7fc2ac0b3040 2024-04-20T10:28:59.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.626+0000 7fc2a8c00640 1 -- 172.21.15.47:0/2454616840 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v29) ==== 80+0+0 (secure 0 0 0) 0x7fc2a40152b0 con 0x7fc2ac0b3040 2024-04-20T10:28:59.626 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc28806c8f0 msgr2=0x7fc28806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc28806c8f0 0x7fc28806edb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc29c006b10 tx=0x7fc29c008040 comp rx=0 tx=0).stop 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac0b3040 msgr2=0x7fc2ac09a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac0b3040 0x7fc2ac09a9f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400cd50 tx=0x7fc2a400a650 comp rx=0 tx=0).stop 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 shutdown_connections 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc28806c8f0 0x7fc28806edb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc2ac0b3040 0x7fc2ac09a9f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2ac15f840 0x7fc2ac0b47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 --2- 172.21.15.47:0/2454616840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2ac15b400 0x7fc2ac14db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 >> 172.21.15.47:0/2454616840 conn(0x7fc2ac0d0040 msgr2=0x7fc2ac0722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 shutdown_connections 2024-04-20T10:28:59.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:28:59.629+0000 7fc2b1800640 1 -- 172.21.15.47:0/2454616840 wait complete. 2024-04-20T10:28:59.649 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi175.front.sepia.ceph.com... 2024-04-20T10:28:59.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:59.649 DEBUG:teuthology.orchestra.run.smithi175:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2024-04-20T10:28:59.739 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:28:59.739 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:28:59.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:28:59.739 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T10:28:59.790 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T10:28:59.790 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.009+0000 7f04d7000640 1 Processor -- start 2024-04-20T10:29:00.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.009+0000 7f04d7000640 1 -- start start 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d015b1b0 0x7f04d006dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d7000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04d015b590 0x7f04d00b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d7000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f04d00b4720 0x7f04d00730b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d7000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f04d006d2a0 con 0x7f04d015b590 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d7000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f04d00b4d50 con 0x7f04d00b4720 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04d00b5030 con 0x7f04d015b1b0 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d5200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04d015b590 0x7f04d00b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d5200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f04d015b590 0x7f04d00b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59750/0 (socket says 172.21.15.47:59750) 2024-04-20T10:29:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.011+0000 7f04d5200640 1 -- 172.21.15.47:0/1724747038 learned_addr learned my addr 172.21.15.47:0/1724747038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d5c00640 1 --2- 172.21.15.47:0/1724747038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d015b1b0 0x7f04d006dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d6600640 1 --2- 172.21.15.47:0/1724747038 >> v2:172.21.15.175:3300/0 conn(0x7f04d00b4720 0x7f04d00730b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d5200640 1 -- 172.21.15.47:0/1724747038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d015b1b0 msgr2=0x7f04d006dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d5200640 1 --2- 172.21.15.47:0/1724747038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d015b1b0 0x7f04d006dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d5200640 1 -- 172.21.15.47:0/1724747038 >> v2:172.21.15.175:3300/0 conn(0x7f04d00b4720 msgr2=0x7f04d00730b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d5200640 1 --2- 172.21.15.47:0/1724747038 >> v2:172.21.15.175:3300/0 conn(0x7f04d00b4720 0x7f04d00730b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d5200640 1 -- 172.21.15.47:0/1724747038 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d006d4f0 con 0x7f04d015b590 2024-04-20T10:29:00.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.012+0000 7f04d6600640 1 --2- 172.21.15.47:0/1724747038 >> v2:172.21.15.175:3300/0 conn(0x7f04d00b4720 0x7f04d00730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:00.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.013+0000 7f04d5200640 1 --2- 172.21.15.47:0/1724747038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d00b41e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f04d0074190 tx=0x7f04bc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44095b9074caebe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.013+0000 7f04cfe00640 1 -- 172.21.15.47:0/1724747038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc02e070 con 0x7f04d015b590 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.013+0000 7f04cfe00640 1 -- 172.21.15.47:0/1724747038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04bc037070 con 0x7f04d015b590 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.013+0000 7f04cfe00640 1 -- 172.21.15.47:0/1724747038 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04bc031a00 con 0x7f04d015b590 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 -- 172.21.15.47:0/1724747038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 msgr2=0x7f04d00b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 --2- 172.21.15.47:0/1724747038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d00b41e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f04d0074190 tx=0x7f04bc02b880 comp rx=0 tx=0).stop 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 -- 172.21.15.47:0/1724747038 shutdown_connections 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 --2- 172.21.15.47:0/1724747038 >> v2:172.21.15.175:3300/0 conn(0x7f04d00b4720 0x7f04d00730b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 --2- 172.21.15.47:0/1724747038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d00b41e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 --2- 172.21.15.47:0/1724747038 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d015b1b0 0x7f04d006dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 -- 172.21.15.47:0/1724747038 >> 172.21.15.47:0/1724747038 conn(0x7f04d00b3300 msgr2=0x7f04d00b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 -- 172.21.15.47:0/1724747038 shutdown_connections 2024-04-20T10:29:00.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.014+0000 7f04d7000640 1 -- 172.21.15.47:0/1724747038 wait complete. 2024-04-20T10:29:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.016+0000 7f04d7000640 1 Processor -- start 2024-04-20T10:29:00.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.016+0000 7f04d7000640 1 -- start start 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.017+0000 7f04d7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d00b4720 0x7f04d00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.017+0000 7f04d7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d015b1b0 0x7f04d01467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.017+0000 7f04d7000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d01487b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.017+0000 7f04d7000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f04d006e230 con 0x7f04d015b590 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.017+0000 7f04d7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04d00b5d40 con 0x7f04d015b1b0 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.017+0000 7f04d7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04d006e3b0 con 0x7f04d00b4720 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d01487b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d01487b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59760/0 (socket says 172.21.15.47:59760) 2024-04-20T10:29:00.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 -- 172.21.15.47:0/154146253 learned_addr learned my addr 172.21.15.47:0/154146253 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:00.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d5200640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d015b1b0 0x7f04d01467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d5c00640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d00b4720 0x7f04d00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 -- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d00b4720 msgr2=0x7f04d00afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d00b4720 0x7f04d00afd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 -- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d015b1b0 msgr2=0x7f04d01467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d015b1b0 0x7f04d01467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 -- 172.21.15.47:0/154146253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d00b0270 con 0x7f04d015b590 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d5200640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d015b1b0 0x7f04d01467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.018+0000 7f04d6600640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d01487b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04c800cd10 tx=0x7f04c800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.019+0000 7f04cd600640 1 -- 172.21.15.47:0/154146253 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800f680 con 0x7f04d015b590 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.019+0000 7f04cd600640 1 -- 172.21.15.47:0/154146253 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04c800f840 con 0x7f04d015b590 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.019+0000 7f04cd600640 1 -- 172.21.15.47:0/154146253 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04c800fc30 con 0x7f04d015b590 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.019+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d0147710 con 0x7f04d015b590 2024-04-20T10:29:00.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.019+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d0147cd0 con 0x7f04d015b590 2024-04-20T10:29:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.020+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d0075b30 con 0x7f04d015b590 2024-04-20T10:29:00.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.022+0000 7f04cd600640 1 -- 172.21.15.47:0/154146253 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f04c800e3e0 con 0x7f04d015b590 2024-04-20T10:29:00.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.022+0000 7f04cd600640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f04b006c8f0 0x7f04b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.023+0000 7f04cd600640 1 -- 172.21.15.47:0/154146253 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f04c808cc40 con 0x7f04d015b590 2024-04-20T10:29:00.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.023+0000 7f04d5c00640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f04b006c8f0 0x7f04b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.025+0000 7f04d5c00640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f04b006c8f0 0x7f04b006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04c0006fd0 tx=0x7f04c0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.027+0000 7f04cd600640 1 -- 172.21.15.47:0/154146253 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f04c8014030 con 0x7f04d015b590 2024-04-20T10:29:00.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.277+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04d00b2520 con 0x7f04d015b590 2024-04-20T10:29:00.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.278+0000 7f04cd600640 1 -- 172.21.15.47:0/154146253 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7f04c805acb0 con 0x7f04d015b590 2024-04-20T10:29:00.279 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:00.279 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:29:00.279 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.281+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f04b006c8f0 msgr2=0x7f04b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.281+0000 7f04d7000640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f04b006c8f0 0x7f04b006edb0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f04c0006fd0 tx=0x7f04c0008040 comp rx=0 tx=0).stop 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.281+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 msgr2=0x7f04d01487b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.281+0000 7f04d7000640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d01487b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f04c800cd10 tx=0x7f04c800a650 comp rx=0 tx=0).stop 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 shutdown_connections 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f04b006c8f0 0x7f04b006edb0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f04d015b590 0x7f04d01487b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04d015b1b0 0x7f04d01467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 --2- 172.21.15.47:0/154146253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04d00b4720 0x7f04d00afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 >> 172.21.15.47:0/154146253 conn(0x7f04d00b3300 msgr2=0x7f04d00730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 shutdown_connections 2024-04-20T10:29:00.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.282+0000 7f04d7000640 1 -- 172.21.15.47:0/154146253 wait complete. 2024-04-20T10:29:00.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.510+0000 7f251d200640 1 Processor -- start 2024-04-20T10:29:00.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.510+0000 7f251d200640 1 -- start start 2024-04-20T10:29:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.511+0000 7f251d200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2518159e10 0x7f2518071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.511+0000 7f251d200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f251815a1f0 0x7f2518071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.511+0000 7f251d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251806da90 0x7f2518160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.511+0000 7f251d200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2518072900 con 0x7f2518159e10 2024-04-20T10:29:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.511+0000 7f251d200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f251815ca40 con 0x7f251815a1f0 2024-04-20T10:29:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.511+0000 7f251d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f251815cc90 con 0x7f251806da90 2024-04-20T10:29:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.511+0000 7f2517400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2518159e10 0x7f2518071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2518159e10 0x7f2518071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59766/0 (socket says 172.21.15.47:59766) 2024-04-20T10:29:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517400640 1 -- 172.21.15.47:0/632252229 learned_addr learned my addr 172.21.15.47:0/632252229 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2516a00640 1 --2- 172.21.15.47:0/632252229 >> v2:172.21.15.175:3300/0 conn(0x7f251815a1f0 0x7f2518071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517e00640 1 --2- 172.21.15.47:0/632252229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251806da90 0x7f2518160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517400640 1 -- 172.21.15.47:0/632252229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251806da90 msgr2=0x7f2518160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517400640 1 --2- 172.21.15.47:0/632252229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251806da90 0x7f2518160030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517400640 1 -- 172.21.15.47:0/632252229 >> v2:172.21.15.175:3300/0 conn(0x7f251815a1f0 msgr2=0x7f2518071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517400640 1 --2- 172.21.15.47:0/632252229 >> v2:172.21.15.175:3300/0 conn(0x7f251815a1f0 0x7f2518071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2517400640 1 -- 172.21.15.47:0/632252229 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f251815cf70 con 0x7f2518159e10 2024-04-20T10:29:00.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.512+0000 7f2516a00640 1 --2- 172.21.15.47:0/632252229 >> v2:172.21.15.175:3300/0 conn(0x7f251815a1f0 0x7f2518071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.513+0000 7f2517400640 1 --2- 172.21.15.47:0/632252229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518071a80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2518050400 tx=0x7f250802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a3b377c62b1032 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.513+0000 7f2516000640 1 -- 172.21.15.47:0/632252229 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508035020 con 0x7f2518159e10 2024-04-20T10:29:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f2516000640 1 -- 172.21.15.47:0/632252229 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f250802d040 con 0x7f2518159e10 2024-04-20T10:29:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f2516000640 1 -- 172.21.15.47:0/632252229 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508031a80 con 0x7f2518159e10 2024-04-20T10:29:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 -- 172.21.15.47:0/632252229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 msgr2=0x7f2518071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 --2- 172.21.15.47:0/632252229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518071a80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f2518050400 tx=0x7f250802b900 comp rx=0 tx=0).stop 2024-04-20T10:29:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 -- 172.21.15.47:0/632252229 shutdown_connections 2024-04-20T10:29:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 --2- 172.21.15.47:0/632252229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251806da90 0x7f2518160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 --2- 172.21.15.47:0/632252229 >> v2:172.21.15.175:3300/0 conn(0x7f251815a1f0 0x7f2518071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 --2- 172.21.15.47:0/632252229 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518071a80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 -- 172.21.15.47:0/632252229 >> 172.21.15.47:0/632252229 conn(0x7f25180d0040 msgr2=0x7f25180d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 -- 172.21.15.47:0/632252229 shutdown_connections 2024-04-20T10:29:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.514+0000 7f251d200640 1 -- 172.21.15.47:0/632252229 wait complete. 2024-04-20T10:29:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.516+0000 7f251d200640 1 Processor -- start 2024-04-20T10:29:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.516+0000 7f251d200640 1 -- start start 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.517+0000 7f251d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f251806da90 0x7f25180b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.517+0000 7f251d200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.517+0000 7f251d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251815a1f0 0x7f2518152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.517+0000 7f251d200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f251815d810 con 0x7f2518159e10 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.517+0000 7f251d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f251815d530 con 0x7f251806da90 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.517+0000 7f251d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f251806b900 con 0x7f251815a1f0 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59774/0 (socket says 172.21.15.47:59774) 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 -- 172.21.15.47:0/2004192501 learned_addr learned my addr 172.21.15.47:0/2004192501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2517400640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f251806da90 0x7f25180b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2517e00640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251815a1f0 0x7f2518152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 -- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251815a1f0 msgr2=0x7f2518152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251815a1f0 0x7f2518152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 -- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f251806da90 msgr2=0x7f25180b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f251806da90 0x7f25180b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2516a00640 1 -- 172.21.15.47:0/2004192501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25180b3ce0 con 0x7f2518159e10 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.518+0000 7f2517400640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f251806da90 0x7f25180b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.519+0000 7f2516a00640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518151280 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f250000c7f0 tx=0x7f250000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.519+0000 7f250f400640 1 -- 172.21.15.47:0/2004192501 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500015070 con 0x7f2518159e10 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.519+0000 7f250f400640 1 -- 172.21.15.47:0/2004192501 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2500007820 con 0x7f2518159e10 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.519+0000 7f250f400640 1 -- 172.21.15.47:0/2004192501 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2500007c10 con 0x7f2518159e10 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.519+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25180b3fc0 con 0x7f2518159e10 2024-04-20T10:29:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.519+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25180b4580 con 0x7f2518159e10 2024-04-20T10:29:00.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.520+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2518075b60 con 0x7f2518159e10 2024-04-20T10:29:00.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.521+0000 7f250f400640 1 -- 172.21.15.47:0/2004192501 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2500018070 con 0x7f2518159e10 2024-04-20T10:29:00.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.522+0000 7f250f400640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f24f406c920 0x7f24f406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.522+0000 7f250f400640 1 -- 172.21.15.47:0/2004192501 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f250008bc80 con 0x7f2518159e10 2024-04-20T10:29:00.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.523+0000 7f2517400640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f24f406c920 0x7f24f406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.524+0000 7f2517400640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f24f406c920 0x7f24f406ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2508036000 tx=0x7f2508008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.527+0000 7f250f400640 1 -- 172.21.15.47:0/2004192501 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2500059cf0 con 0x7f2518159e10 2024-04-20T10:29:00.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.733+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f25180b2e30 con 0x7f2518159e10 2024-04-20T10:29:00.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.736+0000 7f250f400640 1 -- 172.21.15.47:0/2004192501 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f25000595f0 con 0x7f2518159e10 2024-04-20T10:29:00.737 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:00.737 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f24f406c920 msgr2=0x7f24f406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f24f406c920 0x7f24f406ede0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2508036000 tx=0x7f2508008030 comp rx=0 tx=0).stop 2024-04-20T10:29:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 msgr2=0x7f2518151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518151280 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f250000c7f0 tx=0x7f250000ccb0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 shutdown_connections 2024-04-20T10:29:00.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f251815a1f0 0x7f2518152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f24f406c920 0x7f24f406ede0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2518159e10 0x7f2518151280 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 --2- 172.21.15.47:0/2004192501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f251806da90 0x7f25180b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 >> 172.21.15.47:0/2004192501 conn(0x7f25180d0040 msgr2=0x7f25181607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 shutdown_connections 2024-04-20T10:29:00.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.739+0000 7f251d200640 1 -- 172.21.15.47:0/2004192501 wait complete. 2024-04-20T10:29:00.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:27:30.221834+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:27:31.436137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-20T10:27:35.089154+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '27', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-20T10:29:00.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:29:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.976+0000 7f37d5000640 1 Processor -- start 2024-04-20T10:29:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.977+0000 7f37d5000640 1 -- start start 2024-04-20T10:29:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.978+0000 7f37d5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d0161b40 0x7f37d00c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.978+0000 7f37d5000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f37d00cdea0 0x7f37d00c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.978+0000 7f37d5000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37d00aded0 con 0x7f37d00ae3e0 2024-04-20T10:29:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.978+0000 7f37d5000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f37d00ad2c0 con 0x7f37d00cdea0 2024-04-20T10:29:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.978+0000 7f37d5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37d00ad5a0 con 0x7f37d0161b40 2024-04-20T10:29:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.979+0000 7f37cf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d0161b40 0x7f37d00c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.979+0000 7f37cfe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f37d00cdea0 0x7f37d00c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.979+0000 7f37cf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d0161b40 0x7f37d00c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:43946/0 (socket says 172.21.15.47:43946) 2024-04-20T10:29:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.979+0000 7f37cf400640 1 -- 172.21.15.47:0/3076629169 learned_addr learned my addr 172.21.15.47:0/3076629169 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.980+0000 7f37cf400640 1 -- 172.21.15.47:0/3076629169 >> v2:172.21.15.175:3300/0 conn(0x7f37d00cdea0 msgr2=0x7f37d00c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.980+0000 7f37cf400640 1 --2- 172.21.15.47:0/3076629169 >> v2:172.21.15.175:3300/0 conn(0x7f37d00cdea0 0x7f37d00c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.980+0000 7f37cf400640 1 -- 172.21.15.47:0/3076629169 >> v1:172.21.15.175:6789/0 conn(0x7f37d00ae3e0 legacy=0x7f37d01620d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.980+0000 7f37cf400640 1 -- 172.21.15.47:0/3076629169 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d00ad880 con 0x7f37d0161b40 2024-04-20T10:29:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.981+0000 7f37cf400640 1 --2- 172.21.15.47:0/3076629169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d0161b40 0x7f37d00c92f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009870 tx=0x7f37c002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2400505c1136f6d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.982+0000 7f37cea00640 1 -- 172.21.15.47:0/3076629169 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0034070 con 0x7f37d0161b40 2024-04-20T10:29:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.982+0000 7f37cea00640 1 -- 172.21.15.47:0/3076629169 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c0030840 con 0x7f37d0161b40 2024-04-20T10:29:00.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.982+0000 7f37cea00640 1 -- 172.21.15.47:0/3076629169 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0030c10 con 0x7f37d0161b40 2024-04-20T10:29:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 -- 172.21.15.47:0/3076629169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d0161b40 msgr2=0x7f37d00c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 --2- 172.21.15.47:0/3076629169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d0161b40 0x7f37d00c92f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009870 tx=0x7f37c002b980 comp rx=0 tx=0).stop 2024-04-20T10:29:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 -- 172.21.15.47:0/3076629169 shutdown_connections 2024-04-20T10:29:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 --2- 172.21.15.47:0/3076629169 >> v2:172.21.15.175:3300/0 conn(0x7f37d00cdea0 0x7f37d00c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 --2- 172.21.15.47:0/3076629169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d0161b40 0x7f37d00c92f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 -- 172.21.15.47:0/3076629169 >> 172.21.15.47:0/3076629169 conn(0x7f37d00cffa0 msgr2=0x7f37d00d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:00.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 -- 172.21.15.47:0/3076629169 shutdown_connections 2024-04-20T10:29:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.983+0000 7f37d5000640 1 -- 172.21.15.47:0/3076629169 wait complete. 2024-04-20T10:29:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.985+0000 7f37d5000640 1 Processor -- start 2024-04-20T10:29:00.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.985+0000 7f37d5000640 1 -- start start 2024-04-20T10:29:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37d5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00ae3e0 0x7f37d00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37d5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cdea0 0x7f37d00b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37d5000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0161b40 0x7f37d0071960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37d5000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f37d006d0c0 con 0x7f37d0161b40 2024-04-20T10:29:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37d5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37d00acf50 con 0x7f37d00cdea0 2024-04-20T10:29:00.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37d5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37d006d240 con 0x7f37d00ae3e0 2024-04-20T10:29:00.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0161b40 0x7f37d0071960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37cfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0161b40 0x7f37d0071960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59782/0 (socket says 172.21.15.47:59782) 2024-04-20T10:29:00.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.986+0000 7f37cfe00640 1 -- 172.21.15.47:0/2492339591 learned_addr learned my addr 172.21.15.47:0/2492339591 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37c7e00640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cdea0 0x7f37d00b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cf400640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00ae3e0 0x7f37d00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cfe00640 1 -- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00ae3e0 msgr2=0x7f37d00b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cfe00640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00ae3e0 0x7f37d00b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cfe00640 1 -- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cdea0 msgr2=0x7f37d00b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cfe00640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cdea0 0x7f37d00b1120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cfe00640 1 -- 172.21.15.47:0/2492339591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d009cd60 con 0x7f37d0161b40 2024-04-20T10:29:00.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cf400640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00ae3e0 0x7f37d00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:00.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37cfe00640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0161b40 0x7f37d0071960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37c800efc0 tx=0x7f37c800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37c7400640 1 -- 172.21.15.47:0/2492339591 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8018020 con 0x7f37d0161b40 2024-04-20T10:29:00.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37c7400640 1 -- 172.21.15.47:0/2492339591 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37c8014480 con 0x7f37d0161b40 2024-04-20T10:29:00.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.987+0000 7f37c7400640 1 -- 172.21.15.47:0/2492339591 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c8014870 con 0x7f37d0161b40 2024-04-20T10:29:00.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.988+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d009d040 con 0x7f37d0161b40 2024-04-20T10:29:00.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.988+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d009d600 con 0x7f37d0161b40 2024-04-20T10:29:00.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.989+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d0075b30 con 0x7f37d0161b40 2024-04-20T10:29:00.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.990+0000 7f37c7400640 1 -- 172.21.15.47:0/2492339591 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37c8005d10 con 0x7f37d0161b40 2024-04-20T10:29:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.991+0000 7f37c7400640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ac06c8f0 0x7f37ac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.991+0000 7f37c7400640 1 -- 172.21.15.47:0/2492339591 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f37c808ca00 con 0x7f37d0161b40 2024-04-20T10:29:00.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.991+0000 7f37cf400640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ac06c8f0 0x7f37ac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:00.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.993+0000 7f37cf400640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ac06c8f0 0x7f37ac06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37c0036000 tx=0x7f37c003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:00.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:00.995+0000 7f37c7400640 1 -- 172.21.15.47:0/2492339591 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f37c8022070 con 0x7f37d0161b40 2024-04-20T10:29:01.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.204+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f37d00ae110 con 0x7f37d0161b40 2024-04-20T10:29:01.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.205+0000 7f37c7400640 1 -- 172.21.15.47:0/2492339591 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v29) ==== 93+0+10 (secure 0 0 0) 0x7f37c8010040 con 0x7f37d0161b40 2024-04-20T10:29:01.205 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.208+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ac06c8f0 msgr2=0x7f37ac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.208+0000 7f37d5000640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ac06c8f0 0x7f37ac06edb0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f37c0036000 tx=0x7f37c003b040 comp rx=0 tx=0).stop 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.208+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0161b40 msgr2=0x7f37d0071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.208+0000 7f37d5000640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0161b40 0x7f37d0071960 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f37c800efc0 tx=0x7f37c800c490 comp rx=0 tx=0).stop 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 shutdown_connections 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ac06c8f0 0x7f37ac06edb0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f37d0161b40 0x7f37d0071960 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37d00cdea0 0x7f37d00b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 --2- 172.21.15.47:0/2492339591 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37d00ae3e0 0x7f37d00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 >> 172.21.15.47:0/2492339591 conn(0x7f37d00cffa0 msgr2=0x7f37d006e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 shutdown_connections 2024-04-20T10:29:01.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.209+0000 7f37d5000640 1 -- 172.21.15.47:0/2492339591 wait complete. 2024-04-20T10:29:01.228 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:29:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.432+0000 7f563b200640 1 Processor -- start 2024-04-20T10:29:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.433+0000 7f563b200640 1 -- start start 2024-04-20T10:29:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.434+0000 7f563b200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f563c161a50 0x7f563c0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.434+0000 7f563b200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563c0d0980 con 0x7f563c0d3b20 2024-04-20T10:29:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.434+0000 7f563b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f563c162490 con 0x7f563c06c010 2024-04-20T10:29:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.434+0000 7f563b200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f563c0d0c60 con 0x7f563c161a50 2024-04-20T10:29:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.435+0000 7f5633e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f563c0d3b20 0x7f563c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34738/0 (socket says 172.21.15.47:34738) 2024-04-20T10:29:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.435+0000 7f5633e00640 1 -- 172.21.15.47:0/3148727068 learned_addr learned my addr 172.21.15.47:0/3148727068 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.435+0000 7f5639e00640 1 --2- 172.21.15.47:0/3148727068 >> v2:172.21.15.47:3300/0 conn(0x7f563c161a50 0x7f563c0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.435+0000 7f5639e00640 1 -- 172.21.15.47:0/3148727068 >> v1:172.21.15.175:6789/0 conn(0x7f563c06c010 legacy=0x7f563c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.435+0000 7f5639e00640 1 -- 172.21.15.47:0/3148727068 >> v1:172.21.15.47:6789/0 conn(0x7f563c0d3b20 legacy=0x7f563c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.435+0000 7f5639e00640 1 -- 172.21.15.47:0/3148727068 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c162770 con 0x7f563c161a50 2024-04-20T10:29:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.435+0000 7f5639400640 1 -- 172.21.15.47:0/3148727068 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 181851886 0 0) 0x7f563c0d0980 con 0x7f563c0d3b20 2024-04-20T10:29:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.437+0000 7f5639e00640 1 --2- 172.21.15.47:0/3148727068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0b3180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f563c0d0de0 tx=0x7f562c02b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da018a5fce5789e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.437+0000 7f5639400640 1 -- 172.21.15.47:0/3148727068 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c035020 con 0x7f563c161a50 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.437+0000 7f5639400640 1 -- 172.21.15.47:0/3148727068 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f562c031800 con 0x7f563c161a50 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.437+0000 7f5639400640 1 -- 172.21.15.47:0/3148727068 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f562c031bd0 con 0x7f563c161a50 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.437+0000 7f563b200640 1 -- 172.21.15.47:0/3148727068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 msgr2=0x7f563c0b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.437+0000 7f563b200640 1 --2- 172.21.15.47:0/3148727068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0b3180 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f563c0d0de0 tx=0x7f562c02b8a0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.438+0000 7f563b200640 1 -- 172.21.15.47:0/3148727068 shutdown_connections 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.438+0000 7f563b200640 1 --2- 172.21.15.47:0/3148727068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0b3180 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.438+0000 7f563b200640 1 -- 172.21.15.47:0/3148727068 >> 172.21.15.47:0/3148727068 conn(0x7f563c0d0ed0 msgr2=0x7f563c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.438+0000 7f563b200640 1 -- 172.21.15.47:0/3148727068 shutdown_connections 2024-04-20T10:29:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.438+0000 7f563b200640 1 -- 172.21.15.47:0/3148727068 wait complete. 2024-04-20T10:29:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.439+0000 7f563b200640 1 Processor -- start 2024-04-20T10:29:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.440+0000 7f563b200640 1 -- start start 2024-04-20T10:29:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f563b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f563c06c010 0x7f563c15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f563b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0c7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f563b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f563c160790 0x7f563c0acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f563b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f563c162ae0 con 0x7f563c161a50 2024-04-20T10:29:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f563b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f563c0d0690 con 0x7f563c160790 2024-04-20T10:29:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f563b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f563c161e30 con 0x7f563c06c010 2024-04-20T10:29:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0c7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0c7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59796/0 (socket says 172.21.15.47:59796) 2024-04-20T10:29:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 -- 172.21.15.47:0/2689101684 learned_addr learned my addr 172.21.15.47:0/2689101684 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 -- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f563c06c010 msgr2=0x7f563c15ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f563c06c010 0x7f563c15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 -- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f563c160790 msgr2=0x7f563c0acc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f563c160790 0x7f563c0acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.441+0000 7f5633e00640 1 -- 172.21.15.47:0/2689101684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c0c8620 con 0x7f563c161a50 2024-04-20T10:29:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.442+0000 7f5633e00640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0c7d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f562400c4b0 tx=0x7f562400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.442+0000 7f5632000640 1 -- 172.21.15.47:0/2689101684 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624018070 con 0x7f563c161a50 2024-04-20T10:29:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.442+0000 7f5632000640 1 -- 172.21.15.47:0/2689101684 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5624007500 con 0x7f563c161a50 2024-04-20T10:29:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.442+0000 7f5632000640 1 -- 172.21.15.47:0/2689101684 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5624004a20 con 0x7f563c161a50 2024-04-20T10:29:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.442+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c0c8900 con 0x7f563c161a50 2024-04-20T10:29:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.442+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c0c8ec0 con 0x7f563c161a50 2024-04-20T10:29:01.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.443+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c075b30 con 0x7f563c161a50 2024-04-20T10:29:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.444+0000 7f5632000640 1 -- 172.21.15.47:0/2689101684 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5624004bc0 con 0x7f563c161a50 2024-04-20T10:29:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.445+0000 7f5632000640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f561806c8f0 0x7f561806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.445+0000 7f5632000640 1 -- 172.21.15.47:0/2689101684 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f562408b270 con 0x7f563c161a50 2024-04-20T10:29:01.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.446+0000 7f5639e00640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f561806c8f0 0x7f561806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.447+0000 7f5639e00640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f561806c8f0 0x7f561806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f562c036000 tx=0x7f562c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:01.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.450+0000 7f5632000640 1 -- 172.21.15.47:0/2689101684 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5624059310 con 0x7f563c161a50 2024-04-20T10:29:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.656+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f563c160510 con 0x7f563c161a50 2024-04-20T10:29:01.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.657+0000 7f5632000640 1 -- 172.21.15.47:0/2689101684 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v29) ==== 104+0+11 (secure 0 0 0) 0x7f5624058c10 con 0x7f563c161a50 2024-04-20T10:29:01.657 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:29:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f561806c8f0 msgr2=0x7f561806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f561806c8f0 0x7f561806edb0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f562c036000 tx=0x7f562c030000 comp rx=0 tx=0).stop 2024-04-20T10:29:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 msgr2=0x7f563c0c7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0c7d80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f562400c4b0 tx=0x7f562400c970 comp rx=0 tx=0).stop 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 shutdown_connections 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f563c160790 0x7f563c0acc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f561806c8f0 0x7f561806edb0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f563c161a50 0x7f563c0c7d80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 --2- 172.21.15.47:0/2689101684 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f563c06c010 0x7f563c15ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 >> 172.21.15.47:0/2689101684 conn(0x7f563c0d0ed0 msgr2=0x7f563c0b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 shutdown_connections 2024-04-20T10:29:01.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.660+0000 7f563b200640 1 -- 172.21.15.47:0/2689101684 wait complete. 2024-04-20T10:29:01.680 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:29:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.895+0000 7f956ee00640 1 Processor -- start 2024-04-20T10:29:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.896+0000 7f956ee00640 1 -- start start 2024-04-20T10:29:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.897+0000 7f956ee00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9568073b80 0x7f956806f160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.897+0000 7f956ee00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9568071a60 0x7f956806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.897+0000 7f956ee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f956815cc20 con 0x7f956815a130 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.897+0000 7f956ee00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f956815d7f0 con 0x7f9568073b80 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.897+0000 7f956ee00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f956806fbe0 con 0x7f9568071a60 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956d000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9568073b80 0x7f956806f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956d000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9568073b80 0x7f956806f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59802/0 (socket says 172.21.15.47:59802) 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956d000640 1 -- 172.21.15.47:0/872668019 learned_addr learned my addr 172.21.15.47:0/872668019 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956da00640 1 --1- 172.21.15.47:0/872668019 >> v1:172.21.15.47:6789/0 conn(0x7f956815a130 0x7f9568073470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34746/0 (socket says 172.21.15.47:34746) 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956e400640 1 --2- 172.21.15.47:0/872668019 >> v2:172.21.15.175:3300/0 conn(0x7f9568071a60 0x7f956806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956d000640 1 -- 172.21.15.47:0/872668019 >> v2:172.21.15.175:3300/0 conn(0x7f9568071a60 msgr2=0x7f956806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956d000640 1 --2- 172.21.15.47:0/872668019 >> v2:172.21.15.175:3300/0 conn(0x7f9568071a60 0x7f956806f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956d000640 1 -- 172.21.15.47:0/872668019 >> v1:172.21.15.47:6789/0 conn(0x7f956815a130 legacy=0x7f9568073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956d000640 1 -- 172.21.15.47:0/872668019 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f956815cec0 con 0x7f9568073b80 2024-04-20T10:29:01.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.898+0000 7f956e400640 1 --2- 172.21.15.47:0/872668019 >> v2:172.21.15.175:3300/0 conn(0x7f9568071a60 0x7f956806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.900+0000 7f956d000640 1 --2- 172.21.15.47:0/872668019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f956806f160 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f95680d0f90 tx=0x7f955402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefbb742838dd868 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.900+0000 7f9567e00640 1 -- 172.21.15.47:0/872668019 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554035020 con 0x7f9568073b80 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.900+0000 7f9567e00640 1 -- 172.21.15.47:0/872668019 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9554031870 con 0x7f9568073b80 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.900+0000 7f9567e00640 1 -- 172.21.15.47:0/872668019 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9554031c40 con 0x7f9568073b80 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 -- 172.21.15.47:0/872668019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 msgr2=0x7f956806f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 --2- 172.21.15.47:0/872668019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f956806f160 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f95680d0f90 tx=0x7f955402b910 comp rx=0 tx=0).stop 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 -- 172.21.15.47:0/872668019 shutdown_connections 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 --2- 172.21.15.47:0/872668019 >> v2:172.21.15.175:3300/0 conn(0x7f9568071a60 0x7f956806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 --2- 172.21.15.47:0/872668019 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f956806f160 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 -- 172.21.15.47:0/872668019 >> 172.21.15.47:0/872668019 conn(0x7f95680cffa0 msgr2=0x7f95680d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 -- 172.21.15.47:0/872668019 shutdown_connections 2024-04-20T10:29:01.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.901+0000 7f956ee00640 1 -- 172.21.15.47:0/872668019 wait complete. 2024-04-20T10:29:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.903+0000 7f956ee00640 1 Processor -- start 2024-04-20T10:29:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.903+0000 7f956ee00640 1 -- start start 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9568071a60 0x7f956814db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f95680ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815a130 0x7f95680cf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f956806b900 con 0x7f9568073b80 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f956815d480 con 0x7f9568071a60 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f956806ba80 con 0x7f956815a130 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f95680ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f95680ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59806/0 (socket says 172.21.15.47:59806) 2024-04-20T10:29:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956d000640 1 -- 172.21.15.47:0/1954620422 learned_addr learned my addr 172.21.15.47:0/1954620422 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956da00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9568071a60 0x7f956814db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956e400640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815a130 0x7f95680cf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956d000640 1 -- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815a130 msgr2=0x7f95680cf0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956d000640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815a130 0x7f95680cf0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956d000640 1 -- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9568071a60 msgr2=0x7f956814db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.904+0000 7f956d000640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9568071a60 0x7f956814db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.905+0000 7f956d000640 1 -- 172.21.15.47:0/1954620422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95681f5240 con 0x7f9568073b80 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.905+0000 7f956d000640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f95680ca6a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9554000c00 tx=0x7f95540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.905+0000 7f9565600640 1 -- 172.21.15.47:0/1954620422 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540443f0 con 0x7f9568073b80 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.905+0000 7f9565600640 1 -- 172.21.15.47:0/1954620422 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f95540445b0 con 0x7f9568073b80 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.905+0000 7f9565600640 1 -- 172.21.15.47:0/1954620422 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95540449a0 con 0x7f9568073b80 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.905+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95681f46c0 con 0x7f9568073b80 2024-04-20T10:29:01.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.905+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95681f4c80 con 0x7f9568073b80 2024-04-20T10:29:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.907+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9568075b30 con 0x7f9568073b80 2024-04-20T10:29:01.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.908+0000 7f9565600640 1 -- 172.21.15.47:0/1954620422 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f955404b060 con 0x7f9568073b80 2024-04-20T10:29:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.909+0000 7f9565600640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f954806c8f0 0x7f954806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.909+0000 7f9565600640 1 -- 172.21.15.47:0/1954620422 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f95540ae0f0 con 0x7f9568073b80 2024-04-20T10:29:01.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.909+0000 7f956da00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f954806c8f0 0x7f954806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:01.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.910+0000 7f956da00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f954806c8f0 0x7f954806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f956809d9f0 tx=0x7f955c007ce0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:01.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:01.913+0000 7f9565600640 1 -- 172.21.15.47:0/1954620422 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f955403f070 con 0x7f9568073b80 2024-04-20T10:29:02.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.115+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f956806d010 con 0x7f9568073b80 2024-04-20T10:29:02.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.116+0000 7f9565600640 1 -- 172.21.15.47:0/1954620422 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v29) ==== 100+0+11 (secure 0 0 0) 0x7f955402f320 con 0x7f9568073b80 2024-04-20T10:29:02.116 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:29:02.119 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f954806c8f0 msgr2=0x7f954806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f954806c8f0 0x7f954806edb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f956809d9f0 tx=0x7f955c007ce0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 msgr2=0x7f95680ca6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f95680ca6a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f9554000c00 tx=0x7f95540046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 shutdown_connections 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f956815a130 0x7f95680cf0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f954806c8f0 0x7f954806edb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9568073b80 0x7f95680ca6a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 --2- 172.21.15.47:0/1954620422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9568071a60 0x7f956814db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 >> 172.21.15.47:0/1954620422 conn(0x7f95680cffa0 msgr2=0x7f956806bdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 shutdown_connections 2024-04-20T10:29:02.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.119+0000 7f956ee00640 1 -- 172.21.15.47:0/1954620422 wait complete. 2024-04-20T10:29:02.139 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-20T10:29:02.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.357+0000 7f454f400640 1 Processor -- start 2024-04-20T10:29:02.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.358+0000 7f454f400640 1 -- start start 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.359+0000 7f454f400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4548156c20 0x7f454806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.359+0000 7f454f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f454815ca40 0x7f45480ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.359+0000 7f454f400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4548141c60 con 0x7f45480ad8e0 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.359+0000 7f454f400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f454815cf90 con 0x7f454815ca40 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.359+0000 7f454f400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f454815d270 con 0x7f4548156c20 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f4547e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f454815ca40 0x7f45480ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f4547e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f454815ca40 0x7f45480ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59812/0 (socket says 172.21.15.47:59812) 2024-04-20T10:29:02.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f4547e00640 1 -- 172.21.15.47:0/1542960631 learned_addr learned my addr 172.21.15.47:0/1542960631 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f454ca00640 1 --2- 172.21.15.47:0/1542960631 >> v2:172.21.15.175:3300/0 conn(0x7f4548156c20 0x7f454806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f4547e00640 1 -- 172.21.15.47:0/1542960631 >> v2:172.21.15.175:3300/0 conn(0x7f4548156c20 msgr2=0x7f454806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f4547e00640 1 --2- 172.21.15.47:0/1542960631 >> v2:172.21.15.175:3300/0 conn(0x7f4548156c20 0x7f454806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f4547e00640 1 -- 172.21.15.47:0/1542960631 >> v1:172.21.15.175:6789/0 conn(0x7f45480ad8e0 legacy=0x7f45480d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f4547e00640 1 -- 172.21.15.47:0/1542960631 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454815d550 con 0x7f454815ca40 2024-04-20T10:29:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.360+0000 7f454ca00640 1 --2- 172.21.15.47:0/1542960631 >> v2:172.21.15.175:3300/0 conn(0x7f4548156c20 0x7f454806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:02.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.361+0000 7f4547e00640 1 --2- 172.21.15.47:0/1542960631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f45480ad3a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f454806fdc0 tx=0x7f453402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86c9ad22a9e70946 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.362+0000 7f4547400640 1 -- 172.21.15.47:0/1542960631 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534035020 con 0x7f454815ca40 2024-04-20T10:29:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.362+0000 7f4547400640 1 -- 172.21.15.47:0/1542960631 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4534031870 con 0x7f454815ca40 2024-04-20T10:29:02.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.362+0000 7f4547400640 1 -- 172.21.15.47:0/1542960631 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4534031c40 con 0x7f454815ca40 2024-04-20T10:29:02.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.362+0000 7f454f400640 1 -- 172.21.15.47:0/1542960631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 msgr2=0x7f45480ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.362+0000 7f454f400640 1 --2- 172.21.15.47:0/1542960631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f45480ad3a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f454806fdc0 tx=0x7f453402b910 comp rx=0 tx=0).stop 2024-04-20T10:29:02.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.363+0000 7f454f400640 1 -- 172.21.15.47:0/1542960631 shutdown_connections 2024-04-20T10:29:02.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.363+0000 7f454f400640 1 --2- 172.21.15.47:0/1542960631 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f45480ad3a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.363+0000 7f454f400640 1 --2- 172.21.15.47:0/1542960631 >> v2:172.21.15.175:3300/0 conn(0x7f4548156c20 0x7f454806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.363+0000 7f454f400640 1 -- 172.21.15.47:0/1542960631 >> 172.21.15.47:0/1542960631 conn(0x7f45480b3350 msgr2=0x7f45480b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:02.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.363+0000 7f454f400640 1 -- 172.21.15.47:0/1542960631 shutdown_connections 2024-04-20T10:29:02.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.363+0000 7f454f400640 1 -- 172.21.15.47:0/1542960631 wait complete. 2024-04-20T10:29:02.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.365+0000 7f454f400640 1 Processor -- start 2024-04-20T10:29:02.364 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.365+0000 7f454f400640 1 -- start start 2024-04-20T10:29:02.367 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45480ad8e0 0x7f45481f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4548156c20 0x7f454806e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f454815b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454f400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f454806bfc0 con 0x7f454815ca40 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f454806be40 con 0x7f45480ad8e0 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f45480d0a70 con 0x7f4548156c20 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f454815b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f454815b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59822/0 (socket says 172.21.15.47:59822) 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454d400640 1 -- 172.21.15.47:0/402384407 learned_addr learned my addr 172.21.15.47:0/402384407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454ca00640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45480ad8e0 0x7f45481f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f4547e00640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4548156c20 0x7f454806e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.366+0000 7f454d400640 1 -- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4548156c20 msgr2=0x7f454806e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f454d400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4548156c20 0x7f454806e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f454d400640 1 -- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45480ad8e0 msgr2=0x7f45481f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f454d400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45480ad8e0 0x7f45481f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f454d400640 1 -- 172.21.15.47:0/402384407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f454806ead0 con 0x7f454815ca40 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f454d400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f454815b410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f454000c780 tx=0x7f454000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f4544c00640 1 -- 172.21.15.47:0/402384407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540017070 con 0x7f454815ca40 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f4544c00640 1 -- 172.21.15.47:0/402384407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4540003b70 con 0x7f454815ca40 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f454806f2b0 con 0x7f454815ca40 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f4544c00640 1 -- 172.21.15.47:0/402384407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4540005070 con 0x7f454815ca40 2024-04-20T10:29:02.369 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.367+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4548151170 con 0x7f454815ca40 2024-04-20T10:29:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.368+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4548068980 con 0x7f454815ca40 2024-04-20T10:29:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.370+0000 7f4544c00640 1 -- 172.21.15.47:0/402384407 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f454001a070 con 0x7f454815ca40 2024-04-20T10:29:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.371+0000 7f4544c00640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f452806c8f0 0x7f452806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.371+0000 7f4544c00640 1 -- 172.21.15.47:0/402384407 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f454008b5b0 con 0x7f454815ca40 2024-04-20T10:29:02.371 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.371+0000 7f454ca00640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f452806c8f0 0x7f452806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.372+0000 7f454ca00640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f452806c8f0 0x7f452806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f454806f770 tx=0x7f453c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:02.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.375+0000 7f4544c00640 1 -- 172.21.15.47:0/402384407 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4540059590 con 0x7f454815ca40 2024-04-20T10:29:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.578+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f45480cfd90 con 0x7f454815ca40 2024-04-20T10:29:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.579+0000 7f4544c00640 1 -- 172.21.15.47:0/402384407 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v29) ==== 103+0+11 (secure 0 0 0) 0x7f4540058e90 con 0x7f454815ca40 2024-04-20T10:29:02.579 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:29:02.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f452806c8f0 msgr2=0x7f452806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f452806c8f0 0x7f452806edb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f454806f770 tx=0x7f453c008040 comp rx=0 tx=0).stop 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 msgr2=0x7f454815b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f454815b410 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f454000c780 tx=0x7f454000cc40 comp rx=0 tx=0).stop 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 shutdown_connections 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f452806c8f0 0x7f452806edb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f454815ca40 0x7f454815b410 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4548156c20 0x7f454806e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 --2- 172.21.15.47:0/402384407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45480ad8e0 0x7f45481f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.582+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 >> 172.21.15.47:0/402384407 conn(0x7f45480b3350 msgr2=0x7f45481544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.583+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 shutdown_connections 2024-04-20T10:29:02.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.583+0000 7f454f400640 1 -- 172.21.15.47:0/402384407 wait complete. 2024-04-20T10:29:02.603 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.819+0000 7ffa20a00640 1 Processor -- start 2024-04-20T10:29:02.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.819+0000 7ffa20a00640 1 -- start start 2024-04-20T10:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa20a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ffa1c161d50 0x7ffa1c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa20a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffa1c0b3330 0x7ffa1c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa20a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ffa1c06c190 con 0x7ffa1c06c6d0 2024-04-20T10:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa20a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c162130 con 0x7ffa1c0b3330 2024-04-20T10:29:02.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa20a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ffa1c06c470 con 0x7ffa1c161d50 2024-04-20T10:29:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa12a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ffa1c0b3330 0x7ffa1c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa1be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ffa1c06c6d0 0x7ffa1c0cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34748/0 (socket says 172.21.15.47:34748) 2024-04-20T10:29:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.820+0000 7ffa1be00640 1 -- 172.21.15.47:0/3007796146 learned_addr learned my addr 172.21.15.47:0/3007796146 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.821+0000 7ffa1b400640 1 --2- 172.21.15.47:0/3007796146 >> v2:172.21.15.175:3300/0 conn(0x7ffa1c161d50 0x7ffa1c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.821+0000 7ffa12a00640 1 -- 172.21.15.47:0/3007796146 >> v2:172.21.15.175:3300/0 conn(0x7ffa1c161d50 msgr2=0x7ffa1c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.821+0000 7ffa12a00640 1 --2- 172.21.15.47:0/3007796146 >> v2:172.21.15.175:3300/0 conn(0x7ffa1c161d50 0x7ffa1c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.821+0000 7ffa1aa00640 1 -- 172.21.15.47:0/3007796146 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2866608078 0 0) 0x7ffa1c06c190 con 0x7ffa1c06c6d0 2024-04-20T10:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.821+0000 7ffa12a00640 1 -- 172.21.15.47:0/3007796146 >> v1:172.21.15.47:6789/0 conn(0x7ffa1c06c6d0 legacy=0x7ffa1c0cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.821+0000 7ffa12a00640 1 -- 172.21.15.47:0/3007796146 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0d2e00 con 0x7ffa1c0b3330 2024-04-20T10:29:02.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.821+0000 7ffa1b400640 1 --2- 172.21.15.47:0/3007796146 >> v2:172.21.15.175:3300/0 conn(0x7ffa1c161d50 0x7ffa1c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.822+0000 7ffa12a00640 1 --2- 172.21.15.47:0/3007796146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c0b3880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0b6f50 tx=0x7ffa0402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e2a1498c0b5a66a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.822+0000 7ffa1aa00640 1 -- 172.21.15.47:0/3007796146 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04035020 con 0x7ffa1c0b3330 2024-04-20T10:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.822+0000 7ffa1aa00640 1 -- 172.21.15.47:0/3007796146 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa04031830 con 0x7ffa1c0b3330 2024-04-20T10:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.822+0000 7ffa1aa00640 1 -- 172.21.15.47:0/3007796146 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04031c00 con 0x7ffa1c0b3330 2024-04-20T10:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 -- 172.21.15.47:0/3007796146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 msgr2=0x7ffa1c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3007796146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c0b3880 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c0b6f50 tx=0x7ffa0402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 -- 172.21.15.47:0/3007796146 shutdown_connections 2024-04-20T10:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3007796146 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c0b3880 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3007796146 >> v2:172.21.15.175:3300/0 conn(0x7ffa1c161d50 0x7ffa1c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 -- 172.21.15.47:0/3007796146 >> 172.21.15.47:0/3007796146 conn(0x7ffa1c0d4450 msgr2=0x7ffa1c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 -- 172.21.15.47:0/3007796146 shutdown_connections 2024-04-20T10:29:02.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.823+0000 7ffa20a00640 1 -- 172.21.15.47:0/3007796146 wait complete. 2024-04-20T10:29:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.825+0000 7ffa20a00640 1 Processor -- start 2024-04-20T10:29:02.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.825+0000 7ffa20a00640 1 -- start start 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.826+0000 7ffa20a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1c06c6d0 0x7ffa1c0c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.826+0000 7ffa20a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c14f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.826+0000 7ffa20a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa1c161d50 0x7ffa1c0cf850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.826+0000 7ffa20a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c06cb20 con 0x7ffa1c0b3330 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.826+0000 7ffa20a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1c0d33c0 con 0x7ffa1c161d50 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.826+0000 7ffa20a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa1c06cca0 con 0x7ffa1c06c6d0 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.826+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c14f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c14f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59836/0 (socket says 172.21.15.47:59836) 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 -- 172.21.15.47:0/3676361366 learned_addr learned my addr 172.21.15.47:0/3676361366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:02.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa1b400640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1c06c6d0 0x7ffa1c0c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa1be00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa1c161d50 0x7ffa1c0cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 -- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1c06c6d0 msgr2=0x7ffa1c0c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1c06c6d0 0x7ffa1c0c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 -- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa1c161d50 msgr2=0x7ffa1c0cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa1c161d50 0x7ffa1c0cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:02.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 -- 172.21.15.47:0/3676361366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1c0c9400 con 0x7ffa1c0b3330 2024-04-20T10:29:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.827+0000 7ffa12a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c14f770 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffa0402bee0 tx=0x7ffa040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.828+0000 7ffa13e00640 1 -- 172.21.15.47:0/3676361366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04041070 con 0x7ffa1c0b3330 2024-04-20T10:29:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.828+0000 7ffa13e00640 1 -- 172.21.15.47:0/3676361366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa0403aa30 con 0x7ffa1c0b3330 2024-04-20T10:29:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.828+0000 7ffa13e00640 1 -- 172.21.15.47:0/3676361366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa04035040 con 0x7ffa1c0b3330 2024-04-20T10:29:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.828+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1c0c8100 con 0x7ffa1c0b3330 2024-04-20T10:29:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.828+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1c0c86c0 con 0x7ffa1c0b3330 2024-04-20T10:29:02.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.829+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa1c075b60 con 0x7ffa1c0b3330 2024-04-20T10:29:02.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.830+0000 7ffa13e00640 1 -- 172.21.15.47:0/3676361366 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ffa04038020 con 0x7ffa1c0b3330 2024-04-20T10:29:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.831+0000 7ffa13e00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff9f806c8f0 0x7ff9f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:02.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.831+0000 7ffa13e00640 1 -- 172.21.15.47:0/3676361366 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ffa040ad780 con 0x7ffa1c0b3330 2024-04-20T10:29:02.832 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.832+0000 7ffa1b400640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff9f806c8f0 0x7ff9f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:02.833 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.833+0000 7ffa1b400640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff9f806c8f0 0x7ff9f806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007510 tx=0x7ffa0c0074a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:02.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:02.836+0000 7ffa13e00640 1 -- 172.21.15.47:0/3676361366 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ffa0407b760 con 0x7ffa1c0b3330 2024-04-20T10:29:03.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.083+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa1c0d07a0 con 0x7ffa1c0b3330 2024-04-20T10:29:03.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.084+0000 7ffa13e00640 1 -- 172.21.15.47:0/3676361366 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7ffa0407b060 con 0x7ffa1c0b3330 2024-04-20T10:29:03.085 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:03.085 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:29:03.085 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:29:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff9f806c8f0 msgr2=0x7ff9f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff9f806c8f0 0x7ff9f806edb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007510 tx=0x7ffa0c0074a0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 msgr2=0x7ffa1c14f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c14f770 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ffa0402bee0 tx=0x7ffa040046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 shutdown_connections 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa1c161d50 0x7ffa1c0cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff9f806c8f0 0x7ff9f806edb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ffa1c0b3330 0x7ffa1c14f770 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 --2- 172.21.15.47:0/3676361366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa1c06c6d0 0x7ffa1c0c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 >> 172.21.15.47:0/3676361366 conn(0x7ffa1c0d4450 msgr2=0x7ffa1c0b81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 shutdown_connections 2024-04-20T10:29:03.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.087+0000 7ffa20a00640 1 -- 172.21.15.47:0/3676361366 wait complete. 2024-04-20T10:29:03.107 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.323+0000 7f1397200640 1 Processor -- start 2024-04-20T10:29:03.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.323+0000 7f1397200640 1 -- start start 2024-04-20T10:29:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.325+0000 7f1397200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1390156c20 0x7f139006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.325+0000 7f1397200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f139015ca40 0x7f13900ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.325+0000 7f1397200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1390141c60 con 0x7f13900ad8e0 2024-04-20T10:29:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.325+0000 7f1397200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f139015cf90 con 0x7f139015ca40 2024-04-20T10:29:03.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.325+0000 7f1397200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f139015d270 con 0x7f1390156c20 2024-04-20T10:29:03.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.325+0000 7f138fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1390156c20 0x7f139006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.325+0000 7f1387e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f139015ca40 0x7f13900ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.326+0000 7f138fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1390156c20 0x7f139006b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:43992/0 (socket says 172.21.15.47:43992) 2024-04-20T10:29:03.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.326+0000 7f138fe00640 1 -- 172.21.15.47:0/406734062 learned_addr learned my addr 172.21.15.47:0/406734062 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:03.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.326+0000 7f138fe00640 1 -- 172.21.15.47:0/406734062 >> v2:172.21.15.175:3300/0 conn(0x7f139015ca40 msgr2=0x7f13900ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:03.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.326+0000 7f138fe00640 1 --2- 172.21.15.47:0/406734062 >> v2:172.21.15.175:3300/0 conn(0x7f139015ca40 0x7f13900ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.326+0000 7f138fe00640 1 -- 172.21.15.47:0/406734062 >> v1:172.21.15.175:6789/0 conn(0x7f13900ad8e0 legacy=0x7f13900d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.326+0000 7f138fe00640 1 -- 172.21.15.47:0/406734062 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139015d550 con 0x7f1390156c20 2024-04-20T10:29:03.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.328+0000 7f138fe00640 1 --2- 172.21.15.47:0/406734062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1390156c20 0x7f139006b900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1380009870 tx=0x7f138002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a889572d00f707 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f138f400640 1 -- 172.21.15.47:0/406734062 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380034070 con 0x7f1390156c20 2024-04-20T10:29:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f138f400640 1 -- 172.21.15.47:0/406734062 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1380030840 con 0x7f1390156c20 2024-04-20T10:29:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f138f400640 1 -- 172.21.15.47:0/406734062 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380030c10 con 0x7f1390156c20 2024-04-20T10:29:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f1397200640 1 -- 172.21.15.47:0/406734062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1390156c20 msgr2=0x7f139006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f1397200640 1 --2- 172.21.15.47:0/406734062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1390156c20 0x7f139006b900 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f1380009870 tx=0x7f138002b980 comp rx=0 tx=0).stop 2024-04-20T10:29:03.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f1397200640 1 -- 172.21.15.47:0/406734062 shutdown_connections 2024-04-20T10:29:03.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f1397200640 1 --2- 172.21.15.47:0/406734062 >> v2:172.21.15.175:3300/0 conn(0x7f139015ca40 0x7f13900ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f1397200640 1 --2- 172.21.15.47:0/406734062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1390156c20 0x7f139006b900 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.329+0000 7f1397200640 1 -- 172.21.15.47:0/406734062 >> 172.21.15.47:0/406734062 conn(0x7f13900b3350 msgr2=0x7f13900b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:03.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.330+0000 7f1397200640 1 -- 172.21.15.47:0/406734062 shutdown_connections 2024-04-20T10:29:03.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.330+0000 7f1397200640 1 -- 172.21.15.47:0/406734062 wait complete. 2024-04-20T10:29:03.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.332+0000 7f1397200640 1 Processor -- start 2024-04-20T10:29:03.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.332+0000 7f1397200640 1 -- start start 2024-04-20T10:29:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.333+0000 7f1397200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13900ad8e0 0x7f13901f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.333+0000 7f1397200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1390156c20 0x7f13900ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.333+0000 7f1397200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139015ca40 0x7f1390073440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.333+0000 7f1397200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f139006bfc0 con 0x7f13900ad8e0 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.333+0000 7f1397200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f139006be40 con 0x7f1390156c20 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.333+0000 7f1397200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13900d0a70 con 0x7f139015ca40 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13900ad8e0 0x7f13901f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13900ad8e0 0x7f13901f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59850/0 (socket says 172.21.15.47:59850) 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 -- 172.21.15.47:0/974944655 learned_addr learned my addr 172.21.15.47:0/974944655 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f1387e00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1390156c20 0x7f13900ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f1395200640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139015ca40 0x7f1390073440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 -- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139015ca40 msgr2=0x7f1390073440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139015ca40 0x7f1390073440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 -- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1390156c20 msgr2=0x7f13900ccb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1390156c20 0x7f13900ccb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f138fe00640 1 -- 172.21.15.47:0/974944655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13900cd380 con 0x7f13900ad8e0 2024-04-20T10:29:03.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.334+0000 7f1387e00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1390156c20 0x7f13900ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:29:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.335+0000 7f138fe00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13900ad8e0 0x7f13901f8c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1380000c00 tx=0x7f1380004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.335+0000 7f138cc00640 1 -- 172.21.15.47:0/974944655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380048070 con 0x7f13900ad8e0 2024-04-20T10:29:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.335+0000 7f138cc00640 1 -- 172.21.15.47:0/974944655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1380030640 con 0x7f13900ad8e0 2024-04-20T10:29:03.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.335+0000 7f138cc00640 1 -- 172.21.15.47:0/974944655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380034040 con 0x7f13900ad8e0 2024-04-20T10:29:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.335+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13900cd660 con 0x7f13900ad8e0 2024-04-20T10:29:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.335+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1390072c40 con 0x7f13900ad8e0 2024-04-20T10:29:03.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.337+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f139006fbc0 con 0x7f13900ad8e0 2024-04-20T10:29:03.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.338+0000 7f138cc00640 1 -- 172.21.15.47:0/974944655 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1380037070 con 0x7f13900ad8e0 2024-04-20T10:29:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.339+0000 7f138cc00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136c06c8f0 0x7f136c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.339+0000 7f138cc00640 1 -- 172.21.15.47:0/974944655 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f13800adb90 con 0x7f13900ad8e0 2024-04-20T10:29:03.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.340+0000 7f1387e00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136c06c8f0 0x7f136c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.341+0000 7f1387e00640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136c06c8f0 0x7f136c06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1390155090 tx=0x7f1378008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:03.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.346+0000 7f138cc00640 1 -- 172.21.15.47:0/974944655 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f139006fbc0 con 0x7f13900ad8e0 2024-04-20T10:29:03.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.551+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1390068980 con 0x7f13900ad8e0 2024-04-20T10:29:03.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.554+0000 7f138cc00640 1 -- 172.21.15.47:0/974944655 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f138007bd30 con 0x7f13900ad8e0 2024-04-20T10:29:03.555 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:03.555 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:03.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136c06c8f0 msgr2=0x7f136c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136c06c8f0 0x7f136c06edb0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1390155090 tx=0x7f1378008040 comp rx=0 tx=0).stop 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13900ad8e0 msgr2=0x7f13901f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13900ad8e0 0x7f13901f8c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1380000c00 tx=0x7f1380004730 comp rx=0 tx=0).stop 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 shutdown_connections 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f139015ca40 0x7f1390073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1390156c20 0x7f13900ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136c06c8f0 0x7f136c06edb0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 --2- 172.21.15.47:0/974944655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13900ad8e0 0x7f13901f8c80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 >> 172.21.15.47:0/974944655 conn(0x7f13900b3350 msgr2=0x7f139015db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 shutdown_connections 2024-04-20T10:29:03.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.557+0000 7f1397200640 1 -- 172.21.15.47:0/974944655 wait complete. 2024-04-20T10:29:03.574 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T10:29:03.574 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:03.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.774+0000 7ff82be00640 1 Processor -- start 2024-04-20T10:29:03.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.774+0000 7ff82be00640 1 -- start start 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff82c159e10 0x7ff82c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff82c06da90 0x7ff82c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff82c15cc90 con 0x7ff82c15a1f0 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82be00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff82c072900 con 0x7ff82c159e10 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff82c15ca40 con 0x7ff82c06da90 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff82c159e10 0x7ff82c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff82c159e10 0x7ff82c071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59858/0 (socket says 172.21.15.47:59858) 2024-04-20T10:29:03.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.776+0000 7ff82aa00640 1 -- 172.21.15.47:0/2684829758 learned_addr learned my addr 172.21.15.47:0/2684829758 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.777+0000 7ff82b400640 1 --2- 172.21.15.47:0/2684829758 >> v2:172.21.15.175:3300/0 conn(0x7ff82c06da90 0x7ff82c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.777+0000 7ff82aa00640 1 -- 172.21.15.47:0/2684829758 >> v2:172.21.15.175:3300/0 conn(0x7ff82c06da90 msgr2=0x7ff82c071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.777+0000 7ff82aa00640 1 --2- 172.21.15.47:0/2684829758 >> v2:172.21.15.175:3300/0 conn(0x7ff82c06da90 0x7ff82c071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.777+0000 7ff82aa00640 1 -- 172.21.15.47:0/2684829758 >> v1:172.21.15.175:6789/0 conn(0x7ff82c15a1f0 legacy=0x7ff82c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.777+0000 7ff82aa00640 1 -- 172.21.15.47:0/2684829758 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c15cf70 con 0x7ff82c159e10 2024-04-20T10:29:03.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.777+0000 7ff82b400640 1 --2- 172.21.15.47:0/2684829758 >> v2:172.21.15.175:3300/0 conn(0x7ff82c06da90 0x7ff82c071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.778+0000 7ff82aa00640 1 --2- 172.21.15.47:0/2684829758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c071a80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff82c050430 tx=0x7ff81c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2989922f6a289a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.778+0000 7ff82a000640 1 -- 172.21.15.47:0/2684829758 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c035020 con 0x7ff82c159e10 2024-04-20T10:29:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.778+0000 7ff82a000640 1 -- 172.21.15.47:0/2684829758 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff81c031870 con 0x7ff82c159e10 2024-04-20T10:29:03.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.778+0000 7ff82a000640 1 -- 172.21.15.47:0/2684829758 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c031c40 con 0x7ff82c159e10 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 -- 172.21.15.47:0/2684829758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 msgr2=0x7ff82c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 --2- 172.21.15.47:0/2684829758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c071a80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff82c050430 tx=0x7ff81c02b910 comp rx=0 tx=0).stop 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 -- 172.21.15.47:0/2684829758 shutdown_connections 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 --2- 172.21.15.47:0/2684829758 >> v2:172.21.15.175:3300/0 conn(0x7ff82c06da90 0x7ff82c071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 --2- 172.21.15.47:0/2684829758 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c071a80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 -- 172.21.15.47:0/2684829758 >> 172.21.15.47:0/2684829758 conn(0x7ff82c0d0040 msgr2=0x7ff82c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 -- 172.21.15.47:0/2684829758 shutdown_connections 2024-04-20T10:29:03.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.779+0000 7ff82be00640 1 -- 172.21.15.47:0/2684829758 wait complete. 2024-04-20T10:29:03.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.782+0000 7ff82be00640 1 Processor -- start 2024-04-20T10:29:03.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.782+0000 7ff82be00640 1 -- start start 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.783+0000 7ff82be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff82c06da90 0x7ff82c15c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.783+0000 7ff82be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c146a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.783+0000 7ff82be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff82c15a1f0 0x7ff82c15a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.783+0000 7ff82be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c15d810 con 0x7ff82c159e10 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.783+0000 7ff82be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff82c15d530 con 0x7ff82c15a1f0 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.783+0000 7ff82be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff82c06b900 con 0x7ff82c06da90 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.783+0000 7ff823e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c146a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c146a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59874/0 (socket says 172.21.15.47:59874) 2024-04-20T10:29:03.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 -- 172.21.15.47:0/3206925656 learned_addr learned my addr 172.21.15.47:0/3206925656 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff82b400640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff82c15a1f0 0x7ff82c15a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff82aa00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff82c06da90 0x7ff82c15c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 -- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff82c06da90 msgr2=0x7ff82c15c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff82c06da90 0x7ff82c15c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 -- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff82c15a1f0 msgr2=0x7ff82c15a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff82c15a1f0 0x7ff82c15a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:03.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 -- 172.21.15.47:0/3206925656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82c15b310 con 0x7ff82c159e10 2024-04-20T10:29:03.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.784+0000 7ff823e00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c146a10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b3f0 tx=0x7ff81400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:03.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.785+0000 7ff822a00640 1 -- 172.21.15.47:0/3206925656 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814015020 con 0x7ff82c159e10 2024-04-20T10:29:03.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.785+0000 7ff822a00640 1 -- 172.21.15.47:0/3206925656 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff814011830 con 0x7ff82c159e10 2024-04-20T10:29:03.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.785+0000 7ff822a00640 1 -- 172.21.15.47:0/3206925656 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814011c00 con 0x7ff82c159e10 2024-04-20T10:29:03.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.785+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82c15b5f0 con 0x7ff82c159e10 2024-04-20T10:29:03.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.785+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff82c15bbb0 con 0x7ff82c159e10 2024-04-20T10:29:03.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.786+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff82c075b60 con 0x7ff82c159e10 2024-04-20T10:29:03.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.788+0000 7ff822a00640 1 -- 172.21.15.47:0/3206925656 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff814018020 con 0x7ff82c159e10 2024-04-20T10:29:03.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.788+0000 7ff822a00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff80806c8f0 0x7ff80806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:03.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.789+0000 7ff822a00640 1 -- 172.21.15.47:0/3206925656 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff81408bf50 con 0x7ff82c159e10 2024-04-20T10:29:03.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.789+0000 7ff82aa00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff80806c8f0 0x7ff80806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:03.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.791+0000 7ff82aa00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff80806c8f0 0x7ff80806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff81c036000 tx=0x7ff81c03c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:03.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:03.793+0000 7ff822a00640 1 -- 172.21.15.47:0/3206925656 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff81401f020 con 0x7ff82c159e10 2024-04-20T10:29:04.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.040+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff82c1591e0 con 0x7ff82c159e10 2024-04-20T10:29:04.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.042+0000 7ff822a00640 1 -- 172.21.15.47:0/3206925656 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7ff8140172c0 con 0x7ff82c159e10 2024-04-20T10:29:04.042 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:04.042 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:29:04.043 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.044+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff80806c8f0 msgr2=0x7ff80806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.044+0000 7ff82be00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff80806c8f0 0x7ff80806edb0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7ff81c036000 tx=0x7ff81c03c040 comp rx=0 tx=0).stop 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.044+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 msgr2=0x7ff82c146a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.044+0000 7ff82be00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c146a10 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff81400b3f0 tx=0x7ff81400b8b0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 shutdown_connections 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff82c15a1f0 0x7ff82c15a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff80806c8f0 0x7ff80806edb0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff82c159e10 0x7ff82c146a10 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 --2- 172.21.15.47:0/3206925656 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff82c06da90 0x7ff82c15c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 >> 172.21.15.47:0/3206925656 conn(0x7ff82c0d0040 msgr2=0x7ff82c06f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:04.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 shutdown_connections 2024-04-20T10:29:04.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.045+0000 7ff82be00640 1 -- 172.21.15.47:0/3206925656 wait complete. 2024-04-20T10:29:04.065 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.260+0000 7ff4d4c00640 1 Processor -- start 2024-04-20T10:29:04.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.260+0000 7ff4d4c00640 1 -- start start 2024-04-20T10:29:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4d4c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff4d015a1f0 0x7ff4d0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4d4c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 0x7ff4d0073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4d4c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4d015cc90 con 0x7ff4d0159e10 2024-04-20T10:29:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4d4c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff4d0072900 con 0x7ff4d015a1f0 2024-04-20T10:29:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4d4c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d015ca40 con 0x7ff4d0072190 2024-04-20T10:29:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4cf400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff4d0159e10 0x7ff4d0071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43484/0 (socket says 172.21.15.47:43484) 2024-04-20T10:29:04.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4cf400640 1 -- 172.21.15.47:0/336007759 learned_addr learned my addr 172.21.15.47:0/336007759 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 0x7ff4d0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4cea00640 1 --2- 172.21.15.47:0/336007759 >> v2:172.21.15.175:3300/0 conn(0x7ff4d015a1f0 0x7ff4d0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.264 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.262+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2145230947 0 0) 0x7ff4d015cc90 con 0x7ff4d0159e10 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff4b4003520 con 0x7ff4d0159e10 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2910729825 0 0) 0x7ff4b4003520 con 0x7ff4d0159e10 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 msgr2=0x7ff4d0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 0x7ff4d0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 >> v2:172.21.15.175:3300/0 conn(0x7ff4d015a1f0 msgr2=0x7ff4d0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 --2- 172.21.15.47:0/336007759 >> v2:172.21.15.175:3300/0 conn(0x7ff4d015a1f0 0x7ff4d0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d015cf70 con 0x7ff4d0159e10 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7ff4c0001670 con 0x7ff4d0159e10 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 >> v1:172.21.15.47:6789/0 conn(0x7ff4d0159e10 legacy=0x7ff4d0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 0x7ff4b40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b4003f80 0x7ff4b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b4004d40 0x7ff4b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d015ca40 con 0x7ff4b4004d40 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4b400b9f0 con 0x7ff4b4003f80 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4b400bbf0 con 0x7ff4d0072190 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4cea00640 1 --2- 172.21.15.47:0/336007759 >> v2:172.21.15.175:3300/0 conn(0x7ff4d015a1f0 0x7ff4d0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.263+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b4004d40 0x7ff4b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.264+0000 7ff4cea00640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b4003f80 0x7ff4b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.264+0000 7ff4cf400640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 0x7ff4b40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.264+0000 7ff4cfe00640 1 -- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 msgr2=0x7ff4b40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.264+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4d0072190 0x7ff4b40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.264+0000 7ff4cfe00640 1 -- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b4003f80 msgr2=0x7ff4b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.264+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4b4003f80 0x7ff4b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.264+0000 7ff4cfe00640 1 -- 172.21.15.47:0/336007759 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4b4003520 con 0x7ff4b4004d40 2024-04-20T10:29:04.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.265+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b4004d40 0x7ff4b40051c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009fc0 tx=0x7ff4b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99006a9e70b055cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.265+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8039070 con 0x7ff4b4004d40 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.265+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4c802cc20 con 0x7ff4b4004d40 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.265+0000 7ff4ce000640 1 -- 172.21.15.47:0/336007759 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8009c10 con 0x7ff4b4004d40 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.266+0000 7ff4d4c00640 1 -- 172.21.15.47:0/336007759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b4004d40 msgr2=0x7ff4b40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.266+0000 7ff4d4c00640 1 --2- 172.21.15.47:0/336007759 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4b4004d40 0x7ff4b40051c0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009fc0 tx=0x7ff4b4005830 comp rx=0 tx=0).stop 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.266+0000 7ff4cfe00640 1 -- 172.21.15.47:0/336007759 reap_dead start 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.266+0000 7ff4d4c00640 1 -- 172.21.15.47:0/336007759 shutdown_connections 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.266+0000 7ff4d4c00640 1 -- 172.21.15.47:0/336007759 >> 172.21.15.47:0/336007759 conn(0x7ff4d00d0040 msgr2=0x7ff4d00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.266+0000 7ff4d4c00640 1 -- 172.21.15.47:0/336007759 shutdown_connections 2024-04-20T10:29:04.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.266+0000 7ff4d4c00640 1 -- 172.21.15.47:0/336007759 wait complete. 2024-04-20T10:29:04.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.268+0000 7ff4d4c00640 1 Processor -- start 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.268+0000 7ff4d4c00640 1 -- start start 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.269+0000 7ff4d4c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4b4004d40 0x7ff4d00c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.269+0000 7ff4d4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01f4ab0 0x7ff4d0154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.269+0000 7ff4d4c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d015af70 0x7ff4d015b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.269+0000 7ff4d4c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d015d810 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.269+0000 7ff4d4c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d015d530 con 0x7ff4d015af70 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.269+0000 7ff4d4c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4d006da90 con 0x7ff4b4004d40 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.269+0000 7ff4cea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01f4ab0 0x7ff4d0154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01f4ab0 0x7ff4d0154a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52140/0 (socket says 172.21.15.47:52140) 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 -- 172.21.15.47:0/426872888 learned_addr learned my addr 172.21.15.47:0/426872888 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cf400640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4b4004d40 0x7ff4d00c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cfe00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d015af70 0x7ff4d015b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 -- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4b4004d40 msgr2=0x7ff4d00c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4b4004d40 0x7ff4d00c97b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 -- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d015af70 msgr2=0x7ff4d015b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.271 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d015af70 0x7ff4d015b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 -- 172.21.15.47:0/426872888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d00c9e50 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.270+0000 7ff4cea00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01f4ab0 0x7ff4d0154a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00f550 tx=0x7ff4bc00fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.271+0000 7ff4c7400640 1 -- 172.21.15.47:0/426872888 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc018070 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.271+0000 7ff4c7400640 1 -- 172.21.15.47:0/426872888 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4bc004220 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.271+0000 7ff4c7400640 1 -- 172.21.15.47:0/426872888 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4bc014690 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.271+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d00ca130 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.271+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d00ca6f0 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.272 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.272+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d0075b60 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.273 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.273+0000 7ff4c7400640 1 -- 172.21.15.47:0/426872888 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4bc012070 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.274+0000 7ff4c7400640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4b006c8f0 0x7ff4b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.274+0000 7ff4c7400640 1 -- 172.21.15.47:0/426872888 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7ff4bc08bf40 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.274 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.275+0000 7ff4cf400640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4b006c8f0 0x7ff4b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.276+0000 7ff4cf400640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4b006c8f0 0x7ff4b006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4d006c1c0 tx=0x7ff4c0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:04.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.279+0000 7ff4c7400640 1 -- 172.21.15.47:0/426872888 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff4bc059fb0 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.524+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4d006c550 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.525+0000 7ff4c7400640 1 -- 172.21.15.47:0/426872888 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+5201 (secure 0 0 0) 0x7ff4bc0598b0 con 0x7ff4d01f4ab0 2024-04-20T10:29:04.525 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:04.525 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/4291009552","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":4291009552},{"type":"v1","addr":"172.21.15.47:6833","nonce":4291009552}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/1539467747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1539467747},{"type":"v1","addr":"172.21.15.47:6835","nonce":1539467747}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4282,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/1560845712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1560845712},{"type":"v1","addr":"172.21.15.175:6836","nonce":1560845712}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:27:32.532668+0000","modified":"2024-04-20T10:27:53.683601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4285},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4285":{"gid":4285,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2667922035","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2667922035},{"type":"v1","addr":"172.21.15.175:6837","nonce":2667922035}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4285,"qdb_cluster":[4285]},"id":1}]} 2024-04-20T10:29:04.526 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 9 2024-04-20T10:29:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4b006c8f0 msgr2=0x7ff4b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4b006c8f0 0x7ff4b006edb0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff4d006c1c0 tx=0x7ff4c0009040 comp rx=0 tx=0).stop 2024-04-20T10:29:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01f4ab0 msgr2=0x7ff4d0154a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01f4ab0 0x7ff4d0154a70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc00f550 tx=0x7ff4bc00fa10 comp rx=0 tx=0).stop 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 shutdown_connections 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4d015af70 0x7ff4d015b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4b006c8f0 0x7ff4b006edb0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4d01f4ab0 0x7ff4d0154a70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 --2- 172.21.15.47:0/426872888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4b4004d40 0x7ff4d00c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 >> 172.21.15.47:0/426872888 conn(0x7ff4d00d0040 msgr2=0x7ff4d006ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 shutdown_connections 2024-04-20T10:29:04.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.528+0000 7ff4d4c00640 1 -- 172.21.15.47:0/426872888 wait complete. 2024-04-20T10:29:04.547 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:04.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.761+0000 7f1389c00640 1 Processor -- start 2024-04-20T10:29:04.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.761+0000 7f1389c00640 1 -- start start 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.763+0000 7f1389c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f138415b400 0x7f1384072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.763+0000 7f1389c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f138415fa40 0x7f138406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.763+0000 7f1389c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138415c7f0 con 0x7f138415eec0 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.763+0000 7f1389c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f138406d280 con 0x7f138415b400 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.763+0000 7f1389c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f138406ccc0 con 0x7f138415fa40 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.763+0000 7f1383e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f138415b400 0x7f1384072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.764+0000 7f1383e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f138415b400 0x7f1384072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52146/0 (socket says 172.21.15.47:52146) 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.764+0000 7f1383e00640 1 -- 172.21.15.47:0/4058401819 learned_addr learned my addr 172.21.15.47:0/4058401819 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.764+0000 7f1389200640 1 --2- 172.21.15.47:0/4058401819 >> v2:172.21.15.175:3300/0 conn(0x7f138415fa40 0x7f138406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.764+0000 7f1383e00640 1 -- 172.21.15.47:0/4058401819 >> v2:172.21.15.175:3300/0 conn(0x7f138415fa40 msgr2=0x7f138406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.764+0000 7f1383e00640 1 --2- 172.21.15.47:0/4058401819 >> v2:172.21.15.175:3300/0 conn(0x7f138415fa40 0x7f138406ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.764+0000 7f1383e00640 1 -- 172.21.15.47:0/4058401819 >> v1:172.21.15.175:6789/0 conn(0x7f138415eec0 legacy=0x7f138415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:04.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.764+0000 7f1383e00640 1 -- 172.21.15.47:0/4058401819 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138406cfa0 con 0x7f138415b400 2024-04-20T10:29:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.765+0000 7f1383e00640 1 --2- 172.21.15.47:0/4058401819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415b400 0x7f1384072d00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1384072250 tx=0x7f137402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=198291c430560c1f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.765+0000 7f1383400640 1 -- 172.21.15.47:0/4058401819 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137402ca80 con 0x7f138415b400 2024-04-20T10:29:04.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1383400640 1 -- 172.21.15.47:0/4058401819 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f137402cc20 con 0x7f138415b400 2024-04-20T10:29:04.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1383400640 1 -- 172.21.15.47:0/4058401819 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13740343f0 con 0x7f138415b400 2024-04-20T10:29:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 -- 172.21.15.47:0/4058401819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415b400 msgr2=0x7f1384072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 --2- 172.21.15.47:0/4058401819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415b400 0x7f1384072d00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1384072250 tx=0x7f137402bee0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 -- 172.21.15.47:0/4058401819 shutdown_connections 2024-04-20T10:29:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 --2- 172.21.15.47:0/4058401819 >> v2:172.21.15.175:3300/0 conn(0x7f138415fa40 0x7f138406ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 --2- 172.21.15.47:0/4058401819 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415b400 0x7f1384072d00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 -- 172.21.15.47:0/4058401819 >> 172.21.15.47:0/4058401819 conn(0x7f13840d0040 msgr2=0x7f13840d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 -- 172.21.15.47:0/4058401819 shutdown_connections 2024-04-20T10:29:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.766+0000 7f1389c00640 1 -- 172.21.15.47:0/4058401819 wait complete. 2024-04-20T10:29:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.768+0000 7f1389c00640 1 Processor -- start 2024-04-20T10:29:04.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.768+0000 7f1389c00640 1 -- start start 2024-04-20T10:29:04.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.769+0000 7f1389c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f138415b400 0x7f1384156b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.769+0000 7f1389c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415eec0 0x7f13840c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.769+0000 7f1389c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f138415fa40 0x7f13840b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.769+0000 7f1389c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1384073240 con 0x7f138415eec0 2024-04-20T10:29:04.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.769+0000 7f1389c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f138406e720 con 0x7f138415fa40 2024-04-20T10:29:04.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.769+0000 7f1389c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f138406f180 con 0x7f138415b400 2024-04-20T10:29:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.769+0000 7f137b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415eec0 0x7f13840c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415eec0 0x7f13840c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52156/0 (socket says 172.21.15.47:52156) 2024-04-20T10:29:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 -- 172.21.15.47:0/42912379 learned_addr learned my addr 172.21.15.47:0/42912379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f1383e00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f138415b400 0x7f1384156b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f1389200640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f138415fa40 0x7f13840b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 -- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f138415b400 msgr2=0x7f1384156b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f138415b400 0x7f1384156b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 -- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f138415fa40 msgr2=0x7f13840b3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f138415fa40 0x7f13840b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 -- 172.21.15.47:0/42912379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1384158e40 con 0x7f138415eec0 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.770+0000 7f137b400640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415eec0 0x7f13840c92f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f136c00c410 tx=0x7f136c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.771+0000 7f1380c00640 1 -- 172.21.15.47:0/42912379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c017070 con 0x7f138415eec0 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.771+0000 7f1380c00640 1 -- 172.21.15.47:0/42912379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f136c0035f0 con 0x7f138415eec0 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.771+0000 7f1380c00640 1 -- 172.21.15.47:0/42912379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f136c003a20 con 0x7f138415eec0 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.771+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1384159120 con 0x7f138415eec0 2024-04-20T10:29:04.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.771+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841596e0 con 0x7f138415eec0 2024-04-20T10:29:04.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.772+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1384075b60 con 0x7f138415eec0 2024-04-20T10:29:04.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.773+0000 7f1380c00640 1 -- 172.21.15.47:0/42912379 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f136c003bc0 con 0x7f138415eec0 2024-04-20T10:29:04.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.774+0000 7f1380c00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136006c8f0 0x7f136006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:04.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.775+0000 7f1380c00640 1 -- 172.21.15.47:0/42912379 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7f136c08b220 con 0x7f138415eec0 2024-04-20T10:29:04.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.775+0000 7f1383e00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136006c8f0 0x7f136006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:04.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.776+0000 7f1383e00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136006c8f0 0x7f136006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1374035000 tx=0x7f1374030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:04.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.779+0000 7f1380c00640 1 -- 172.21.15.47:0/42912379 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f136c0592a0 con 0x7f138415eec0 2024-04-20T10:29:04.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.984+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f138415abb0 con 0x7f138415eec0 2024-04-20T10:29:04.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.987+0000 7f1380c00640 1 -- 172.21.15.47:0/42912379 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v29) ==== 74+0+18718 (secure 0 0 0) 0x7f136c058ba0 con 0x7f138415eec0 2024-04-20T10:29:04.988 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:04.988 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":29,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:28:14.324450+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:27:30.221834+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-20T10:27:31.436137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-20T10:27:35.089154+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"27","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.31","mappings":[{"from":0,"to":7}]},{"pgid":"4.1","mappings":[{"from":3,"to":4}]},{"pgid":"4.9","mappings":[{"from":1,"to":0}]},{"pgid":"4.1c","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:04.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136006c8f0 msgr2=0x7f136006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136006c8f0 0x7f136006edb0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1374035000 tx=0x7f1374030000 comp rx=0 tx=0).stop 2024-04-20T10:29:04.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415eec0 msgr2=0x7f13840c92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415eec0 0x7f13840c92f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f136c00c410 tx=0x7f136c00c8d0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 shutdown_connections 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f138415fa40 0x7f13840b3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f136006c8f0 0x7f136006edb0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f138415eec0 0x7f13840c92f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 --2- 172.21.15.47:0/42912379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f138415b400 0x7f1384156b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 >> 172.21.15.47:0/42912379 conn(0x7f13840d0040 msgr2=0x7f13840b0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 shutdown_connections 2024-04-20T10:29:04.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:04.990+0000 7f1389c00640 1 -- 172.21.15.47:0/42912379 wait complete. 2024-04-20T10:29:05.008 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T10:29:05.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.215+0000 7fe0bf200640 1 Processor -- start 2024-04-20T10:29:05.216 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.216+0000 7fe0bf200640 1 -- start start 2024-04-20T10:29:05.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.217+0000 7fe0bf200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe0c015a1f0 0x7fe0c0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.217+0000 7fe0bf200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe0c0072190 0x7fe0c0073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.217+0000 7fe0bf200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe0c015cc90 con 0x7fe0c0159e10 2024-04-20T10:29:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.217+0000 7fe0bf200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c0072900 con 0x7fe0c0072190 2024-04-20T10:29:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.217+0000 7fe0bf200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe0c015ca40 con 0x7fe0c015a1f0 2024-04-20T10:29:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0bde00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe0c0159e10 0x7fe0c0071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43490/0 (socket says 172.21.15.47:43490) 2024-04-20T10:29:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0bde00640 1 -- 172.21.15.47:0/3381421919 learned_addr learned my addr 172.21.15.47:0/3381421919 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:05.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0be800640 1 --2- 172.21.15.47:0/3381421919 >> v2:172.21.15.47:3300/0 conn(0x7fe0c0072190 0x7fe0c0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0b7e00640 1 --2- 172.21.15.47:0/3381421919 >> v2:172.21.15.175:3300/0 conn(0x7fe0c015a1f0 0x7fe0c0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0be800640 1 -- 172.21.15.47:0/3381421919 >> v2:172.21.15.175:3300/0 conn(0x7fe0c015a1f0 msgr2=0x7fe0c0072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0be800640 1 --2- 172.21.15.47:0/3381421919 >> v2:172.21.15.175:3300/0 conn(0x7fe0c015a1f0 0x7fe0c0072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0be800640 1 -- 172.21.15.47:0/3381421919 >> v1:172.21.15.47:6789/0 conn(0x7fe0c0159e10 legacy=0x7fe0c0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.218+0000 7fe0be800640 1 -- 172.21.15.47:0/3381421919 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c015cf70 con 0x7fe0c0072190 2024-04-20T10:29:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.219+0000 7fe0be800640 1 --2- 172.21.15.47:0/3381421919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0072190 0x7fe0c0073420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0050400 tx=0x7fe0b802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f516896760d77bf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bd400640 1 -- 172.21.15.47:0/3381421919 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8039070 con 0x7fe0c0072190 2024-04-20T10:29:05.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bd400640 1 -- 172.21.15.47:0/3381421919 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0b802cc70 con 0x7fe0c0072190 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bd400640 1 -- 172.21.15.47:0/3381421919 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b8034660 con 0x7fe0c0072190 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 -- 172.21.15.47:0/3381421919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0072190 msgr2=0x7fe0c0073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 --2- 172.21.15.47:0/3381421919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0072190 0x7fe0c0073420 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0050400 tx=0x7fe0b802bec0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 -- 172.21.15.47:0/3381421919 shutdown_connections 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 --2- 172.21.15.47:0/3381421919 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0072190 0x7fe0c0073420 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 --2- 172.21.15.47:0/3381421919 >> v2:172.21.15.175:3300/0 conn(0x7fe0c015a1f0 0x7fe0c0072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 -- 172.21.15.47:0/3381421919 >> 172.21.15.47:0/3381421919 conn(0x7fe0c00d0040 msgr2=0x7fe0c00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 -- 172.21.15.47:0/3381421919 shutdown_connections 2024-04-20T10:29:05.221 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.220+0000 7fe0bf200640 1 -- 172.21.15.47:0/3381421919 wait complete. 2024-04-20T10:29:05.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.222+0000 7fe0bf200640 1 Processor -- start 2024-04-20T10:29:05.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.222+0000 7fe0bf200640 1 -- start start 2024-04-20T10:29:05.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.223+0000 7fe0bf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0072190 0x7fe0c01475d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.223+0000 7fe0bf200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0159e10 0x7fe0c015ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.223+0000 7fe0bf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c015a1f0 0x7fe0c0157bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.223+0000 7fe0bf200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c015d810 con 0x7fe0c0159e10 2024-04-20T10:29:05.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.223+0000 7fe0bf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c015d530 con 0x7fe0c015a1f0 2024-04-20T10:29:05.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.223+0000 7fe0bf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0c006da90 con 0x7fe0c0072190 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0159e10 0x7fe0c015ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0159e10 0x7fe0c015ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52186/0 (socket says 172.21.15.47:52186) 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 -- 172.21.15.47:0/2827219306 learned_addr learned my addr 172.21.15.47:0/2827219306 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0bde00640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0072190 0x7fe0c01475d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0be800640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c015a1f0 0x7fe0c0157bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 -- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0072190 msgr2=0x7fe0c01475d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0072190 0x7fe0c01475d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 -- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c015a1f0 msgr2=0x7fe0c0157bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c015a1f0 0x7fe0c0157bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 -- 172.21.15.47:0/2827219306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c0158700 con 0x7fe0c0159e10 2024-04-20T10:29:05.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.224+0000 7fe0b7e00640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0159e10 0x7fe0c015ba50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d600 tx=0x7fe0a800dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:05.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.225+0000 7fe0b6000640 1 -- 172.21.15.47:0/2827219306 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8004240 con 0x7fe0c0159e10 2024-04-20T10:29:05.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.225+0000 7fe0b6000640 1 -- 172.21.15.47:0/2827219306 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe0a80043e0 con 0x7fe0c0159e10 2024-04-20T10:29:05.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.225+0000 7fe0b6000640 1 -- 172.21.15.47:0/2827219306 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0a8003830 con 0x7fe0c0159e10 2024-04-20T10:29:05.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.225+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c01589e0 con 0x7fe0c0159e10 2024-04-20T10:29:05.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.225+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c0158fa0 con 0x7fe0c0159e10 2024-04-20T10:29:05.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.226+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0c0075b60 con 0x7fe0c0159e10 2024-04-20T10:29:05.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.227+0000 7fe0b6000640 1 -- 172.21.15.47:0/2827219306 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe0a80039d0 con 0x7fe0c0159e10 2024-04-20T10:29:05.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.228+0000 7fe0b6000640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe09c06c8f0 0x7fe09c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.228+0000 7fe0b6000640 1 -- 172.21.15.47:0/2827219306 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6648+0+0 (secure 0 0 0) 0x7fe0a808c200 con 0x7fe0c0159e10 2024-04-20T10:29:05.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.229+0000 7fe0bde00640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe09c06c8f0 0x7fe09c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.230+0000 7fe0bde00640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe09c06c8f0 0x7fe09c06edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0000f60 tx=0x7fe0b0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:05.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.232+0000 7fe0b6000640 1 -- 172.21.15.47:0/2827219306 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe0a805a270 con 0x7fe0c0159e10 2024-04-20T10:29:05.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.478+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe0c0156370 con 0x7fe0c0159e10 2024-04-20T10:29:05.479 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-20T10:29:05.479+0000 7f06e1000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:29:05.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.483+0000 7fe0b6000640 1 -- 172.21.15.47:0/2827219306 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) ==== 161+0+0 (secure 0 0 0) 0x7fe0a8059b70 con 0x7fe0c0159e10 2024-04-20T10:29:05.484 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T10:29:05.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe09c06c8f0 msgr2=0x7fe09c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe09c06c8f0 0x7fe09c06edb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0000f60 tx=0x7fe0b0009040 comp rx=0 tx=0).stop 2024-04-20T10:29:05.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0159e10 msgr2=0x7fe0c015ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0159e10 0x7fe0c015ba50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe0a800d600 tx=0x7fe0a800dac0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 shutdown_connections 2024-04-20T10:29:05.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0c015a1f0 0x7fe0c0157bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe09c06c8f0 0x7fe09c06edb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe0c0159e10 0x7fe0c015ba50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 --2- 172.21.15.47:0/2827219306 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0c0072190 0x7fe0c01475d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.488+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 >> 172.21.15.47:0/2827219306 conn(0x7fe0c00d0040 msgr2=0x7fe0c0071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:05.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.489+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 shutdown_connections 2024-04-20T10:29:05.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.489+0000 7fe0bf200640 1 -- 172.21.15.47:0/2827219306 wait complete. 2024-04-20T10:29:05.508 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T10:29:05.523 INFO:tasks.ceph.mds.b.smithi175.stderr: -5824> 2024-04-20T10:27:38.215+0000 7fc5b5400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:29:05.616 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T10:29:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.717+0000 7f72a4400640 1 Processor -- start 2024-04-20T10:29:05.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.717+0000 7f72a4400640 1 -- start start 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.718+0000 7f72a4400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 0x7f729c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.718+0000 7f72a4400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f729c0ad8e0 0x7f729c0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.718+0000 7f72a4400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f729c141c60 con 0x7f729c156c20 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.718+0000 7f72a4400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f729c06c010 con 0x7f729c0ad8e0 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.718+0000 7f72a4400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f729c15cf90 con 0x7f729c15ca40 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a1a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f729c156c20 0x7f729c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43498/0 (socket says 172.21.15.47:43498) 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a1a00640 1 -- 172.21.15.47:0/1087618328 learned_addr learned my addr 172.21.15.47:0/1087618328 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a2400640 1 --2- 172.21.15.47:0/1087618328 >> v2:172.21.15.47:3300/0 conn(0x7f729c0ad8e0 0x7f729c0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a1000640 1 --2- 172.21.15.47:0/1087618328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 0x7f729c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f729be00640 1 -- 172.21.15.47:0/1087618328 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 56873729 0 0) 0x7f729c141c60 con 0x7f729c156c20 2024-04-20T10:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f729be00640 1 -- 172.21.15.47:0/1087618328 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7284003520 con 0x7f729c156c20 2024-04-20T10:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a2400640 1 -- 172.21.15.47:0/1087618328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 msgr2=0x7f729c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a2400640 1 --2- 172.21.15.47:0/1087618328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 0x7f729c0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a2400640 1 -- 172.21.15.47:0/1087618328 >> v1:172.21.15.47:6789/0 conn(0x7f729c156c20 legacy=0x7f729c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.719+0000 7f72a2400640 1 -- 172.21.15.47:0/1087618328 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c15d270 con 0x7f729c0ad8e0 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.721+0000 7f72a2400640 1 --2- 172.21.15.47:0/1087618328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c0d0360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7294009d90 tx=0x7f729402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b68e64f3a33d74 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.721+0000 7f729be00640 1 -- 172.21.15.47:0/1087618328 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294039070 con 0x7f729c0ad8e0 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.721+0000 7f729be00640 1 -- 172.21.15.47:0/1087618328 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f729402cc50 con 0x7f729c0ad8e0 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.721+0000 7f729be00640 1 -- 172.21.15.47:0/1087618328 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294034440 con 0x7f729c0ad8e0 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.721+0000 7f72a4400640 1 -- 172.21.15.47:0/1087618328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 msgr2=0x7f729c0d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.721+0000 7f72a4400640 1 --2- 172.21.15.47:0/1087618328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c0d0360 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7294009d90 tx=0x7f729402bea0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.722+0000 7f72a4400640 1 -- 172.21.15.47:0/1087618328 shutdown_connections 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.722+0000 7f72a4400640 1 --2- 172.21.15.47:0/1087618328 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c0d0360 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.722+0000 7f72a4400640 1 --2- 172.21.15.47:0/1087618328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 0x7f729c0ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.722+0000 7f72a4400640 1 -- 172.21.15.47:0/1087618328 >> 172.21.15.47:0/1087618328 conn(0x7f729c0b3350 msgr2=0x7f729c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.722+0000 7f72a4400640 1 -- 172.21.15.47:0/1087618328 shutdown_connections 2024-04-20T10:29:05.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.722+0000 7f72a4400640 1 -- 172.21.15.47:0/1087618328 wait complete. 2024-04-20T10:29:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.723+0000 7f72a4400640 1 Processor -- start 2024-04-20T10:29:05.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.724+0000 7f72a4400640 1 -- start start 2024-04-20T10:29:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.724+0000 7f72a4400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.724+0000 7f72a4400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f729c156c20 0x7f729c0731e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a4400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 0x7f729c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a4400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f729c0d08a0 con 0x7f729c0ad8e0 2024-04-20T10:29:05.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a4400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f729c15d830 con 0x7f729c156c20 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a4400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f729c0d0a20 con 0x7f729c15ca40 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c146840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52198/0 (socket says 172.21.15.47:52198) 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 -- 172.21.15.47:0/1235208590 learned_addr learned my addr 172.21.15.47:0/1235208590 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1000640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f729c156c20 0x7f729c0731e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 -- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 msgr2=0x7f729c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 0x7f729c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 -- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f729c156c20 msgr2=0x7f729c0731e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f729c156c20 0x7f729c0731e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:05.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.725+0000 7f72a1a00640 1 -- 172.21.15.47:0/1235208590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c146d80 con 0x7f729c0ad8e0 2024-04-20T10:29:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.726+0000 7f72a1a00640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c146840 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f729000c460 tx=0x7f729000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.726+0000 7f7299600640 1 -- 172.21.15.47:0/1235208590 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290017070 con 0x7f729c0ad8e0 2024-04-20T10:29:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.726+0000 7f7299600640 1 -- 172.21.15.47:0/1235208590 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7290005ce0 con 0x7f729c0ad8e0 2024-04-20T10:29:05.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.726+0000 7f7299600640 1 -- 172.21.15.47:0/1235208590 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7290007290 con 0x7f729c0ad8e0 2024-04-20T10:29:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.726+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c0ce820 con 0x7f729c0ad8e0 2024-04-20T10:29:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.726+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c0cede0 con 0x7f729c0ad8e0 2024-04-20T10:29:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.727+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f729c068980 con 0x7f729c0ad8e0 2024-04-20T10:29:05.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.729+0000 7f7299600640 1 -- 172.21.15.47:0/1235208590 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f729001a070 con 0x7f729c0ad8e0 2024-04-20T10:29:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.729+0000 7f7299600640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f727c06c8f0 0x7f727c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.730+0000 7f7299600640 1 -- 172.21.15.47:0/1235208590 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6734+0+0 (secure 0 0 0) 0x7f729008b2f0 con 0x7f729c0ad8e0 2024-04-20T10:29:05.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.730+0000 7f72a1000640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f727c06c8f0 0x7f727c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:05.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.731+0000 7f72a1000640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f727c06c8f0 0x7f727c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:05.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.734+0000 7f7299600640 1 -- 172.21.15.47:0/1235208590 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7290059320 con 0x7f729c0ad8e0 2024-04-20T10:29:05.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:05.968+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f729c0cfd90 con 0x7f729c0ad8e0 2024-04-20T10:29:06.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.487+0000 7f7299600640 1 -- 172.21.15.47:0/1235208590 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f7290058c20 con 0x7f729c0ad8e0 2024-04-20T10:29:06.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f727c06c8f0 msgr2=0x7f727c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f727c06c8f0 0x7f727c06edb0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7288006fd0 tx=0x7f7288008040 comp rx=0 tx=0).stop 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 msgr2=0x7f729c146840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c146840 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f729000c460 tx=0x7f729000c920 comp rx=0 tx=0).stop 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 shutdown_connections 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f729c15ca40 0x7f729c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f729c156c20 0x7f729c0731e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f727c06c8f0 0x7f727c06edb0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 --2- 172.21.15.47:0/1235208590 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f729c0ad8e0 0x7f729c146840 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 >> 172.21.15.47:0/1235208590 conn(0x7f729c0b3350 msgr2=0x7f729c15db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 shutdown_connections 2024-04-20T10:29:06.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.491+0000 7f72a4400640 1 -- 172.21.15.47:0/1235208590 wait complete. 2024-04-20T10:29:06.513 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T10:29:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.733+0000 7f03b9e00640 1 Processor -- start 2024-04-20T10:29:06.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.733+0000 7f03b9e00640 1 -- start start 2024-04-20T10:29:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.735+0000 7f03b9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 0x7f03b4073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.735+0000 7f03b9e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b415cc20 con 0x7f03b406f870 2024-04-20T10:29:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.735+0000 7f03b9e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03b415cec0 con 0x7f03b4071a60 2024-04-20T10:29:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.735+0000 7f03b9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03b415d7f0 con 0x7f03b415a130 2024-04-20T10:29:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.735+0000 7f03b9400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f03b406f870 0x7f03b406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43506/0 (socket says 172.21.15.47:43506) 2024-04-20T10:29:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.735+0000 7f03b9400640 1 -- 172.21.15.47:0/1784673698 learned_addr learned my addr 172.21.15.47:0/1784673698 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b8a00640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 0x7f03b4073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2041842986 0 0) 0x7f03b415cc20 con 0x7f03b406f870 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0398003520 con 0x7f03b406f870 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2188762305 0 0) 0x7f0398003520 con 0x7f03b406f870 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 msgr2=0x7f03b4073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 0x7f03b4073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 >> v1:172.21.15.175:6789/0 conn(0x7f03b4071a60 legacy=0x7f03b406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b415d1a0 con 0x7f03b406f870 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f03ac001670 con 0x7f03b406f870 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 >> v1:172.21.15.47:6789/0 conn(0x7f03b406f870 legacy=0x7f03b406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03980039e0 0x7f0398003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0398004800 0x7f0398004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 0x7f039800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f03b415d7f0 con 0x7f03980039e0 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0398005520 con 0x7f0398004800 2024-04-20T10:29:06.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.736+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f039800bf00 con 0x7f03b415a130 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b8a00640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03980039e0 0x7f0398003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b3e00640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0398004800 0x7f0398004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b9400640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 0x7f039800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b8a00640 1 -- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 msgr2=0x7f039800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b8a00640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 0x7f039800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b8a00640 1 -- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0398004800 msgr2=0x7f0398004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b8a00640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0398004800 0x7f0398004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b8a00640 1 -- 172.21.15.47:0/1784673698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0398003520 con 0x7f03980039e0 2024-04-20T10:29:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.737+0000 7f03b9400640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03b415a130 0x7f039800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.738+0000 7f03b8a00640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03980039e0 0x7f0398003e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009fc0 tx=0x7f0398005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc631037a5f4758 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4039070 con 0x7f03980039e0 2024-04-20T10:29:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a402cbf0 con 0x7f03980039e0 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b3400640 1 -- 172.21.15.47:0/1784673698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4009c10 con 0x7f03980039e0 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b9e00640 1 -- 172.21.15.47:0/1784673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03980039e0 msgr2=0x7f0398003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b9e00640 1 --2- 172.21.15.47:0/1784673698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03980039e0 0x7f0398003e00 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009fc0 tx=0x7f0398005830 comp rx=0 tx=0).stop 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b9400640 1 -- 172.21.15.47:0/1784673698 reap_dead start 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b9e00640 1 -- 172.21.15.47:0/1784673698 shutdown_connections 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b9e00640 1 -- 172.21.15.47:0/1784673698 >> 172.21.15.47:0/1784673698 conn(0x7f03b40cffa0 msgr2=0x7f03b40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b9e00640 1 -- 172.21.15.47:0/1784673698 shutdown_connections 2024-04-20T10:29:06.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.739+0000 7f03b9e00640 1 -- 172.21.15.47:0/1784673698 wait complete. 2024-04-20T10:29:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.741+0000 7f03b9e00640 1 Processor -- start 2024-04-20T10:29:06.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.741+0000 7f03b9e00640 1 -- start start 2024-04-20T10:29:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.742+0000 7f03b9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03980039e0 0x7f03b414f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.742+0000 7f03b9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03b40b2fd0 0x7f03b41541f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.742+0000 7f03b9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b4154730 0x7f03b415b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.742+0000 7f03b9e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f03b406c190 con 0x7f03b40b2fd0 2024-04-20T10:29:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.742+0000 7f03b9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03b406c010 con 0x7f03b4154730 2024-04-20T10:29:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.742+0000 7f03b9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03b406c310 con 0x7f03980039e0 2024-04-20T10:29:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03b40b2fd0 0x7f03b41541f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03b40b2fd0 0x7f03b41541f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52210/0 (socket says 172.21.15.47:52210) 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 -- 172.21.15.47:0/550210214 learned_addr learned my addr 172.21.15.47:0/550210214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b8a00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03980039e0 0x7f03b414f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b9400640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b4154730 0x7f03b415b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 -- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03980039e0 msgr2=0x7f03b414f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03980039e0 0x7f03b414f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 -- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b4154730 msgr2=0x7f03b415b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b4154730 0x7f03b415b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:06.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.743+0000 7f03b3e00640 1 -- 172.21.15.47:0/550210214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b409a9f0 con 0x7f03b40b2fd0 2024-04-20T10:29:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.744+0000 7f03b3e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03b40b2fd0 0x7f03b41541f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f03a000e550 tx=0x7f03a000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.744+0000 7f03b0c00640 1 -- 172.21.15.47:0/550210214 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a00052e0 con 0x7f03b40b2fd0 2024-04-20T10:29:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.744+0000 7f03b0c00640 1 -- 172.21.15.47:0/550210214 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f03a0018070 con 0x7f03b40b2fd0 2024-04-20T10:29:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.744+0000 7f03b0c00640 1 -- 172.21.15.47:0/550210214 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a0003810 con 0x7f03b40b2fd0 2024-04-20T10:29:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.744+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b4154bd0 con 0x7f03b40b2fd0 2024-04-20T10:29:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.744+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b409b250 con 0x7f03b40b2fd0 2024-04-20T10:29:06.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.745+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03b4075b30 con 0x7f03b40b2fd0 2024-04-20T10:29:06.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.746+0000 7f03b0c00640 1 -- 172.21.15.47:0/550210214 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f03a00039b0 con 0x7f03b40b2fd0 2024-04-20T10:29:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.747+0000 7f03b0c00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f039406c8f0 0x7f039406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:06.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.747+0000 7f03b0c00640 1 -- 172.21.15.47:0/550210214 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 6734+0+0 (secure 0 0 0) 0x7f03a008bc00 con 0x7f03b40b2fd0 2024-04-20T10:29:06.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.748+0000 7f03b8a00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f039406c8f0 0x7f039406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:06.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.749+0000 7f03b8a00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f039406c8f0 0x7f039406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03a4036000 tx=0x7f03a402e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:06.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.752+0000 7f03b0c00640 1 -- 172.21.15.47:0/550210214 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f03a0059c20 con 0x7f03b40b2fd0 2024-04-20T10:29:06.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:06.962+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f03b406d010 con 0x7f03b40b2fd0 2024-04-20T10:29:07.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.665+0000 7f03b0c00640 1 -- 172.21.15.47:0/550210214 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v31) ==== 181+0+0 (secure 0 0 0) 0x7f03a0059520 con 0x7f03b40b2fd0 2024-04-20T10:29:07.666 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.670+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f039406c8f0 msgr2=0x7f039406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f039406c8f0 0x7f039406edb0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f03a4036000 tx=0x7f03a402e000 comp rx=0 tx=0).stop 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03b40b2fd0 msgr2=0x7f03b41541f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03b40b2fd0 0x7f03b41541f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f03a000e550 tx=0x7f03a000ea10 comp rx=0 tx=0).stop 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 shutdown_connections 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03b4154730 0x7f03b415b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f039406c8f0 0x7f039406edb0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f03b40b2fd0 0x7f03b41541f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 --2- 172.21.15.47:0/550210214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03980039e0 0x7f03b414f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 >> 172.21.15.47:0/550210214 conn(0x7f03b40cffa0 msgr2=0x7f03b40ccda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:07.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.671+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 shutdown_connections 2024-04-20T10:29:07.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.672+0000 7f03b9e00640 1 -- 172.21.15.47:0/550210214 wait complete. 2024-04-20T10:29:07.700 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T10:29:07.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.916+0000 7fa8b5e00640 1 Processor -- start 2024-04-20T10:29:07.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.916+0000 7fa8b5e00640 1 -- start start 2024-04-20T10:29:07.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.917+0000 7fa8b5e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa8b015ca40 0x7fa8b00ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:07.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.917+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b00d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:07.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.917+0000 7fa8b5e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8b0141c60 con 0x7fa8b0156c20 2024-04-20T10:29:07.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.917+0000 7fa8b5e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa8b006c010 con 0x7fa8b015ca40 2024-04-20T10:29:07.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.917+0000 7fa8b5e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b015cf90 con 0x7fa8b00ad8e0 2024-04-20T10:29:07.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.917+0000 7fa8aea00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa8b0156c20 0x7fa8b006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43522/0 (socket says 172.21.15.47:43522) 2024-04-20T10:29:07.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.917+0000 7fa8aea00640 1 -- 172.21.15.47:0/1503380886 learned_addr learned my addr 172.21.15.47:0/1503380886 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:07.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8a6000640 1 --2- 172.21.15.47:0/1503380886 >> v2:172.21.15.47:3300/0 conn(0x7fa8b015ca40 0x7fa8b00ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:07.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8af400640 1 --2- 172.21.15.47:0/1503380886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b00d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:07.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8ae000640 1 -- 172.21.15.47:0/1503380886 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2819648301 0 0) 0x7fa8b0141c60 con 0x7fa8b0156c20 2024-04-20T10:29:07.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8ae000640 1 -- 172.21.15.47:0/1503380886 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa894003520 con 0x7fa8b0156c20 2024-04-20T10:29:07.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8a6000640 1 -- 172.21.15.47:0/1503380886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 msgr2=0x7fa8b00d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:07.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8a6000640 1 --2- 172.21.15.47:0/1503380886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b00d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8a6000640 1 -- 172.21.15.47:0/1503380886 >> v1:172.21.15.47:6789/0 conn(0x7fa8b0156c20 legacy=0x7fa8b006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:07.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.918+0000 7fa8a6000640 1 -- 172.21.15.47:0/1503380886 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b015d270 con 0x7fa8b015ca40 2024-04-20T10:29:07.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.919+0000 7fa8a6000640 1 --2- 172.21.15.47:0/1503380886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b015ca40 0x7fa8b00ad3a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa8980097a0 tx=0x7fa89802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9c6f194813f0e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:07.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8ae000640 1 -- 172.21.15.47:0/1503380886 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035020 con 0x7fa8b015ca40 2024-04-20T10:29:07.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8ae000640 1 -- 172.21.15.47:0/1503380886 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa898031810 con 0x7fa8b015ca40 2024-04-20T10:29:07.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8ae000640 1 -- 172.21.15.47:0/1503380886 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898031be0 con 0x7fa8b015ca40 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1503380886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b015ca40 msgr2=0x7fa8b00ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/1503380886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b015ca40 0x7fa8b00ad3a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa8980097a0 tx=0x7fa89802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1503380886 shutdown_connections 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/1503380886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b00d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/1503380886 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b015ca40 0x7fa8b00ad3a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1503380886 >> 172.21.15.47:0/1503380886 conn(0x7fa8b00b3350 msgr2=0x7fa8b00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1503380886 shutdown_connections 2024-04-20T10:29:07.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.920+0000 7fa8b5e00640 1 -- 172.21.15.47:0/1503380886 wait complete. 2024-04-20T10:29:07.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.922+0000 7fa8b5e00640 1 Processor -- start 2024-04-20T10:29:07.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.922+0000 7fa8b5e00640 1 -- start start 2024-04-20T10:29:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.923+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b014f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.923+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b0156c20 0x7fa8b0151020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.923+0000 7fa8b5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8b015ca40 0x7fa8b0073490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.923+0000 7fa8b5e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b00d08a0 con 0x7fa8b0156c20 2024-04-20T10:29:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.923+0000 7fa8b5e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8b015d830 con 0x7fa8b015ca40 2024-04-20T10:29:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.923+0000 7fa8b5e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8b00d0a20 con 0x7fa8b00ad8e0 2024-04-20T10:29:07.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b0156c20 0x7fa8b0151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b0156c20 0x7fa8b0151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52240/0 (socket says 172.21.15.47:52240) 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 -- 172.21.15.47:0/508253275 learned_addr learned my addr 172.21.15.47:0/508253275 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8aea00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b014f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 -- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 msgr2=0x7fa8b014f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b014f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 -- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8b015ca40 msgr2=0x7fa8b0073490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8af400640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8b015ca40 0x7fa8b0073490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8b015ca40 0x7fa8b0073490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.924+0000 7fa8a6000640 1 -- 172.21.15.47:0/508253275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8b01e5e00 con 0x7fa8b0156c20 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.925+0000 7fa8a6000640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b0156c20 0x7fa8b0151020 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa8980098c0 tx=0x7fa8980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.925+0000 7fa8a7e00640 1 -- 172.21.15.47:0/508253275 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898035040 con 0x7fa8b0156c20 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.925+0000 7fa8a7e00640 1 -- 172.21.15.47:0/508253275 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa898039cf0 con 0x7fa8b0156c20 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.925+0000 7fa8a7e00640 1 -- 172.21.15.47:0/508253275 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898039ea0 con 0x7fa8b0156c20 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.925+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8b01e60e0 con 0x7fa8b0156c20 2024-04-20T10:29:07.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.925+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8b01e66a0 con 0x7fa8b0156c20 2024-04-20T10:29:07.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.926+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8b0068980 con 0x7fa8b0156c20 2024-04-20T10:29:07.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.927+0000 7fa8a7e00640 1 -- 172.21.15.47:0/508253275 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa898038020 con 0x7fa8b0156c20 2024-04-20T10:29:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.928+0000 7fa8a7e00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88c06c8f0 0x7fa88c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:07.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.928+0000 7fa8a7e00640 1 -- 172.21.15.47:0/508253275 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 6296+0+0 (secure 0 0 0) 0x7fa8980adf90 con 0x7fa8b0156c20 2024-04-20T10:29:07.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.929+0000 7fa8aea00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88c06c8f0 0x7fa88c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:07.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.930+0000 7fa8aea00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88c06c8f0 0x7fa88c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0000f60 tx=0x7fa8a0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:07.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:07.933+0000 7fa8a7e00640 1 -- 172.21.15.47:0/508253275 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa89807c0d0 con 0x7fa8b0156c20 2024-04-20T10:29:08.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.137+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa8b00ae860 con 0x7fa8b0156c20 2024-04-20T10:29:08.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.669+0000 7fa8a7e00640 1 -- 172.21.15.47:0/508253275 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v32) ==== 169+0+0 (secure 0 0 0) 0x7fa89807b9d0 con 0x7fa8b0156c20 2024-04-20T10:29:08.670 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-20T10:29:08.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.674+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88c06c8f0 msgr2=0x7fa88c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:08.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.674+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88c06c8f0 0x7fa88c06edb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0000f60 tx=0x7fa8a0009040 comp rx=0 tx=0).stop 2024-04-20T10:29:08.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.674+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b0156c20 msgr2=0x7fa8b0151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:08.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.674+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b0156c20 0x7fa8b0151020 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa8980098c0 tx=0x7fa8980046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.675+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 shutdown_connections 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.675+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8b015ca40 0x7fa8b0073490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.675+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88c06c8f0 0x7fa88c06edb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.675+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8b0156c20 0x7fa8b0151020 secure :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa8980098c0 tx=0x7fa8980046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.675+0000 7fa8b5e00640 1 --2- 172.21.15.47:0/508253275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8b00ad8e0 0x7fa8b014f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.675+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 >> 172.21.15.47:0/508253275 conn(0x7fa8b00b3350 msgr2=0x7fa8b015ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.676+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 shutdown_connections 2024-04-20T10:29:08.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.676+0000 7fa8b5e00640 1 -- 172.21.15.47:0/508253275 wait complete. 2024-04-20T10:29:08.703 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-20T10:29:08.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.918+0000 7f0349200640 1 Processor -- start 2024-04-20T10:29:08.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.918+0000 7f0349200640 1 -- start start 2024-04-20T10:29:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.919+0000 7f0349200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f034415a130 0x7f0344073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.919+0000 7f0349200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0344071a60 0x7f03440739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.919+0000 7f0349200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034406f160 0x7f034406f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.919+0000 7f0349200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f034415d7f0 con 0x7f0344071a60 2024-04-20T10:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.919+0000 7f0349200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f034415cc20 con 0x7f034415a130 2024-04-20T10:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.919+0000 7f0349200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f034415cec0 con 0x7f034406f160 2024-04-20T10:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.919+0000 7f033be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0344071a60 0x7f03440739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f033be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f0344071a60 0x7f03440739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52242/0 (socket says 172.21.15.47:52242) 2024-04-20T10:29:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f033be00640 1 -- 172.21.15.47:0/2162959627 learned_addr learned my addr 172.21.15.47:0/2162959627 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f0343400640 1 --2- 172.21.15.47:0/2162959627 >> v2:172.21.15.175:3300/0 conn(0x7f034415a130 0x7f0344073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f0343e00640 1 --2- 172.21.15.47:0/2162959627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034406f160 0x7f034406f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f033be00640 1 -- 172.21.15.47:0/2162959627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034406f160 msgr2=0x7f034406f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f033be00640 1 --2- 172.21.15.47:0/2162959627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034406f160 0x7f034406f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f033be00640 1 -- 172.21.15.47:0/2162959627 >> v2:172.21.15.175:3300/0 conn(0x7f034415a130 msgr2=0x7f0344073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f033be00640 1 --2- 172.21.15.47:0/2162959627 >> v2:172.21.15.175:3300/0 conn(0x7f034415a130 0x7f0344073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.920+0000 7f033be00640 1 -- 172.21.15.47:0/2162959627 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034415d1a0 con 0x7f0344071a60 2024-04-20T10:29:08.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.921+0000 7f033be00640 1 --2- 172.21.15.47:0/2162959627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0344071a60 0x7f03440739b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f03440cda90 tx=0x7f032c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bc0a066b7ff831 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.922+0000 7f0342a00640 1 -- 172.21.15.47:0/2162959627 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c02e070 con 0x7f0344071a60 2024-04-20T10:29:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.922+0000 7f0342a00640 1 -- 172.21.15.47:0/2162959627 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032c037070 con 0x7f0344071a60 2024-04-20T10:29:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.922+0000 7f0342a00640 1 -- 172.21.15.47:0/2162959627 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032c031a00 con 0x7f0344071a60 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.922+0000 7f0349200640 1 -- 172.21.15.47:0/2162959627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0344071a60 msgr2=0x7f03440739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.922+0000 7f0349200640 1 --2- 172.21.15.47:0/2162959627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0344071a60 0x7f03440739b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f03440cda90 tx=0x7f032c02b880 comp rx=0 tx=0).stop 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.923+0000 7f0349200640 1 -- 172.21.15.47:0/2162959627 shutdown_connections 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.923+0000 7f0349200640 1 --2- 172.21.15.47:0/2162959627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f034406f160 0x7f034406f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.923+0000 7f0349200640 1 --2- 172.21.15.47:0/2162959627 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0344071a60 0x7f03440739b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.923+0000 7f0349200640 1 --2- 172.21.15.47:0/2162959627 >> v2:172.21.15.175:3300/0 conn(0x7f034415a130 0x7f0344073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.923+0000 7f0349200640 1 -- 172.21.15.47:0/2162959627 >> 172.21.15.47:0/2162959627 conn(0x7f03440cffa0 msgr2=0x7f03440d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.923+0000 7f0349200640 1 -- 172.21.15.47:0/2162959627 shutdown_connections 2024-04-20T10:29:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.923+0000 7f0349200640 1 -- 172.21.15.47:0/2162959627 wait complete. 2024-04-20T10:29:08.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.924+0000 7f0349200640 1 Processor -- start 2024-04-20T10:29:08.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.925+0000 7f0349200640 1 -- start start 2024-04-20T10:29:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.925+0000 7f0349200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034406f160 0x7f0344153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.925+0000 7f0349200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0344071a60 0x7f03440b5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.925+0000 7f0349200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034415a130 0x7f034409b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0349200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0344160100 con 0x7f034406f160 2024-04-20T10:29:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0349200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f034406fb30 con 0x7f034415a130 2024-04-20T10:29:08.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0349200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0344160280 con 0x7f0344071a60 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034406f160 0x7f0344153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034406f160 0x7f0344153c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:52254/0 (socket says 172.21.15.47:52254) 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 -- 172.21.15.47:0/640230740 learned_addr learned my addr 172.21.15.47:0/640230740 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f033be00640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0344071a60 0x7f03440b5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 -- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0344071a60 msgr2=0x7f03440b5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0344071a60 0x7f03440b5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 -- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034415a130 msgr2=0x7f034409b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343e00640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034415a130 0x7f034409b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034415a130 0x7f034409b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.926+0000 7f0343400640 1 -- 172.21.15.47:0/640230740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034409aca0 con 0x7f034406f160 2024-04-20T10:29:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.927+0000 7f0343400640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034406f160 0x7f0344153c60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f033400d5f0 tx=0x7f033400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.927+0000 7f033b400640 1 -- 172.21.15.47:0/640230740 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340125f0 con 0x7f034406f160 2024-04-20T10:29:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.927+0000 7f033b400640 1 -- 172.21.15.47:0/640230740 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0334019070 con 0x7f034406f160 2024-04-20T10:29:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.927+0000 7f033b400640 1 -- 172.21.15.47:0/640230740 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03340129d0 con 0x7f034406f160 2024-04-20T10:29:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.927+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03440b4460 con 0x7f034406f160 2024-04-20T10:29:08.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.927+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03440b4a20 con 0x7f034406f160 2024-04-20T10:29:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.928+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0344075b30 con 0x7f034406f160 2024-04-20T10:29:08.929 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.929+0000 7f033b400640 1 -- 172.21.15.47:0/640230740 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0334012b70 con 0x7f034406f160 2024-04-20T10:29:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.930+0000 7f033b400640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f032006c8f0 0x7f032006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:08.930 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.930+0000 7f033b400640 1 -- 172.21.15.47:0/640230740 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5853+0+0 (secure 0 0 0) 0x7f033408bfe0 con 0x7f034406f160 2024-04-20T10:29:08.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.931+0000 7f033be00640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f032006c8f0 0x7f032006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:08.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.932+0000 7f033be00640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f032006c8f0 0x7f032006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f032c036000 tx=0x7f032c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:08.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:08.935+0000 7f033b400640 1 -- 172.21.15.47:0/640230740 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f033405a320 con 0x7f034406f160 2024-04-20T10:29:09.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.144+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f034406d010 con 0x7f034406f160 2024-04-20T10:29:09.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.685+0000 7f033b400640 1 -- 172.21.15.47:0/640230740 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v33) ==== 178+0+0 (secure 0 0 0) 0x7f0334059c20 con 0x7f034406f160 2024-04-20T10:29:09.686 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data_ec' removed 2024-04-20T10:29:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.691+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f032006c8f0 msgr2=0x7f032006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.691+0000 7f0349200640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f032006c8f0 0x7f032006edb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f032c036000 tx=0x7f032c008030 comp rx=0 tx=0).stop 2024-04-20T10:29:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.691+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034406f160 msgr2=0x7f0344153c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:09.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.691+0000 7f0349200640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034406f160 0x7f0344153c60 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f033400d5f0 tx=0x7f033400dab0 comp rx=0 tx=0).stop 2024-04-20T10:29:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.693+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 shutdown_connections 2024-04-20T10:29:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.693+0000 7f0349200640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f034415a130 0x7f034409b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.693+0000 7f0349200640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0344071a60 0x7f03440b5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.693+0000 7f0349200640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f032006c8f0 0x7f032006edb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.693+0000 7f0349200640 1 --2- 172.21.15.47:0/640230740 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f034406f160 0x7f0344153c60 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.693+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 >> 172.21.15.47:0/640230740 conn(0x7f03440cffa0 msgr2=0x7f0344154880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:09.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.694+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 shutdown_connections 2024-04-20T10:29:09.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:09.694+0000 7f0349200640 1 -- 172.21.15.47:0/640230740 wait complete. 2024-04-20T10:29:09.719 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T10:29:09.719 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T10:29:09.720 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T10:29:09.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:29:09.720 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T10:29:09.720 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T10:29:09.720 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T10:29:09.720 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:29:09.721 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:29:09.721 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T10:29:09.721 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T10:29:09.721 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:29:09.721 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T10:29:09.721 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T10:29:09.722 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T10:29:09.722 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:29:09.743 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T10:29:09.743 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:29:09.744 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T10:29:09.744 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T10:29:09.745 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T10:29:09.745 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T10:29:09.791 INFO:tasks.ceph.mds.c:Started 2024-04-20T10:29:09.792 DEBUG:teuthology.parallel:result is None 2024-04-20T10:29:09.792 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:29:09.793 DEBUG:teuthology.parallel:result is None 2024-04-20T10:29:09.799 INFO:tasks.ceph.mds.d:Started 2024-04-20T10:29:09.799 DEBUG:teuthology.parallel:result is None 2024-04-20T10:29:09.945 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T10:29:09.948 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T10:29:09.971 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-20T10:29:11.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-20T10:29:15.743 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T10:29:15.743 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T10:29:15.751 INFO:tasks.ceph.mds.a:Started 2024-04-20T10:29:15.752 DEBUG:teuthology.parallel:result is None 2024-04-20T10:29:15.752 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T10:29:15.910 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-20T10:29:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.942+0000 7f4913200640 1 Processor -- start 2024-04-20T10:29:15.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.942+0000 7f4913200640 1 -- start start 2024-04-20T10:29:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.943+0000 7f4913200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f491415a1f0 0x7f4914072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.943+0000 7f4913200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4914072190 0x7f4914073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.943+0000 7f4913200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f491415cc90 con 0x7f4914159e10 2024-04-20T10:29:15.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.943+0000 7f4913200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4914072900 con 0x7f491415a1f0 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f4913200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f491415ca40 con 0x7f4914072190 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f490be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f491415a1f0 0x7f4914072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f490be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f491415a1f0 0x7f4914072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58516/0 (socket says 172.21.15.47:58516) 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f490be00640 1 -- 172.21.15.47:0/1256783072 learned_addr learned my addr 172.21.15.47:0/1256783072 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f4911e00640 1 --1- 172.21.15.47:0/1256783072 >> v1:172.21.15.47:6789/0 conn(0x7f4914159e10 0x7f4914071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43388/0 (socket says 172.21.15.47:43388) 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f4912800640 1 --2- 172.21.15.47:0/1256783072 >> v2:172.21.15.175:3300/0 conn(0x7f4914072190 0x7f4914073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f490be00640 1 -- 172.21.15.47:0/1256783072 >> v2:172.21.15.175:3300/0 conn(0x7f4914072190 msgr2=0x7f4914073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f490be00640 1 --2- 172.21.15.47:0/1256783072 >> v2:172.21.15.175:3300/0 conn(0x7f4914072190 0x7f4914073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f490be00640 1 -- 172.21.15.47:0/1256783072 >> v1:172.21.15.47:6789/0 conn(0x7f4914159e10 legacy=0x7f4914071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f490be00640 1 -- 172.21.15.47:0/1256783072 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491415cf70 con 0x7f491415a1f0 2024-04-20T10:29:15.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.944+0000 7f4911400640 1 -- 172.21.15.47:0/1256783072 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3116281291 0 0) 0x7f491415cc90 con 0x7f4914159e10 2024-04-20T10:29:15.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.945+0000 7f490be00640 1 --2- 172.21.15.47:0/1256783072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491415a1f0 0x7f4914072ee0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f491415daf0 tx=0x7f48fc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e73bf676869a815 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4911400640 1 -- 172.21.15.47:0/1256783072 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc034070 con 0x7f491415a1f0 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4911400640 1 -- 172.21.15.47:0/1256783072 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f48fc030860 con 0x7f491415a1f0 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4911400640 1 -- 172.21.15.47:0/1256783072 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48fc030c30 con 0x7f491415a1f0 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4913200640 1 -- 172.21.15.47:0/1256783072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491415a1f0 msgr2=0x7f4914072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4913200640 1 --2- 172.21.15.47:0/1256783072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491415a1f0 0x7f4914072ee0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f491415daf0 tx=0x7f48fc02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4913200640 1 -- 172.21.15.47:0/1256783072 shutdown_connections 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4913200640 1 --2- 172.21.15.47:0/1256783072 >> v2:172.21.15.175:3300/0 conn(0x7f4914072190 0x7f4914073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4913200640 1 --2- 172.21.15.47:0/1256783072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f491415a1f0 0x7f4914072ee0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.946+0000 7f4913200640 1 -- 172.21.15.47:0/1256783072 >> 172.21.15.47:0/1256783072 conn(0x7f49140d0040 msgr2=0x7f49140d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.947+0000 7f4913200640 1 -- 172.21.15.47:0/1256783072 shutdown_connections 2024-04-20T10:29:15.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.947+0000 7f4913200640 1 -- 172.21.15.47:0/1256783072 wait complete. 2024-04-20T10:29:15.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.948+0000 7f4913200640 1 Processor -- start 2024-04-20T10:29:15.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.948+0000 7f4913200640 1 -- start start 2024-04-20T10:29:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.949+0000 7f4913200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4914072190 0x7f491415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.949+0000 7f4913200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4914159e10 0x7f4914146c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.949+0000 7f4913200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f491415a1f0 0x7f491414aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.949+0000 7f4913200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f491415d810 con 0x7f4914072190 2024-04-20T10:29:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.949+0000 7f4913200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f491415d530 con 0x7f4914159e10 2024-04-20T10:29:15.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.949+0000 7f4913200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f491406da90 con 0x7f491415a1f0 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4914072190 0x7f491415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4914072190 0x7f491415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58530/0 (socket says 172.21.15.47:58530) 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 -- 172.21.15.47:0/1104218298 learned_addr learned my addr 172.21.15.47:0/1104218298 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f490be00640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4914159e10 0x7f4914146c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 -- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f491415a1f0 msgr2=0x7f491414aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f491415a1f0 0x7f491414aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 -- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4914159e10 msgr2=0x7f4914146c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4914159e10 0x7f4914146c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 -- 172.21.15.47:0/1104218298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491414d050 con 0x7f4914072190 2024-04-20T10:29:15.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.950+0000 7f4911e00640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4914072190 0x7f491415bc60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f490400c4d0 tx=0x7f490400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.951+0000 7f490a000640 1 -- 172.21.15.47:0/1104218298 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904018070 con 0x7f4914072190 2024-04-20T10:29:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.951+0000 7f490a000640 1 -- 172.21.15.47:0/1104218298 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4904007500 con 0x7f4914072190 2024-04-20T10:29:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.951+0000 7f490a000640 1 -- 172.21.15.47:0/1104218298 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4904004a20 con 0x7f4914072190 2024-04-20T10:29:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.951+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f491415c360 con 0x7f4914072190 2024-04-20T10:29:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.951+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4914147190 con 0x7f4914072190 2024-04-20T10:29:15.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.952+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4914075b60 con 0x7f4914072190 2024-04-20T10:29:15.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.953+0000 7f490a000640 1 -- 172.21.15.47:0/1104218298 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4904004bc0 con 0x7f4914072190 2024-04-20T10:29:15.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.954+0000 7f490a000640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f48f006c8f0 0x7f48f006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.954+0000 7f490a000640 1 -- 172.21.15.47:0/1104218298 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5335+0+0 (secure 0 0 0) 0x7f490408ac90 con 0x7f4914072190 2024-04-20T10:29:15.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.954+0000 7f490be00640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f48f006c8f0 0x7f48f006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:15.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.956+0000 7f490be00640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f48f006c8f0 0x7f48f006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f48fc036000 tx=0x7f48fc008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:15.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:15.958+0000 7f490a000640 1 -- 172.21.15.47:0/1104218298 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4904059220 con 0x7f4914072190 2024-04-20T10:29:16.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:16.181+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f491406d220 con 0x7f4914072190 2024-04-20T10:29:17.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.750+0000 7f490a000640 1 -- 172.21.15.47:0/1104218298 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) ==== 97+0+0 (secure 0 0 0) 0x7f4904058b20 con 0x7f4914072190 2024-04-20T10:29:17.750 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f48f006c8f0 msgr2=0x7f48f006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f48f006c8f0 0x7f48f006edb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f48fc036000 tx=0x7f48fc008030 comp rx=0 tx=0).stop 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4914072190 msgr2=0x7f491415bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4914072190 0x7f491415bc60 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f490400c4d0 tx=0x7f490400c990 comp rx=0 tx=0).stop 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 shutdown_connections 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f491415a1f0 0x7f491414aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4914159e10 0x7f4914146c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f48f006c8f0 0x7f48f006edb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 --2- 172.21.15.47:0/1104218298 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4914072190 0x7f491415bc60 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 >> 172.21.15.47:0/1104218298 conn(0x7f49140d0040 msgr2=0x7f4914071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 shutdown_connections 2024-04-20T10:29:17.755 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.754+0000 7f4913200640 1 -- 172.21.15.47:0/1104218298 wait complete. 2024-04-20T10:29:17.779 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T10:29:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.993+0000 7f69ef800640 1 Processor -- start 2024-04-20T10:29:17.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.993+0000 7f69ef800640 1 -- start start 2024-04-20T10:29:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.995+0000 7f69ef800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f69e815a050 0x7f69e806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.995+0000 7f69ef800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 0x7f69e80aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.995+0000 7f69ef800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f69e806bce0 con 0x7f69e80af4d0 2024-04-20T10:29:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.995+0000 7f69ef800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f69e806d2c0 con 0x7f69e815a050 2024-04-20T10:29:17.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.995+0000 7f69ef800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69e806c8f0 con 0x7f69e80ad520 2024-04-20T10:29:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.995+0000 7f69ed800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f69e80af4d0 0x7f69e8071ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43400/0 (socket says 172.21.15.47:43400) 2024-04-20T10:29:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.995+0000 7f69ed800640 1 -- 172.21.15.47:0/2549821342 learned_addr learned my addr 172.21.15.47:0/2549821342 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69ece00640 1 --2- 172.21.15.47:0/2549821342 >> v2:172.21.15.175:3300/0 conn(0x7f69e815a050 0x7f69e806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7e00640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 0x7f69e80aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3446786651 0 0) 0x7f69e806bce0 con 0x7f69e80af4d0 2024-04-20T10:29:17.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f69d0003520 con 0x7f69e80af4d0 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1667282448 0 0) 0x7f69d0003520 con 0x7f69e80af4d0 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 msgr2=0x7f69e80aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 0x7f69e80aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 >> v2:172.21.15.175:3300/0 conn(0x7f69e815a050 msgr2=0x7f69e806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 --2- 172.21.15.47:0/2549821342 >> v2:172.21.15.175:3300/0 conn(0x7f69e815a050 0x7f69e806cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e806bfc0 con 0x7f69e80af4d0 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f69e00019f0 con 0x7f69e80af4d0 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 >> v1:172.21.15.47:6789/0 conn(0x7f69e80af4d0 legacy=0x7f69e8071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d00039b0 0x7f69d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 0x7f69d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:17.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f69e806d2c0 con 0x7f69d00039b0 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69e806c8f0 con 0x7f69d0004800 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.996+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69d000b9c0 con 0x7f69e80ad520 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ece00640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d00039b0 0x7f69d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69e7e00640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ed800640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 0x7f69d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ece00640 1 -- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 msgr2=0x7f69d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ece00640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 0x7f69d00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ece00640 1 -- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d0004800 msgr2=0x7f69d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ece00640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ece00640 1 -- 172.21.15.47:0/2549821342 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69d0003520 con 0x7f69d00039b0 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69ed800640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e80ad520 0x7f69d00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.997+0000 7f69e7e00640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.998+0000 7f69ece00640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d00039b0 0x7f69d0003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009ac0 tx=0x7f69d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f6a481f17ddeef server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc039070 con 0x7f69d00039b0 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69dc02c710 con 0x7f69d00039b0 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69e7400640 1 -- 172.21.15.47:0/2549821342 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69dc009690 con 0x7f69d00039b0 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69ef800640 1 -- 172.21.15.47:0/2549821342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d00039b0 msgr2=0x7f69d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69ef800640 1 --2- 172.21.15.47:0/2549821342 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69d00039b0 0x7f69d0003dd0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f69dc009ac0 tx=0x7f69d0005830 comp rx=0 tx=0).stop 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69ed800640 1 -- 172.21.15.47:0/2549821342 reap_dead start 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69ef800640 1 -- 172.21.15.47:0/2549821342 shutdown_connections 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:17.999+0000 7f69ef800640 1 -- 172.21.15.47:0/2549821342 >> 172.21.15.47:0/2549821342 conn(0x7f69e80b1d70 msgr2=0x7f69e80b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.000+0000 7f69ef800640 1 -- 172.21.15.47:0/2549821342 shutdown_connections 2024-04-20T10:29:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.000+0000 7f69ef800640 1 -- 172.21.15.47:0/2549821342 wait complete. 2024-04-20T10:29:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.001+0000 7f69ef800640 1 Processor -- start 2024-04-20T10:29:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.002+0000 7f69ef800640 1 -- start start 2024-04-20T10:29:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.002+0000 7f69ef800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d00039b0 0x7f69e81543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.002+0000 7f69ef800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e80cdbe0 0x7f69e81f4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.002+0000 7f69ef800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e8151440 0x7f69e81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.002+0000 7f69ef800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f69e80af920 con 0x7f69e80cdbe0 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.002+0000 7f69ef800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69e806c580 con 0x7f69d00039b0 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.002+0000 7f69ef800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69e80ad190 con 0x7f69e8151440 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e80cdbe0 0x7f69e81f4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e80cdbe0 0x7f69e81f4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58556/0 (socket says 172.21.15.47:58556) 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 -- 172.21.15.47:0/2851003974 learned_addr learned my addr 172.21.15.47:0/2851003974 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69ece00640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d00039b0 0x7f69e81543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 -- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e8151440 msgr2=0x7f69e81518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e8151440 0x7f69e81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 -- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d00039b0 msgr2=0x7f69e81543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d00039b0 0x7f69e81543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.003+0000 7f69e7e00640 1 -- 172.21.15.47:0/2851003974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69e81f4650 con 0x7f69e80cdbe0 2024-04-20T10:29:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.004+0000 7f69e7e00640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e80cdbe0 0x7f69e81f4110 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f69d40108b0 tx=0x7f69d4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.004+0000 7f69e4c00640 1 -- 172.21.15.47:0/2851003974 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d400f040 con 0x7f69e80cdbe0 2024-04-20T10:29:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.004+0000 7f69e4c00640 1 -- 172.21.15.47:0/2851003974 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f69d400eb60 con 0x7f69e80cdbe0 2024-04-20T10:29:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.004+0000 7f69e4c00640 1 -- 172.21.15.47:0/2851003974 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d401c400 con 0x7f69e80cdbe0 2024-04-20T10:29:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.004+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69e80ce380 con 0x7f69e80cdbe0 2024-04-20T10:29:18.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.004+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69e80ce940 con 0x7f69e80cdbe0 2024-04-20T10:29:18.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.005+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69e8068980 con 0x7f69e80cdbe0 2024-04-20T10:29:18.006 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.006+0000 7f69e4c00640 1 -- 172.21.15.47:0/2851003974 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f69d401c5a0 con 0x7f69e80cdbe0 2024-04-20T10:29:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.007+0000 7f69e4c00640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69c806c920 0x7f69c806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.007+0000 7f69e4c00640 1 -- 172.21.15.47:0/2851003974 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f69d408bff0 con 0x7f69e80cdbe0 2024-04-20T10:29:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.008+0000 7f69ece00640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69c806c920 0x7f69c806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.009+0000 7f69ece00640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69c806c920 0x7f69c806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f69dc036000 tx=0x7f69dc030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:18.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.012+0000 7f69e4c00640 1 -- 172.21.15.47:0/2851003974 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f69d4018030 con 0x7f69e80cdbe0 2024-04-20T10:29:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.287+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f69e80b2ab0 con 0x7f69e80cdbe0 2024-04-20T10:29:18.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.288+0000 7f69e4c00640 1 -- 172.21.15.47:0/2851003974 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7f69d405a5f0 con 0x7f69e80cdbe0 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmFJVUGBAAWWv9IYGbahiOT/gLniKBOw==", 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:29:18.289 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmTUa9NRAAqv2igp+zzxGgHeyPdWTaMg==", 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.290 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmSGsVJxAA9UQVq15Djoww3cb3RYpnvw==", 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-04-20T10:29:18.291 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNm6XxVCBAA2JRaNbycMKfG+H5MRYd/BQ==", 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBZmCNmA30rIRAAVq4KbsC5TmRgR2fuF6Ioow==", 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.292 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBcmCNmTJtrLxAAJmSqkjORGtnWDI0g4to51A==", 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.293 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBfmCNmxQDLORAAVyQxgnBHyR13yRBAGF+zNQ==", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBjmCNmL4FwChAA11Oo/q6Csl2I4KAhh7pPDg==", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.294 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBpmCNmc0UlHhAA1/xDr2QsraxlgUXGj5zKJA==", 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-04-20T10:29:18.295 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBsmCNmyZ92ORAAkP7I8i8eANOvmITsE4Sv8Q==", 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBwmCNmag3+CRAAr+SJC24tUSlkRpbdRnZydA==", 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.296 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBzmCNmF+LYFBAAXXFXRj5Har31kqZtljDnWg==", 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNmrHlNFRAAY90gnx6m90huEzUUDh9rtw==", 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.297 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNmaFnFJxAAIjkRNWpCrcCA8aq28sHHqQ==", 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.298 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNmSmJdHhAAsLDDN4yZBlkSAAPckAd0rw==", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNm4Nm7MRAA8yPbfNcKnJjd0QiaBTOcMw==", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:29:18.299 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBVmCNmv2ymFRAAJV17P7iHLBSO40PM78whWA==", 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.300 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mds", 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7mCNmg4YCLRAAml1kHEf0AbsyvM6h2dnANQ==", 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mds" 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-mgr", 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7mCNmvZcCLRAAGHTEfugX0jvWJmNkaSo7Rg==", 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-osd", 2024-04-20T10:29:18.301 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7mCNmcaYCLRAAB7PSDCIOcYYgIrJTJkW27A==", 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-osd" 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd", 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7mCNmaLQCLRAAFDXyEi9kCsZWS5MkFc2cwg==", 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-20T10:29:18.302 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7mCNmmMQCLRAAO3C7a3T7zKuCsG0bdhwJCw==", 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.bootstrap-rgw", 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQB7mCNmOtQCLRAABwf5+QrOi9YM94lmgND0TQ==", 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmVaHQDRAA1jKTsPvvO24I4zZ996gBQg==", 2024-04-20T10:29:18.303 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNm862fAxAA3IcbrK5u7lUlerIjzUqIwA==", 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:29:18.304 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.305 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:18.305 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T10:29:18.305 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:29:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69c806c920 msgr2=0x7f69c806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69c806c920 0x7f69c806ede0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f69dc036000 tx=0x7f69dc030000 comp rx=0 tx=0).stop 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e80cdbe0 msgr2=0x7f69e81f4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e80cdbe0 0x7f69e81f4110 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f69d40108b0 tx=0x7f69d4010d70 comp rx=0 tx=0).stop 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 shutdown_connections 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69e8151440 0x7f69e81518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f69c806c920 0x7f69c806ede0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69e80cdbe0 0x7f69e81f4110 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 --2- 172.21.15.47:0/2851003974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69d00039b0 0x7f69e81543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 >> 172.21.15.47:0/2851003974 conn(0x7f69e80b1d70 msgr2=0x7f69e80afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 shutdown_connections 2024-04-20T10:29:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.291+0000 7f69ef800640 1 -- 172.21.15.47:0/2851003974 wait complete. 2024-04-20T10:29:18.311 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-20T10:29:18.448 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-20T10:29:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.525+0000 7f20d3e00640 1 Processor -- start 2024-04-20T10:29:18.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.525+0000 7f20d3e00640 1 -- start start 2024-04-20T10:29:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20d415a130 0x7f20d4073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d406f870 0x7f20d40739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d3e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20d415cc20 con 0x7f20d4071a60 2024-04-20T10:29:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d3e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f20d415d7f0 con 0x7f20d415a130 2024-04-20T10:29:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20d415cec0 con 0x7f20d406f870 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d2a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20d415a130 0x7f20d4073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d2a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20d415a130 0x7f20d4073470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58566/0 (socket says 172.21.15.47:58566) 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.527+0000 7f20d2a00640 1 -- 172.21.15.47:0/3478302426 learned_addr learned my addr 172.21.15.47:0/3478302426 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.528+0000 7f20d2a00640 1 -- 172.21.15.47:0/3478302426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d406f870 msgr2=0x7f20d40739b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.528+0000 7f20d3400640 1 --2- 172.21.15.47:0/3478302426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d406f870 0x7f20d40739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.528+0000 7f20d2a00640 1 --2- 172.21.15.47:0/3478302426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d406f870 0x7f20d40739b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.528+0000 7f20d2a00640 1 -- 172.21.15.47:0/3478302426 >> v1:172.21.15.175:6789/0 conn(0x7f20d4071a60 legacy=0x7f20d406f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.528+0000 7f20d2a00640 1 -- 172.21.15.47:0/3478302426 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d415d1a0 con 0x7f20d415a130 2024-04-20T10:29:18.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.528+0000 7f20d3400640 1 --2- 172.21.15.47:0/3478302426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d406f870 0x7f20d40739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:18.530 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.529+0000 7f20d2a00640 1 --2- 172.21.15.47:0/3478302426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d415a130 0x7f20d4073470 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=356a064a31deab20 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.529+0000 7f20d2000640 1 -- 172.21.15.47:0/3478302426 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c4038070 con 0x7f20d415a130 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.529+0000 7f20d2000640 1 -- 172.21.15.47:0/3478302426 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20c4031740 con 0x7f20d415a130 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.529+0000 7f20d2000640 1 -- 172.21.15.47:0/3478302426 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c402f050 con 0x7f20d415a130 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 -- 172.21.15.47:0/3478302426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d415a130 msgr2=0x7f20d4073470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 --2- 172.21.15.47:0/3478302426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d415a130 0x7f20d4073470 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009870 tx=0x7f20c402b7e0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 -- 172.21.15.47:0/3478302426 shutdown_connections 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 --2- 172.21.15.47:0/3478302426 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d406f870 0x7f20d40739b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 --2- 172.21.15.47:0/3478302426 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d415a130 0x7f20d4073470 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 -- 172.21.15.47:0/3478302426 >> 172.21.15.47:0/3478302426 conn(0x7f20d40cffa0 msgr2=0x7f20d40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 -- 172.21.15.47:0/3478302426 shutdown_connections 2024-04-20T10:29:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.530+0000 7f20d3e00640 1 -- 172.21.15.47:0/3478302426 wait complete. 2024-04-20T10:29:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.532+0000 7f20d3e00640 1 Processor -- start 2024-04-20T10:29:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.532+0000 7f20d3e00640 1 -- start start 2024-04-20T10:29:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20d3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d406f870 0x7f20d40c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20d3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d4071a60 0x7f20d41f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d415a130 0x7f20d41f4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20d3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f20d406ba80 con 0x7f20d4071a60 2024-04-20T10:29:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20d3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20d406b900 con 0x7f20d406f870 2024-04-20T10:29:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20d3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20d406bc00 con 0x7f20d415a130 2024-04-20T10:29:18.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d4071a60 0x7f20d41f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20cbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d4071a60 0x7f20d41f22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58580/0 (socket says 172.21.15.47:58580) 2024-04-20T10:29:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.533+0000 7f20cbe00640 1 -- 172.21.15.47:0/429089933 learned_addr learned my addr 172.21.15.47:0/429089933 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20d2a00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d406f870 0x7f20d40c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20cbe00640 1 -- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d415a130 msgr2=0x7f20d41f4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20cbe00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d415a130 0x7f20d41f4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20cbe00640 1 -- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d406f870 msgr2=0x7f20d40c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20cbe00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d406f870 0x7f20d40c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20cbe00640 1 -- 172.21.15.47:0/429089933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20d40c9830 con 0x7f20d4071a60 2024-04-20T10:29:18.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20cbe00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d4071a60 0x7f20d41f22f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c430 tx=0x7f20bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.534+0000 7f20caa00640 1 -- 172.21.15.47:0/429089933 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc011790 con 0x7f20d4071a60 2024-04-20T10:29:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.535+0000 7f20caa00640 1 -- 172.21.15.47:0/429089933 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f20bc017070 con 0x7f20d4071a60 2024-04-20T10:29:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.535+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20d40c9ad0 con 0x7f20d4071a60 2024-04-20T10:29:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.535+0000 7f20caa00640 1 -- 172.21.15.47:0/429089933 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20bc011bb0 con 0x7f20d4071a60 2024-04-20T10:29:18.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.535+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20d40ca090 con 0x7f20d4071a60 2024-04-20T10:29:18.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.536+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20d4075b30 con 0x7f20d4071a60 2024-04-20T10:29:18.538 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.537+0000 7f20caa00640 1 -- 172.21.15.47:0/429089933 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f20bc01c460 con 0x7f20d4071a60 2024-04-20T10:29:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.538+0000 7f20caa00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f20b006c8f0 0x7f20b006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.538+0000 7f20caa00640 1 -- 172.21.15.47:0/429089933 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f20bc08b8c0 con 0x7f20d4071a60 2024-04-20T10:29:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.538+0000 7f20d2a00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f20b006c8f0 0x7f20b006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:18.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.540+0000 7f20d2a00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f20b006c8f0 0x7f20b006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20c4035000 tx=0x7f20c4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:18.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.542+0000 7f20caa00640 1 -- 172.21.15.47:0/429089933 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f20bc059ea0 con 0x7f20d4071a60 2024-04-20T10:29:18.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.827+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f20d40cc5e0 con 0x7f20d4071a60 2024-04-20T10:29:18.831 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.831+0000 7f20caa00640 1 -- 172.21.15.47:0/429089933 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f20bc0597a0 con 0x7f20d4071a60 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f20b006c8f0 msgr2=0x7f20b006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f20b006c8f0 0x7f20b006edb0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f20c4035000 tx=0x7f20c4008030 comp rx=0 tx=0).stop 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d4071a60 msgr2=0x7f20d41f22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d4071a60 0x7f20d41f22f0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f20bc00c430 tx=0x7f20bc00c8f0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 shutdown_connections 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20d415a130 0x7f20d41f4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f20b006c8f0 0x7f20b006edb0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20d4071a60 0x7f20d41f22f0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 --2- 172.21.15.47:0/429089933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20d406f870 0x7f20d40c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:18.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.834+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 >> 172.21.15.47:0/429089933 conn(0x7f20d40cffa0 msgr2=0x7f20d40acf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:18.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.835+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 shutdown_connections 2024-04-20T10:29:18.836 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:18.835+0000 7f20d3e00640 1 -- 172.21.15.47:0/429089933 wait complete. 2024-04-20T10:29:18.855 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-20T10:29:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.075+0000 7f380be00640 1 Processor -- start 2024-04-20T10:29:19.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.075+0000 7f380be00640 1 -- start start 2024-04-20T10:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.077+0000 7f380be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c1629b0 0x7f380c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.077+0000 7f380be00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f380c0ca740 0x7f380c0acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.077+0000 7f380be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f380c06b9a0 con 0x7f380c0cfb30 2024-04-20T10:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.077+0000 7f380be00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f380c06c540 con 0x7f380c0ca740 2024-04-20T10:29:19.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.077+0000 7f380be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f380c06bc10 con 0x7f380c1629b0 2024-04-20T10:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.078+0000 7f380a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f380c0ca740 0x7f380c0acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.078+0000 7f380aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c1629b0 0x7f380c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.078+0000 7f380aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c1629b0 0x7f380c0cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:43706/0 (socket says 172.21.15.47:43706) 2024-04-20T10:29:19.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.078+0000 7f380aa00640 1 -- 172.21.15.47:0/1871087621 learned_addr learned my addr 172.21.15.47:0/1871087621 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.079+0000 7f380a000640 1 -- 172.21.15.47:0/1871087621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c1629b0 msgr2=0x7f380c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.079+0000 7f380a000640 1 --2- 172.21.15.47:0/1871087621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c1629b0 0x7f380c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.079+0000 7f380a000640 1 -- 172.21.15.47:0/1871087621 >> v1:172.21.15.175:6789/0 conn(0x7f380c0cfb30 legacy=0x7f380c166530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.079+0000 7f380a000640 1 -- 172.21.15.47:0/1871087621 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c06bef0 con 0x7f380c0ca740 2024-04-20T10:29:19.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.080+0000 7f380a000640 1 --2- 172.21.15.47:0/1871087621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c0ca740 0x7f380c0acf50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f37f80097a0 tx=0x7f37f802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be7f05e290bdad36 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.081+0000 7f3809600640 1 -- 172.21.15.47:0/1871087621 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8035020 con 0x7f380c0ca740 2024-04-20T10:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.081+0000 7f3809600640 1 -- 172.21.15.47:0/1871087621 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f8031810 con 0x7f380c0ca740 2024-04-20T10:29:19.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.081+0000 7f3809600640 1 -- 172.21.15.47:0/1871087621 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8031be0 con 0x7f380c0ca740 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 -- 172.21.15.47:0/1871087621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c0ca740 msgr2=0x7f380c0acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 --2- 172.21.15.47:0/1871087621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c0ca740 0x7f380c0acf50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f37f80097a0 tx=0x7f37f802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 -- 172.21.15.47:0/1871087621 shutdown_connections 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 --2- 172.21.15.47:0/1871087621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c0ca740 0x7f380c0acf50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 --2- 172.21.15.47:0/1871087621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c1629b0 0x7f380c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 -- 172.21.15.47:0/1871087621 >> 172.21.15.47:0/1871087621 conn(0x7f380c0cffa0 msgr2=0x7f380c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 -- 172.21.15.47:0/1871087621 shutdown_connections 2024-04-20T10:29:19.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.082+0000 7f380be00640 1 -- 172.21.15.47:0/1871087621 wait complete. 2024-04-20T10:29:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.084+0000 7f380be00640 1 Processor -- start 2024-04-20T10:29:19.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.084+0000 7f380be00640 1 -- start start 2024-04-20T10:29:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c0ca740 0x7f380c09b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f380c0cfb30 0x7f380c1f84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c1629b0 0x7f380c1f8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f380c0ceeb0 con 0x7f380c0cfb30 2024-04-20T10:29:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f380c162d90 con 0x7f380c1629b0 2024-04-20T10:29:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f380c0cf030 con 0x7f380c0ca740 2024-04-20T10:29:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f380c0cfb30 0x7f380c1f84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f380c0cfb30 0x7f380c1f84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58588/0 (socket says 172.21.15.47:58588) 2024-04-20T10:29:19.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.085+0000 7f380a000640 1 -- 172.21.15.47:0/1728229009 learned_addr learned my addr 172.21.15.47:0/1728229009 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.086+0000 7f380aa00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c0ca740 0x7f380c09b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.086+0000 7f380b400640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c1629b0 0x7f380c1f8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.086+0000 7f380a000640 1 -- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c0ca740 msgr2=0x7f380c09b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.086+0000 7f380a000640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c0ca740 0x7f380c09b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.086+0000 7f380a000640 1 -- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c1629b0 msgr2=0x7f380c1f8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.086+0000 7f380a000640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c1629b0 0x7f380c1f8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.086+0000 7f380a000640 1 -- 172.21.15.47:0/1728229009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f380c09d580 con 0x7f380c0cfb30 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.087+0000 7f380a000640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f380c0cfb30 0x7f380c1f84f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f37f802bec0 tx=0x7f37f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.087+0000 7f3802a00640 1 -- 172.21.15.47:0/1728229009 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8041070 con 0x7f380c0cfb30 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.087+0000 7f3802a00640 1 -- 172.21.15.47:0/1728229009 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37f803ac80 con 0x7f380c0cfb30 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.087+0000 7f3802a00640 1 -- 172.21.15.47:0/1728229009 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37f8035040 con 0x7f380c0cfb30 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.087+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f380c09c010 con 0x7f380c0cfb30 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.087+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f380c0b2f70 con 0x7f380c0cfb30 2024-04-20T10:29:19.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.088+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f380c0b2d60 con 0x7f380c0cfb30 2024-04-20T10:29:19.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.089+0000 7f3802a00640 1 -- 172.21.15.47:0/1728229009 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f37f8038020 con 0x7f380c0cfb30 2024-04-20T10:29:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.090+0000 7f3802a00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ec06c8f0 0x7f37ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.090+0000 7f3802a00640 1 -- 172.21.15.47:0/1728229009 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f37f80ad540 con 0x7f380c0cfb30 2024-04-20T10:29:19.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.091+0000 7f380aa00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ec06c8f0 0x7f37ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.092+0000 7f380aa00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ec06c8f0 0x7f37ec06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f380c1623c0 tx=0x7f37fc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:19.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.094+0000 7f3802a00640 1 -- 172.21.15.47:0/1728229009 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f37f807bb60 con 0x7f380c0cfb30 2024-04-20T10:29:19.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.374+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f380c075b30 con 0x7f380c0cfb30 2024-04-20T10:29:19.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.378+0000 7f3802a00640 1 -- 172.21.15.47:0/1728229009 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f37f807b460 con 0x7f380c0cfb30 2024-04-20T10:29:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ec06c8f0 msgr2=0x7f37ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ec06c8f0 0x7f37ec06edb0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f380c1623c0 tx=0x7f37fc008040 comp rx=0 tx=0).stop 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f380c0cfb30 msgr2=0x7f380c1f84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f380c0cfb30 0x7f380c1f84f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f37f802bec0 tx=0x7f37f80046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 shutdown_connections 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f380c1629b0 0x7f380c1f8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f37ec06c8f0 0x7f37ec06edb0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f380c0cfb30 0x7f380c1f84f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 --2- 172.21.15.47:0/1728229009 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f380c0ca740 0x7f380c09b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 >> 172.21.15.47:0/1728229009 conn(0x7f380c0cffa0 msgr2=0x7f380c0b0030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 shutdown_connections 2024-04-20T10:29:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.381+0000 7f380be00640 1 -- 172.21.15.47:0/1728229009 wait complete. 2024-04-20T10:29:19.400 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-20T10:29:19.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.609+0000 7f6447200640 1 Processor -- start 2024-04-20T10:29:19.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.609+0000 7f6447200640 1 -- start start 2024-04-20T10:29:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.611+0000 7f6447200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f64480ceeb0 0x7f644816d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.612+0000 7f6447200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644806b9a0 con 0x7f64481629b0 2024-04-20T10:29:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.612+0000 7f6447200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f644806bc10 con 0x7f64480ca740 2024-04-20T10:29:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.612+0000 7f6447200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f644806c540 con 0x7f64480ceeb0 2024-04-20T10:29:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.612+0000 7f6445e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f64481629b0 0x7f64480cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43416/0 (socket says 172.21.15.47:43416) 2024-04-20T10:29:19.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.612+0000 7f6445e00640 1 -- 172.21.15.47:0/2583230255 learned_addr learned my addr 172.21.15.47:0/2583230255 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.612+0000 7f6446800640 1 --2- 172.21.15.47:0/2583230255 >> v2:172.21.15.175:3300/0 conn(0x7f64480ceeb0 0x7f644816d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.612+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2522877847 0 0) 0x7f644806b9a0 con 0x7f64481629b0 2024-04-20T10:29:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f642c003520 con 0x7f64481629b0 2024-04-20T10:29:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1616747237 0 0) 0x7f642c003520 con 0x7f64481629b0 2024-04-20T10:29:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 >> v2:172.21.15.175:3300/0 conn(0x7f64480ceeb0 msgr2=0x7f644816d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:19.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 --2- 172.21.15.47:0/2583230255 >> v2:172.21.15.175:3300/0 conn(0x7f64480ceeb0 0x7f644816d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 >> v1:172.21.15.175:6789/0 conn(0x7f64480ca740 legacy=0x7f64480acf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644806bef0 con 0x7f64481629b0 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f6438001670 con 0x7f64481629b0 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 >> v1:172.21.15.47:6789/0 conn(0x7f64481629b0 legacy=0x7f64480cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642c0039b0 0x7f642c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c004800 0x7f642c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642c0051c0 0x7f642c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f644806c540 con 0x7f642c0039b0 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f642c00c060 con 0x7f642c0051c0 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f642c00c260 con 0x7f642c004800 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.613+0000 7f6445e00640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642c0039b0 0x7f642c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.614+0000 7f6445400640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c004800 0x7f642c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.614+0000 7f6445e00640 1 -- 172.21.15.47:0/2583230255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c004800 msgr2=0x7f642c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:19.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.614+0000 7f6445e00640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c004800 0x7f642c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.614+0000 7f6445e00640 1 -- 172.21.15.47:0/2583230255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642c0051c0 msgr2=0x7f642c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.614+0000 7f6445e00640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f642c0051c0 0x7f642c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.614+0000 7f6445e00640 1 -- 172.21.15.47:0/2583230255 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f642c003520 con 0x7f642c0039b0 2024-04-20T10:29:19.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.614+0000 7f6446800640 1 -- 172.21.15.47:0/2583230255 reap_dead start 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.615+0000 7f6445e00640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642c0039b0 0x7f642c003dd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f643800c240 tx=0x7f642c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9e9dd2249dce6a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.615+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6438007d10 con 0x7f642c0039b0 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.615+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6438030070 con 0x7f642c0039b0 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.615+0000 7f6444a00640 1 -- 172.21.15.47:0/2583230255 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f643802cda0 con 0x7f642c0039b0 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.616+0000 7f6447200640 1 -- 172.21.15.47:0/2583230255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642c0039b0 msgr2=0x7f642c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.616+0000 7f6447200640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642c0039b0 0x7f642c003dd0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f643800c240 tx=0x7f642c005830 comp rx=0 tx=0).stop 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.616+0000 7f6447200640 1 -- 172.21.15.47:0/2583230255 shutdown_connections 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.616+0000 7f6447200640 1 --2- 172.21.15.47:0/2583230255 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f642c0039b0 0x7f642c003dd0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.616+0000 7f6447200640 1 -- 172.21.15.47:0/2583230255 >> 172.21.15.47:0/2583230255 conn(0x7f64480cffa0 msgr2=0x7f64480d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.616+0000 7f6447200640 1 -- 172.21.15.47:0/2583230255 shutdown_connections 2024-04-20T10:29:19.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.616+0000 7f6447200640 1 -- 172.21.15.47:0/2583230255 wait complete. 2024-04-20T10:29:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.618+0000 7f6447200640 1 Processor -- start 2024-04-20T10:29:19.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.618+0000 7f6447200640 1 -- start start 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6447200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c0039b0 0x7f644815b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6447200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64480b5100 0x7f644809cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6447200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644809d420 0x7f64481fbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6447200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f64480cf5f0 con 0x7f64480b5100 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6447200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6448162d90 con 0x7f644809d420 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6447200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f64480cf770 con 0x7f642c0039b0 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6445400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64480b5100 0x7f644809cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6445400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64480b5100 0x7f644809cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58606/0 (socket says 172.21.15.47:58606) 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.619+0000 7f6445400640 1 -- 172.21.15.47:0/1287633151 learned_addr learned my addr 172.21.15.47:0/1287633151 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:19.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6445e00640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c0039b0 0x7f644815b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6446800640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644809d420 0x7f64481fbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6445400640 1 -- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c0039b0 msgr2=0x7f644815b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6445400640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c0039b0 0x7f644815b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6445400640 1 -- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644809d420 msgr2=0x7f64481fbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6445400640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644809d420 0x7f64481fbb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6445400640 1 -- 172.21.15.47:0/1287633151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64480b2d60 con 0x7f64480b5100 2024-04-20T10:29:19.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f6445400640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64480b5100 0x7f644809cee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f643400e480 tx=0x7f643400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f643e000640 1 -- 172.21.15.47:0/1287633151 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64340052e0 con 0x7f64480b5100 2024-04-20T10:29:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f643e000640 1 -- 172.21.15.47:0/1287633151 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6434018070 con 0x7f64480b5100 2024-04-20T10:29:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.620+0000 7f643e000640 1 -- 172.21.15.47:0/1287633151 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6434003810 con 0x7f64480b5100 2024-04-20T10:29:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.621+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64480b3040 con 0x7f64480b5100 2024-04-20T10:29:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.621+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64480b3600 con 0x7f64480b5100 2024-04-20T10:29:19.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.622+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6448075b30 con 0x7f64480b5100 2024-04-20T10:29:19.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.623+0000 7f643e000640 1 -- 172.21.15.47:0/1287633151 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f64340039b0 con 0x7f64480b5100 2024-04-20T10:29:19.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.624+0000 7f643e000640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f642806c920 0x7f642806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:19.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.624+0000 7f643e000640 1 -- 172.21.15.47:0/1287633151 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f643408b6d0 con 0x7f64480b5100 2024-04-20T10:29:19.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.624+0000 7f6445e00640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f642806c920 0x7f642806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:19.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.626+0000 7f6445e00640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f642806c920 0x7f642806ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6438034000 tx=0x7f6438009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:19.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.628+0000 7f643e000640 1 -- 172.21.15.47:0/1287633151 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6434059cb0 con 0x7f64480b5100 2024-04-20T10:29:19.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.911+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f644809d800 con 0x7f64480b5100 2024-04-20T10:29:19.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.916+0000 7f643e000640 1 -- 172.21.15.47:0/1287633151 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7f64340595b0 con 0x7f64480b5100 2024-04-20T10:29:19.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f642806c920 msgr2=0x7f642806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f642806c920 0x7f642806ede0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f6438034000 tx=0x7f6438009030 comp rx=0 tx=0).stop 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64480b5100 msgr2=0x7f644809cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64480b5100 0x7f644809cee0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f643400e480 tx=0x7f643400e940 comp rx=0 tx=0).stop 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 shutdown_connections 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f644809d420 0x7f64481fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f642806c920 0x7f642806ede0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f64480b5100 0x7f644809cee0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 --2- 172.21.15.47:0/1287633151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f642c0039b0 0x7f644815b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.919+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 >> 172.21.15.47:0/1287633151 conn(0x7f64480cffa0 msgr2=0x7f64480cbe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.920+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 shutdown_connections 2024-04-20T10:29:19.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:19.920+0000 7f6447200640 1 -- 172.21.15.47:0/1287633151 wait complete. 2024-04-20T10:29:19.941 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-20T10:29:20.162 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.161+0000 7f6906c00640 1 Processor -- start 2024-04-20T10:29:20.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.161+0000 7f6906c00640 1 -- start start 2024-04-20T10:29:20.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6906c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690015b3b0 0x7f690006eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6906c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f690006dcb0 0x7f6900071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6906c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6900072090 0x7f69000d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6906c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f69000ade40 con 0x7f6900072090 2024-04-20T10:29:20.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6906c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f69000ae0e0 con 0x7f690006dcb0 2024-04-20T10:29:20.164 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6906c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f690015b790 con 0x7f690015b3b0 2024-04-20T10:29:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6904c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6900072090 0x7f69000d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6904c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6900072090 0x7f69000d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58618/0 (socket says 172.21.15.47:58618) 2024-04-20T10:29:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.163+0000 7f6904c00640 1 -- 172.21.15.47:0/26506682 learned_addr learned my addr 172.21.15.47:0/26506682 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f68f7400640 1 --2- 172.21.15.47:0/26506682 >> v2:172.21.15.175:3300/0 conn(0x7f690006dcb0 0x7f6900071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f68ffe00640 1 --2- 172.21.15.47:0/26506682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690015b3b0 0x7f690006eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f6904c00640 1 -- 172.21.15.47:0/26506682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690015b3b0 msgr2=0x7f690006eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f6904c00640 1 --2- 172.21.15.47:0/26506682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690015b3b0 0x7f690006eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f6904c00640 1 -- 172.21.15.47:0/26506682 >> v2:172.21.15.175:3300/0 conn(0x7f690006dcb0 msgr2=0x7f6900071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f6904c00640 1 --2- 172.21.15.47:0/26506682 >> v2:172.21.15.175:3300/0 conn(0x7f690006dcb0 0x7f6900071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f6904c00640 1 -- 172.21.15.47:0/26506682 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69000ae380 con 0x7f6900072090 2024-04-20T10:29:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f68f7400640 1 --2- 172.21.15.47:0/26506682 >> v2:172.21.15.175:3300/0 conn(0x7f690006dcb0 0x7f6900071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.164+0000 7f68ffe00640 1 --2- 172.21.15.47:0/26506682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690015b3b0 0x7f690006eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:20.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.165+0000 7f6904c00640 1 --2- 172.21.15.47:0/26506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6900072090 0x7f69000d0360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f690006d920 tx=0x7f68f802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bca4289c885a11d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.165+0000 7f68ff400640 1 -- 172.21.15.47:0/26506682 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8035020 con 0x7f6900072090 2024-04-20T10:29:20.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f68ff400640 1 -- 172.21.15.47:0/26506682 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f802d040 con 0x7f6900072090 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f68ff400640 1 -- 172.21.15.47:0/26506682 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8031a80 con 0x7f6900072090 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 -- 172.21.15.47:0/26506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6900072090 msgr2=0x7f69000d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 --2- 172.21.15.47:0/26506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6900072090 0x7f69000d0360 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f690006d920 tx=0x7f68f802b900 comp rx=0 tx=0).stop 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 -- 172.21.15.47:0/26506682 shutdown_connections 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 --2- 172.21.15.47:0/26506682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6900072090 0x7f69000d0360 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 --2- 172.21.15.47:0/26506682 >> v2:172.21.15.175:3300/0 conn(0x7f690006dcb0 0x7f6900071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 --2- 172.21.15.47:0/26506682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690015b3b0 0x7f690006eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 -- 172.21.15.47:0/26506682 >> 172.21.15.47:0/26506682 conn(0x7f69000b3350 msgr2=0x7f69000b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 -- 172.21.15.47:0/26506682 shutdown_connections 2024-04-20T10:29:20.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.166+0000 7f6906c00640 1 -- 172.21.15.47:0/26506682 wait complete. 2024-04-20T10:29:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.169+0000 7f6906c00640 1 Processor -- start 2024-04-20T10:29:20.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.169+0000 7f6906c00640 1 -- start start 2024-04-20T10:29:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.170+0000 7f6906c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690006dcb0 0x7f690014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.170+0000 7f6906c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690015b3b0 0x7f69000b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.170+0000 7f6906c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69001f4d40 0x7f6900154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.170+0000 7f6906c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f69000aec20 con 0x7f69001f4d40 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.170+0000 7f6906c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69000ae940 con 0x7f690015b3b0 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.170+0000 7f6906c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f690006efe0 con 0x7f690006dcb0 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69001f4d40 0x7f6900154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69001f4d40 0x7f6900154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58622/0 (socket says 172.21.15.47:58622) 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 -- 172.21.15.47:0/413354941 learned_addr learned my addr 172.21.15.47:0/413354941 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f68f7400640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690015b3b0 0x7f69000b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f68ffe00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690006dcb0 0x7f690014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 -- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690006dcb0 msgr2=0x7f690014e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690006dcb0 0x7f690014e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 -- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690015b3b0 msgr2=0x7f69000b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690015b3b0 0x7f69000b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f6904c00640 1 -- 172.21.15.47:0/413354941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69000af890 con 0x7f69001f4d40 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.171+0000 7f68ffe00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690006dcb0 0x7f690014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.172+0000 7f6904c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69001f4d40 0x7f6900154640 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f68f802b940 tx=0x7f68f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.172+0000 7f68fcc00640 1 -- 172.21.15.47:0/413354941 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8040070 con 0x7f69001f4d40 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.172+0000 7f68fcc00640 1 -- 172.21.15.47:0/413354941 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f8039c10 con 0x7f69001f4d40 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.172+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69000afb70 con 0x7f69001f4d40 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.172+0000 7f68fcc00640 1 -- 172.21.15.47:0/413354941 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8035040 con 0x7f69001f4d40 2024-04-20T10:29:20.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.172+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69000b0130 con 0x7f69001f4d40 2024-04-20T10:29:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.173+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6900068980 con 0x7f69001f4d40 2024-04-20T10:29:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.174+0000 7f68fcc00640 1 -- 172.21.15.47:0/413354941 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68f8038020 con 0x7f69001f4d40 2024-04-20T10:29:20.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.176+0000 7f68fcc00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68dc06c920 0x7f68dc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.176+0000 7f68fcc00640 1 -- 172.21.15.47:0/413354941 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f68f80ad690 con 0x7f69001f4d40 2024-04-20T10:29:20.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.177+0000 7f68ffe00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68dc06c920 0x7f68dc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.178+0000 7f68ffe00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68dc06c920 0x7f68dc06ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f69001f1c60 tx=0x7f68f0007550 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:20.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.181+0000 7f68fcc00640 1 -- 172.21.15.47:0/413354941 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f68f807bc00 con 0x7f69001f4d40 2024-04-20T10:29:20.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.456+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7f69000b1520 con 0x7f69001f4d40 2024-04-20T10:29:20.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.459+0000 7f68fcc00640 1 -- 172.21.15.47:0/413354941 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7f68f807b500 con 0x7f69001f4d40 2024-04-20T10:29:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68dc06c920 msgr2=0x7f68dc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68dc06c920 0x7f68dc06ede0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f69001f1c60 tx=0x7f68f0007550 comp rx=0 tx=0).stop 2024-04-20T10:29:20.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69001f4d40 msgr2=0x7f6900154640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69001f4d40 0x7f6900154640 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f68f802b940 tx=0x7f68f80046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 shutdown_connections 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68dc06c920 0x7f68dc06ede0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f69001f4d40 0x7f6900154640 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690015b3b0 0x7f69000b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 --2- 172.21.15.47:0/413354941 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f690006dcb0 0x7f690014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 >> 172.21.15.47:0/413354941 conn(0x7f69000b3350 msgr2=0x7f69000b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 shutdown_connections 2024-04-20T10:29:20.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.462+0000 7f6906c00640 1 -- 172.21.15.47:0/413354941 wait complete. 2024-04-20T10:29:20.481 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-20T10:29:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.669+0000 7f13a8800640 1 Processor -- start 2024-04-20T10:29:20.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.670+0000 7f13a8800640 1 -- start start 2024-04-20T10:29:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.671+0000 7f13a8800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13a0162700 0x7f13a0162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.671+0000 7f13a8800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a006f2b0 con 0x7f13a00b6c20 2024-04-20T10:29:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.671+0000 7f13a8800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13a00b6160 con 0x7f13a015ead0 2024-04-20T10:29:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.671+0000 7f13a8800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f13a00b6900 con 0x7f13a0162700 2024-04-20T10:29:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a5400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f13a00b6c20 0x7f13a015e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43424/0 (socket says 172.21.15.47:43424) 2024-04-20T10:29:20.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a5400640 1 -- 172.21.15.47:0/4120435832 learned_addr learned my addr 172.21.15.47:0/4120435832 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a5e00640 1 --2- 172.21.15.47:0/4120435832 >> v2:172.21.15.175:3300/0 conn(0x7f13a0162700 0x7f13a0162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 622027613 0 0) 0x7f13a006f2b0 con 0x7f13a00b6c20 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1388003520 con 0x7f13a00b6c20 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2937501351 0 0) 0x7f1388003520 con 0x7f13a00b6c20 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 >> v2:172.21.15.175:3300/0 conn(0x7f13a0162700 msgr2=0x7f13a0162ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.672+0000 7f13a4a00640 1 --2- 172.21.15.47:0/4120435832 >> v2:172.21.15.175:3300/0 conn(0x7f13a0162700 0x7f13a0162ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 >> v1:172.21.15.175:6789/0 conn(0x7f13a015ead0 legacy=0x7f13a00cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a0163210 con 0x7f13a00b6c20 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f1390001670 con 0x7f13a00b6c20 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 >> v1:172.21.15.47:6789/0 conn(0x7f13a00b6c20 legacy=0x7f13a015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1388003a00 0x7f1388003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388004800 0x7f1388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13880052e0 0x7f138800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f138800c1b0 con 0x7f1388003a00 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f138800c360 con 0x7f1388004800 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f138800c560 con 0x7f13880052e0 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a5e00640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1388003a00 0x7f1388003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a5e00640 1 -- 172.21.15.47:0/4120435832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13880052e0 msgr2=0x7f138800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a5e00640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13880052e0 0x7f138800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a5e00640 1 -- 172.21.15.47:0/4120435832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388004800 msgr2=0x7f1388004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a5400640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388004800 0x7f1388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a5e00640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388004800 0x7f1388004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a5e00640 1 -- 172.21.15.47:0/4120435832 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1388003520 con 0x7f1388003a00 2024-04-20T10:29:20.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.673+0000 7f13a6800640 1 -- 172.21.15.47:0/4120435832 reap_dead start 2024-04-20T10:29:20.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.674+0000 7f13a5e00640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1388003a00 0x7f1388003e20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1394009ac0 tx=0x7f1388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a447e5ea4830b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:20.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394039070 con 0x7f1388003a00 2024-04-20T10:29:20.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139402c710 con 0x7f1388003a00 2024-04-20T10:29:20.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a4a00640 1 -- 172.21.15.47:0/4120435832 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1394009690 con 0x7f1388003a00 2024-04-20T10:29:20.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a8800640 1 -- 172.21.15.47:0/4120435832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1388003a00 msgr2=0x7f1388003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a8800640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1388003a00 0x7f1388003e20 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f1394009ac0 tx=0x7f1388005830 comp rx=0 tx=0).stop 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a8800640 1 -- 172.21.15.47:0/4120435832 shutdown_connections 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a8800640 1 --2- 172.21.15.47:0/4120435832 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1388003a00 0x7f1388003e20 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.675+0000 7f13a8800640 1 -- 172.21.15.47:0/4120435832 >> 172.21.15.47:0/4120435832 conn(0x7f13a009d910 msgr2=0x7f13a00b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.676+0000 7f13a8800640 1 -- 172.21.15.47:0/4120435832 shutdown_connections 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.676+0000 7f13a8800640 1 -- 172.21.15.47:0/4120435832 wait complete. 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.677+0000 7f13a8800640 1 Processor -- start 2024-04-20T10:29:20.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.677+0000 7f13a8800640 1 -- start start 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.678+0000 7f13a8800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388003a00 0x7f13a0160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.678+0000 7f13a8800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00b3ea0 0x7f13a00b4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.678+0000 7f13a8800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13a01f4490 0x7f13a01f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.678+0000 7f13a8800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00b38a0 con 0x7f13a01f4490 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.678+0000 7f13a8800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13a00b3720 con 0x7f1388003a00 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.678+0000 7f13a8800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13a00b6620 con 0x7f13a00b3ea0 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13a01f4490 0x7f13a01f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13a01f4490 0x7f13a01f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58638/0 (socket says 172.21.15.47:58638) 2024-04-20T10:29:20.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 -- 172.21.15.47:0/900616748 learned_addr learned my addr 172.21.15.47:0/900616748 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 -- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00b3ea0 msgr2=0x7f13a00b4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a5e00640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388003a00 0x7f13a0160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00b3ea0 0x7f13a00b4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 -- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388003a00 msgr2=0x7f13a0160bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388003a00 0x7f13a0160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 -- 172.21.15.47:0/900616748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a01f53f0 con 0x7f13a01f4490 2024-04-20T10:29:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.679+0000 7f13a6800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13a01f4490 0x7f13a01f48d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f13a0074970 tx=0x7f139c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.680+0000 7f139a000640 1 -- 172.21.15.47:0/900616748 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c017070 con 0x7f13a01f4490 2024-04-20T10:29:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.680+0000 7f139a000640 1 -- 172.21.15.47:0/900616748 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f139c005d60 con 0x7f13a01f4490 2024-04-20T10:29:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.680+0000 7f139a000640 1 -- 172.21.15.47:0/900616748 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f139c00a850 con 0x7f13a01f4490 2024-04-20T10:29:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.680+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a01f5110 con 0x7f13a01f4490 2024-04-20T10:29:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.680+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a01603d0 con 0x7f13a01f4490 2024-04-20T10:29:20.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.681+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a0068980 con 0x7f13a01f4490 2024-04-20T10:29:20.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.682+0000 7f139a000640 1 -- 172.21.15.47:0/900616748 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f139c01a070 con 0x7f13a01f4490 2024-04-20T10:29:20.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.683+0000 7f139a000640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f138406c8f0 0x7f138406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.684+0000 7f139a000640 1 -- 172.21.15.47:0/900616748 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f139c08ab20 con 0x7f13a01f4490 2024-04-20T10:29:20.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.684+0000 7f13a5e00640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f138406c8f0 0x7f138406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:20.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.686+0000 7f13a5e00640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f138406c8f0 0x7f138406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1394036000 tx=0x7f1394030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:20.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.688+0000 7f139a000640 1 -- 172.21.15.47:0/900616748 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f139c059100 con 0x7f13a01f4490 2024-04-20T10:29:20.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.972+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7f13a00d51b0 con 0x7f13a01f4490 2024-04-20T10:29:20.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.976+0000 7f139a000640 1 -- 172.21.15.47:0/900616748 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7f139c058a00 con 0x7f13a01f4490 2024-04-20T10:29:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f138406c8f0 msgr2=0x7f138406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f138406c8f0 0x7f138406edb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1394036000 tx=0x7f1394030000 comp rx=0 tx=0).stop 2024-04-20T10:29:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13a01f4490 msgr2=0x7f13a01f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13a01f4490 0x7f13a01f48d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f13a0074970 tx=0x7f139c00cb90 comp rx=0 tx=0).stop 2024-04-20T10:29:20.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 shutdown_connections 2024-04-20T10:29:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f138406c8f0 0x7f138406edb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13a01f4490 0x7f13a01f48d0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13a00b3ea0 0x7f13a00b4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 --2- 172.21.15.47:0/900616748 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1388003a00 0x7f13a0160bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 >> 172.21.15.47:0/900616748 conn(0x7f13a009d910 msgr2=0x7f13a00b7160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 shutdown_connections 2024-04-20T10:29:20.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:20.979+0000 7f13a8800640 1 -- 172.21.15.47:0/900616748 wait complete. 2024-04-20T10:29:20.998 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-20T10:29:21.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.194+0000 7f4068a00640 1 Processor -- start 2024-04-20T10:29:21.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.194+0000 7f4068a00640 1 -- start start 2024-04-20T10:29:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.196+0000 7f4068a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 0x7f40640b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.196+0000 7f4068a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40640d3590 con 0x7f4064162210 2024-04-20T10:29:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.196+0000 7f4068a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40640d3870 con 0x7f4064162670 2024-04-20T10:29:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.196+0000 7f4068a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f406415eb50 con 0x7f406415e770 2024-04-20T10:29:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.196+0000 7f405be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f4064162210 0x7f40640b6400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43440/0 (socket says 172.21.15.47:43440) 2024-04-20T10:29:21.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.196+0000 7f405be00640 1 -- 172.21.15.47:0/2428996225 learned_addr learned my addr 172.21.15.47:0/2428996225 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4063400640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 0x7f40640b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542205348 0 0) 0x7f40640d3590 con 0x7f4064162210 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4048003520 con 0x7f4064162210 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3571740911 0 0) 0x7f4048003520 con 0x7f4064162210 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 msgr2=0x7f40640b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 0x7f40640b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 >> v1:172.21.15.175:6789/0 conn(0x7f4064162670 legacy=0x7f40640d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40640d3b50 con 0x7f4064162210 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f404c001670 con 0x7f4064162210 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 >> v1:172.21.15.47:6789/0 conn(0x7f4064162210 legacy=0x7f40640b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f4048003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4048004800 0x7f4048004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.197+0000 7f4062a00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 0x7f4048005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f406415eb50 con 0x7f40480039e0 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f404800bb50 con 0x7f4048004800 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f404800bd50 con 0x7f406415e770 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063400640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f4048003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f405be00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4048004800 0x7f4048004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063e00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 0x7f4048005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063400640 1 -- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 msgr2=0x7f4048005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063400640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 0x7f4048005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063400640 1 -- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4048004800 msgr2=0x7f4048004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063400640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4048004800 0x7f4048004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063400640 1 -- 172.21.15.47:0/2428996225 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4048003520 con 0x7f40480039e0 2024-04-20T10:29:21.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.198+0000 7f4063e00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f406415e770 0x7f4048005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.199+0000 7f4063400640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f4048003e00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4054009ac0 tx=0x7f4048005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88e58ca1097c27a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054039070 con 0x7f40480039e0 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405402c6b0 con 0x7f40480039e0 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4062a00640 1 -- 172.21.15.47:0/2428996225 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054009690 con 0x7f40480039e0 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4068a00640 1 -- 172.21.15.47:0/2428996225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 msgr2=0x7f4048003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4068a00640 1 --2- 172.21.15.47:0/2428996225 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f4048003e00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4054009ac0 tx=0x7f4048005830 comp rx=0 tx=0).stop 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4063e00640 1 -- 172.21.15.47:0/2428996225 reap_dead start 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4068a00640 1 -- 172.21.15.47:0/2428996225 shutdown_connections 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4068a00640 1 -- 172.21.15.47:0/2428996225 >> 172.21.15.47:0/2428996225 conn(0x7f406409d6d0 msgr2=0x7f40640b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4068a00640 1 -- 172.21.15.47:0/2428996225 shutdown_connections 2024-04-20T10:29:21.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.200+0000 7f4068a00640 1 -- 172.21.15.47:0/2428996225 wait complete. 2024-04-20T10:29:21.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.202+0000 7f4068a00640 1 Processor -- start 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.202+0000 7f4068a00640 1 -- start start 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4068a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f406406be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4068a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40640cb520 0x7f4064161070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4068a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40640cdfa0 0x7f40641f7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4068a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f40640b6b10 con 0x7f40480039e0 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4068a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40640d3e30 con 0x7f40640cb520 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4068a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40640b6c90 con 0x7f40640cdfa0 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4063400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f406406be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.203+0000 7f4063400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f406406be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58660/0 (socket says 172.21.15.47:58660) 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4063400640 1 -- 172.21.15.47:0/2007528837 learned_addr learned my addr 172.21.15.47:0/2007528837 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4063400640 1 -- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40640cdfa0 msgr2=0x7f40641f7a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4063400640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40640cdfa0 0x7f40641f7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4063400640 1 -- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40640cb520 msgr2=0x7f4064161070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4063400640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40640cb520 0x7f4064161070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4063400640 1 -- 172.21.15.47:0/2007528837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406406e4b0 con 0x7f40480039e0 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4063400640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f406406be10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4054009be0 tx=0x7f4054004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f405b400640 1 -- 172.21.15.47:0/2007528837 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054042020 con 0x7f40480039e0 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f405b400640 1 -- 172.21.15.47:0/2007528837 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f405403a040 con 0x7f40480039e0 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406406e750 con 0x7f40480039e0 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.204+0000 7f405b400640 1 -- 172.21.15.47:0/2007528837 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4054039070 con 0x7f40480039e0 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.205+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40640cda80 con 0x7f40480039e0 2024-04-20T10:29:21.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.206+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4064075b30 con 0x7f40480039e0 2024-04-20T10:29:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.207+0000 7f405b400640 1 -- 172.21.15.47:0/2007528837 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4054045020 con 0x7f40480039e0 2024-04-20T10:29:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.208+0000 7f405b400640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f404006c8f0 0x7f404006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.208+0000 7f405b400640 1 -- 172.21.15.47:0/2007528837 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f40540ace70 con 0x7f40480039e0 2024-04-20T10:29:21.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.209+0000 7f405be00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f404006c8f0 0x7f404006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.210+0000 7f405be00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f404006c8f0 0x7f404006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f404c0017e0 tx=0x7f404c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:21.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.213+0000 7f405b400640 1 -- 172.21.15.47:0/2007528837 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f405404c070 con 0x7f40480039e0 2024-04-20T10:29:21.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.497+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f40640cb170 con 0x7f40480039e0 2024-04-20T10:29:21.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.502+0000 7f405b400640 1 -- 172.21.15.47:0/2007528837 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f405407b3c0 con 0x7f40480039e0 2024-04-20T10:29:21.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f404006c8f0 msgr2=0x7f404006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f404006c8f0 0x7f404006edb0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f404c0017e0 tx=0x7f404c009040 comp rx=0 tx=0).stop 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 msgr2=0x7f406406be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f406406be10 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4054009be0 tx=0x7f4054004730 comp rx=0 tx=0).stop 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 shutdown_connections 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40640cdfa0 0x7f40641f7a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40640cb520 0x7f4064161070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f404006c8f0 0x7f404006edb0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 --2- 172.21.15.47:0/2007528837 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f40480039e0 0x7f406406be10 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 >> 172.21.15.47:0/2007528837 conn(0x7f406409d6d0 msgr2=0x7f406415d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 shutdown_connections 2024-04-20T10:29:21.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.505+0000 7f4068a00640 1 -- 172.21.15.47:0/2007528837 wait complete. 2024-04-20T10:29:21.524 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:21.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.732+0000 7f7bd1200640 1 Processor -- start 2024-04-20T10:29:21.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.732+0000 7f7bd1200640 1 -- start start 2024-04-20T10:29:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.734+0000 7f7bd1200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7bcc1651d0 0x7f7bcc071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.734+0000 7f7bd1200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7bcc163450 0x7f7bcc1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.734+0000 7f7bd1200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7bcc06c310 con 0x7f7bcc163070 2024-04-20T10:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.734+0000 7f7bd1200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f7bcc073cf0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.734+0000 7f7bd1200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7bcc06c5f0 con 0x7f7bcc163450 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcaa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7bcc1651d0 0x7f7bcc071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcaa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f7bcc1651d0 0x7f7bcc071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58670/0 (socket says 172.21.15.47:58670) 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcaa00640 1 -- 172.21.15.47:0/3600208750 learned_addr learned my addr 172.21.15.47:0/3600208750 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcbe00640 1 --2- 172.21.15.47:0/3600208750 >> v2:172.21.15.175:3300/0 conn(0x7f7bcc163450 0x7f7bcc1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcaa00640 1 -- 172.21.15.47:0/3600208750 >> v2:172.21.15.175:3300/0 conn(0x7f7bcc163450 msgr2=0x7f7bcc1638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcaa00640 1 --2- 172.21.15.47:0/3600208750 >> v2:172.21.15.175:3300/0 conn(0x7f7bcc163450 0x7f7bcc1638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcaa00640 1 -- 172.21.15.47:0/3600208750 >> v1:172.21.15.175:6789/0 conn(0x7f7bcc163070 legacy=0x7f7bcc0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.735+0000 7f7bcaa00640 1 -- 172.21.15.47:0/3600208750 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc06c8d0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.737+0000 7f7bcaa00640 1 --2- 172.21.15.47:0/3600208750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc071d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0d1140 tx=0x7f7bb802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcab8f7b14071f0b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.737+0000 7f7bca000640 1 -- 172.21.15.47:0/3600208750 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8035020 con 0x7f7bcc1651d0 2024-04-20T10:29:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.737+0000 7f7bca000640 1 -- 172.21.15.47:0/3600208750 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bb8031890 con 0x7f7bcc1651d0 2024-04-20T10:29:21.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.737+0000 7f7bca000640 1 -- 172.21.15.47:0/3600208750 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bb8031c60 con 0x7f7bcc1651d0 2024-04-20T10:29:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 -- 172.21.15.47:0/3600208750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 msgr2=0x7f7bcc071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3600208750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc071d90 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc0d1140 tx=0x7f7bb802b930 comp rx=0 tx=0).stop 2024-04-20T10:29:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 -- 172.21.15.47:0/3600208750 shutdown_connections 2024-04-20T10:29:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3600208750 >> v2:172.21.15.175:3300/0 conn(0x7f7bcc163450 0x7f7bcc1638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3600208750 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc071d90 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 -- 172.21.15.47:0/3600208750 >> 172.21.15.47:0/3600208750 conn(0x7f7bcc0d0040 msgr2=0x7f7bcc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:21.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 -- 172.21.15.47:0/3600208750 shutdown_connections 2024-04-20T10:29:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.738+0000 7f7bd1200640 1 -- 172.21.15.47:0/3600208750 wait complete. 2024-04-20T10:29:21.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.740+0000 7f7bd1200640 1 Processor -- start 2024-04-20T10:29:21.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.740+0000 7f7bd1200640 1 -- start start 2024-04-20T10:29:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bd1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bcc163070 0x7f7bcc09d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bd1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc163450 0x7f7bcc153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bd1200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bd1200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc163e20 con 0x7f7bcc1651d0 2024-04-20T10:29:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bd1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7bcc06ce90 con 0x7f7bcc163450 2024-04-20T10:29:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bd1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7bcc1662a0 con 0x7f7bcc163070 2024-04-20T10:29:21.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bcbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc0c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58684/0 (socket says 172.21.15.47:58684) 2024-04-20T10:29:21.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.741+0000 7f7bcbe00640 1 -- 172.21.15.47:0/3201267938 learned_addr learned my addr 172.21.15.47:0/3201267938 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bcbe00640 1 -- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bcc163070 msgr2=0x7f7bcc09d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bcbe00640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bcc163070 0x7f7bcc09d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bcbe00640 1 -- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc163450 msgr2=0x7f7bcc153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bcaa00640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc163450 0x7f7bcc153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bcbe00640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc163450 0x7f7bcc153ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bcbe00640 1 -- 172.21.15.47:0/3201267938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bcc1541e0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bcbe00640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc0c4ee0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d830 tx=0x7f7bc400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bc3400640 1 -- 172.21.15.47:0/3201267938 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400b6c0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bc3400640 1 -- 172.21.15.47:0/3201267938 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7bc4019070 con 0x7f7bcc1651d0 2024-04-20T10:29:21.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bc3400640 1 -- 172.21.15.47:0/3201267938 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc400bac0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bcc0c5420 con 0x7f7bcc1651d0 2024-04-20T10:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.742+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bcc06e920 con 0x7f7bcc1651d0 2024-04-20T10:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.743+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bcc075b60 con 0x7f7bcc1651d0 2024-04-20T10:29:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.745+0000 7f7bc3400640 1 -- 172.21.15.47:0/3201267938 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7bc401e460 con 0x7f7bcc1651d0 2024-04-20T10:29:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.746+0000 7f7bc3400640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:21.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.746+0000 7f7bc3400640 1 -- 172.21.15.47:0/3201267938 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f7bc408bff0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.746+0000 7f7bcb400640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:21.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.748+0000 7f7bcb400640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7bac06c8f0 0x7f7bac06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0012e0 tx=0x7f7bbc007d30 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:21.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.750+0000 7f7bc3400640 1 -- 172.21.15.47:0/3201267938 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7bc405a5e0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.958+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bcc164d70 con 0x7f7bcc1651d0 2024-04-20T10:29:21.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.961+0000 7f7bc3400640 1 -- 172.21.15.47:0/3201267938 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13176 (secure 0 0 0) 0x7f7bc4059ee0 con 0x7f7bcc1651d0 2024-04-20T10:29:21.961 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:21.961 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":38,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:17.737538+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7bac06c8f0 msgr2=0x7f7bac06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7bac06c8f0 0x7f7bac06edb0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc0012e0 tx=0x7f7bbc007d30 comp rx=0 tx=0).stop 2024-04-20T10:29:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 msgr2=0x7f7bcc0c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc0c4ee0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400d830 tx=0x7f7bc400dcf0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 shutdown_connections 2024-04-20T10:29:21.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7bac06c8f0 0x7f7bac06edb0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7bcc1651d0 0x7f7bcc0c4ee0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7bcc163450 0x7f7bcc153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 --2- 172.21.15.47:0/3201267938 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7bcc163070 0x7f7bcc09d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.963+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 >> 172.21.15.47:0/3201267938 conn(0x7f7bcc0d0040 msgr2=0x7f7bcc164030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.964+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 shutdown_connections 2024-04-20T10:29:21.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:21.964+0000 7f7bd1200640 1 -- 172.21.15.47:0/3201267938 wait complete. 2024-04-20T10:29:21.985 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:29:21.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:29:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.198+0000 7f4b1d000640 1 Processor -- start 2024-04-20T10:29:22.198 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.198+0000 7f4b1d000640 1 -- start start 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b1d000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4b1815a130 0x7f4b18073470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b1d000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4b18071a60 0x7f4b180739b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b1d000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4b1815cc20 con 0x7f4b1806f160 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b1d000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1815d7f0 con 0x7f4b18071a60 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b1d000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4b1815cec0 con 0x7f4b1815a130 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b0fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4b18071a60 0x7f4b180739b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b0fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4b18071a60 0x7f4b180739b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58696/0 (socket says 172.21.15.47:58696) 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.200+0000 7f4b0fe00640 1 -- 172.21.15.47:0/3748322976 learned_addr learned my addr 172.21.15.47:0/3748322976 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:22.201 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.201+0000 7f4b17400640 1 --2- 172.21.15.47:0/3748322976 >> v2:172.21.15.175:3300/0 conn(0x7f4b1815a130 0x7f4b18073470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.201+0000 7f4b0fe00640 1 -- 172.21.15.47:0/3748322976 >> v2:172.21.15.175:3300/0 conn(0x7f4b1815a130 msgr2=0x7f4b18073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:22.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.201+0000 7f4b0fe00640 1 --2- 172.21.15.47:0/3748322976 >> v2:172.21.15.175:3300/0 conn(0x7f4b1815a130 0x7f4b18073470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.201+0000 7f4b0fe00640 1 -- 172.21.15.47:0/3748322976 >> v1:172.21.15.175:6789/0 conn(0x7f4b1806f160 legacy=0x7f4b1806f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:22.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.201+0000 7f4b0fe00640 1 -- 172.21.15.47:0/3748322976 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1815d1a0 con 0x7f4b18071a60 2024-04-20T10:29:22.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.201+0000 7f4b17400640 1 --2- 172.21.15.47:0/3748322976 >> v2:172.21.15.175:3300/0 conn(0x7f4b1815a130 0x7f4b18073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.202+0000 7f4b0fe00640 1 --2- 172.21.15.47:0/3748322976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b18071a60 0x7f4b180739b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b180cda90 tx=0x7f4b0002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deab03a86051bd19 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.202+0000 7f4b16a00640 1 -- 172.21.15.47:0/3748322976 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00038070 con 0x7f4b18071a60 2024-04-20T10:29:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.202+0000 7f4b16a00640 1 -- 172.21.15.47:0/3748322976 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b0002f050 con 0x7f4b18071a60 2024-04-20T10:29:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.202+0000 7f4b16a00640 1 -- 172.21.15.47:0/3748322976 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b00031970 con 0x7f4b18071a60 2024-04-20T10:29:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 -- 172.21.15.47:0/3748322976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b18071a60 msgr2=0x7f4b180739b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 --2- 172.21.15.47:0/3748322976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b18071a60 0x7f4b180739b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f4b180cda90 tx=0x7f4b0002b7f0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 -- 172.21.15.47:0/3748322976 shutdown_connections 2024-04-20T10:29:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 --2- 172.21.15.47:0/3748322976 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b18071a60 0x7f4b180739b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 --2- 172.21.15.47:0/3748322976 >> v2:172.21.15.175:3300/0 conn(0x7f4b1815a130 0x7f4b18073470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 -- 172.21.15.47:0/3748322976 >> 172.21.15.47:0/3748322976 conn(0x7f4b180cffa0 msgr2=0x7f4b180d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 -- 172.21.15.47:0/3748322976 shutdown_connections 2024-04-20T10:29:22.204 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.203+0000 7f4b1d000640 1 -- 172.21.15.47:0/3748322976 wait complete. 2024-04-20T10:29:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.205+0000 7f4b1d000640 1 Processor -- start 2024-04-20T10:29:22.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.205+0000 7f4b1d000640 1 -- start start 2024-04-20T10:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.206+0000 7f4b1d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b1806f160 0x7f4b181539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.206+0000 7f4b1d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18071a60 0x7f4b18151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.206+0000 7f4b1d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b1815a130 0x7f4b18157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.206+0000 7f4b1d000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1806ba80 con 0x7f4b1815a130 2024-04-20T10:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.206+0000 7f4b1d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b1806b900 con 0x7f4b1806f160 2024-04-20T10:29:22.207 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.206+0000 7f4b1d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b1806bc00 con 0x7f4b18071a60 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b1815a130 0x7f4b18157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b1815a130 0x7f4b18157c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58708/0 (socket says 172.21.15.47:58708) 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 -- 172.21.15.47:0/1465203495 learned_addr learned my addr 172.21.15.47:0/1465203495 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17400640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b1806f160 0x7f4b181539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b0fe00640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18071a60 0x7f4b18151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 -- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18071a60 msgr2=0x7f4b18151930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18071a60 0x7f4b18151930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 -- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b1806f160 msgr2=0x7f4b181539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b1806f160 0x7f4b181539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 -- 172.21.15.47:0/1465203495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1815b5c0 con 0x7f4b1815a130 2024-04-20T10:29:22.208 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b0fe00640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18071a60 0x7f4b18151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:29:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.207+0000 7f4b17e00640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b1815a130 0x7f4b18157c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000c780 tx=0x7f4b1000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.208+0000 7f4b0f400640 1 -- 172.21.15.47:0/1465203495 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10017070 con 0x7f4b1815a130 2024-04-20T10:29:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.208+0000 7f4b0f400640 1 -- 172.21.15.47:0/1465203495 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4b10003da0 con 0x7f4b1815a130 2024-04-20T10:29:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.208+0000 7f4b0f400640 1 -- 172.21.15.47:0/1465203495 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100052b0 con 0x7f4b1815a130 2024-04-20T10:29:22.209 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.208+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1815b8a0 con 0x7f4b1815a130 2024-04-20T10:29:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.208+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1815be60 con 0x7f4b1815a130 2024-04-20T10:29:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.209+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b18075b30 con 0x7f4b1815a130 2024-04-20T10:29:22.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.210+0000 7f4b0f400640 1 -- 172.21.15.47:0/1465203495 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4b10005450 con 0x7f4b1815a130 2024-04-20T10:29:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.211+0000 7f4b0f400640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4af406c8f0 0x7f4af406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.211 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.211+0000 7f4b0f400640 1 -- 172.21.15.47:0/1465203495 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f4b1008af70 con 0x7f4b1815a130 2024-04-20T10:29:22.212 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.212+0000 7f4b17400640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4af406c8f0 0x7f4af406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.213+0000 7f4b17400640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4af406c8f0 0x7f4af406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800a950 tx=0x7f4b08008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:22.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.216+0000 7f4b0f400640 1 -- 172.21.15.47:0/1465203495 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4b10059550 con 0x7f4b1815a130 2024-04-20T10:29:22.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.418+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4b180cd510 con 0x7f4b1815a130 2024-04-20T10:29:22.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.418+0000 7f4b0f400640 1 -- 172.21.15.47:0/1465203495 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f4b10058e50 con 0x7f4b1815a130 2024-04-20T10:29:22.419 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:29:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4af406c8f0 msgr2=0x7f4af406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4af406c8f0 0x7f4af406edb0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4b0800a950 tx=0x7f4b08008040 comp rx=0 tx=0).stop 2024-04-20T10:29:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b1815a130 msgr2=0x7f4b18157c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b1815a130 0x7f4b18157c00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000c780 tx=0x7f4b1000cc40 comp rx=0 tx=0).stop 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 shutdown_connections 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4af406c8f0 0x7f4af406edb0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4b1815a130 0x7f4b18157c00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18071a60 0x7f4b18151930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 --2- 172.21.15.47:0/1465203495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b1806f160 0x7f4b181539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 >> 172.21.15.47:0/1465203495 conn(0x7f4b180cffa0 msgr2=0x7f4b180acfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 shutdown_connections 2024-04-20T10:29:22.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.421+0000 7f4b1d000640 1 -- 172.21.15.47:0/1465203495 wait complete. 2024-04-20T10:29:22.441 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T10:29:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.655+0000 7f193f200640 1 Processor -- start 2024-04-20T10:29:22.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.655+0000 7f193f200640 1 -- start start 2024-04-20T10:29:22.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f194015b400 0x7f1940072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193f200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f194015eec0 0x7f194015f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193f200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f194006d280 con 0x7f194015eec0 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f194015fcd0 con 0x7f194015b400 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f194006ccc0 con 0x7f194015f870 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f194015eec0 0x7f194015f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f194015eec0 0x7f194015f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58722/0 (socket says 172.21.15.47:58722) 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193d400640 1 -- 172.21.15.47:0/3829557070 learned_addr learned my addr 172.21.15.47:0/3829557070 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.657+0000 7f193de00640 1 --2- 172.21.15.47:0/3829557070 >> v2:172.21.15.175:3300/0 conn(0x7f194015b400 0x7f1940072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.658+0000 7f193e800640 1 --2- 172.21.15.47:0/3829557070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.658+0000 7f193d400640 1 -- 172.21.15.47:0/3829557070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 msgr2=0x7f194015b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.658+0000 7f193d400640 1 --2- 172.21.15.47:0/3829557070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.658+0000 7f193d400640 1 -- 172.21.15.47:0/3829557070 >> v2:172.21.15.175:3300/0 conn(0x7f194015b400 msgr2=0x7f1940072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.658+0000 7f193d400640 1 --2- 172.21.15.47:0/3829557070 >> v2:172.21.15.175:3300/0 conn(0x7f194015b400 0x7f1940072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.658+0000 7f193d400640 1 -- 172.21.15.47:0/3829557070 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194006cfa0 con 0x7f194015eec0 2024-04-20T10:29:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.659+0000 7f193d400640 1 --2- 172.21.15.47:0/3829557070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015eec0 0x7f194015f330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1940074060 tx=0x7f192802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=734329a4afbb78e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.659+0000 7f193ca00640 1 -- 172.21.15.47:0/3829557070 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928035020 con 0x7f194015eec0 2024-04-20T10:29:22.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.659+0000 7f193ca00640 1 -- 172.21.15.47:0/3829557070 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1928031870 con 0x7f194015eec0 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.659+0000 7f193ca00640 1 -- 172.21.15.47:0/3829557070 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1928031c40 con 0x7f194015eec0 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 -- 172.21.15.47:0/3829557070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015eec0 msgr2=0x7f194015f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 --2- 172.21.15.47:0/3829557070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015eec0 0x7f194015f330 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f1940074060 tx=0x7f192802b910 comp rx=0 tx=0).stop 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 -- 172.21.15.47:0/3829557070 shutdown_connections 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 --2- 172.21.15.47:0/3829557070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 --2- 172.21.15.47:0/3829557070 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015eec0 0x7f194015f330 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 --2- 172.21.15.47:0/3829557070 >> v2:172.21.15.175:3300/0 conn(0x7f194015b400 0x7f1940072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 -- 172.21.15.47:0/3829557070 >> 172.21.15.47:0/3829557070 conn(0x7f19400d0040 msgr2=0x7f19400d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 -- 172.21.15.47:0/3829557070 shutdown_connections 2024-04-20T10:29:22.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.660+0000 7f193f200640 1 -- 172.21.15.47:0/3829557070 wait complete. 2024-04-20T10:29:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.662+0000 7f193f200640 1 Processor -- start 2024-04-20T10:29:22.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.662+0000 7f193f200640 1 -- start start 2024-04-20T10:29:22.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.663+0000 7f193f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015b400 0x7f19401531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.663+0000 7f193f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f194015eec0 0x7f1940151820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.663+0000 7f193f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.663+0000 7f193f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1940073240 con 0x7f194015b400 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.663+0000 7f193f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f194006e720 con 0x7f194015eec0 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.663+0000 7f193f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f194015bd20 con 0x7f194015f870 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015b400 0x7f19401531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015b400 0x7f19401531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58736/0 (socket says 172.21.15.47:58736) 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 -- 172.21.15.47:0/1448163230 learned_addr learned my addr 172.21.15.47:0/1448163230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193d400640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f194015eec0 0x7f1940151820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193e800640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 -- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 msgr2=0x7f194015ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 -- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f194015eec0 msgr2=0x7f1940151820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f194015eec0 0x7f1940151820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193de00640 1 -- 172.21.15.47:0/1448163230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1940152100 con 0x7f194015b400 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.664+0000 7f193d400640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f194015eec0 0x7f1940151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.665+0000 7f193de00640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015b400 0x7f19401531a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1930007e90 tx=0x7f193000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.665+0000 7f1936000640 1 -- 172.21.15.47:0/1448163230 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930014620 con 0x7f194015b400 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.665+0000 7f1936000640 1 -- 172.21.15.47:0/1448163230 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1930019070 con 0x7f194015b400 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.665+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19401523e0 con 0x7f194015b400 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.665+0000 7f1936000640 1 -- 172.21.15.47:0/1448163230 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1930014a00 con 0x7f194015b400 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.665+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401529a0 con 0x7f194015b400 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.667+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19400742b0 con 0x7f194015b400 2024-04-20T10:29:22.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.667+0000 7f1936000640 1 -- 172.21.15.47:0/1448163230 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1930014ba0 con 0x7f194015b400 2024-04-20T10:29:22.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.668+0000 7f1936000640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f191c06c8f0 0x7f191c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:22.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.669+0000 7f1936000640 1 -- 172.21.15.47:0/1448163230 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f193008c230 con 0x7f194015b400 2024-04-20T10:29:22.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.669+0000 7f193d400640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f191c06c8f0 0x7f191c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:22.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.670+0000 7f193d400640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f191c06c8f0 0x7f191c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1928036000 tx=0x7f192803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:22.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.673+0000 7f1936000640 1 -- 172.21.15.47:0/1448163230 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f19400742b0 con 0x7f194015b400 2024-04-20T10:29:22.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.881+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f19400b5220 con 0x7f194015b400 2024-04-20T10:29:22.882 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.882+0000 7f1936000640 1 -- 172.21.15.47:0/1448163230 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) ==== 84+0+65 (secure 0 0 0) 0x7f1930011260 con 0x7f194015b400 2024-04-20T10:29:22.882 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:22.882 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-20T10:29:22.882 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:29:22.882 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-20T10:29:22.882 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-20T10:29:22.883 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:22.883 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f191c06c8f0 msgr2=0x7f191c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f191c06c8f0 0x7f191c06edb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1928036000 tx=0x7f192803c040 comp rx=0 tx=0).stop 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015b400 msgr2=0x7f19401531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015b400 0x7f19401531a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f1930007e90 tx=0x7f193000e4b0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 shutdown_connections 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f194015f870 0x7f194015ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f194015eec0 0x7f1940151820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f191c06c8f0 0x7f191c06edb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 --2- 172.21.15.47:0/1448163230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f194015b400 0x7f19401531a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:22.885 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 >> 172.21.15.47:0/1448163230 conn(0x7f19400d0040 msgr2=0x7f1940157850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 shutdown_connections 2024-04-20T10:29:22.886 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:22.885+0000 7f193f200640 1 -- 172.21.15.47:0/1448163230 wait complete. 2024-04-20T10:29:22.905 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T10:29:22.906 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T10:29:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.124+0000 7f0139800640 1 Processor -- start 2024-04-20T10:29:23.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.125+0000 7f0139800640 1 -- start start 2024-04-20T10:29:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.126+0000 7f0139800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 0x7f01340d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.126+0000 7f0139800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01340d2e30 con 0x7f0134161ee0 2024-04-20T10:29:23.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.126+0000 7f0139800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f01340d3110 con 0x7f01340d4e40 2024-04-20T10:29:23.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.126+0000 7f0139800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01340d33f0 con 0x7f01340d2460 2024-04-20T10:29:23.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.127+0000 7f0138e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0134161ee0 0x7f01340cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43452/0 (socket says 172.21.15.47:43452) 2024-04-20T10:29:23.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.127+0000 7f0138e00640 1 -- 172.21.15.47:0/1158977338 learned_addr learned my addr 172.21.15.47:0/1158977338 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:23.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.127+0000 7f0133e00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 0x7f01340d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.127+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4131170002 0 0) 0x7f01340d2e30 con 0x7f0134161ee0 2024-04-20T10:29:23.128 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.127+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0118003520 con 0x7f0134161ee0 2024-04-20T10:29:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1115260487 0 0) 0x7f0118003520 con 0x7f0134161ee0 2024-04-20T10:29:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 msgr2=0x7f01340d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:23.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 0x7f01340d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 >> v1:172.21.15.175:6789/0 conn(0x7f01340d4e40 legacy=0x7f01341617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01340d36d0 con 0x7f0134161ee0 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f012c001670 con 0x7f0134161ee0 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 >> v1:172.21.15.47:6789/0 conn(0x7f0134161ee0 legacy=0x7f01340cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f0118003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0118004800 0x7f0118004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 0x7f01180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01340d33f0 con 0x7f01180039e0 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f011800bb00 con 0x7f0118004800 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f011800bd00 con 0x7f01340d2460 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.128+0000 7f0133e00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f0118003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0138e00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 0x7f01180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0133400640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0118004800 0x7f0118004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0133e00640 1 -- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 msgr2=0x7f01180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:23.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0133e00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01340d2460 0x7f01180051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0133e00640 1 -- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0118004800 msgr2=0x7f0118004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:23.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0133e00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0118004800 0x7f0118004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0133e00640 1 -- 172.21.15.47:0/1158977338 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0118003520 con 0x7f01180039e0 2024-04-20T10:29:23.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.129+0000 7f0133400640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0118004800 0x7f0118004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.130+0000 7f0133e00640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f0118003e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0124009f90 tx=0x7f0118005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223532ad1c4fd365 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.130+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124039070 con 0x7f01180039e0 2024-04-20T10:29:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.130+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f012402cbd0 con 0x7f01180039e0 2024-04-20T10:29:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.130+0000 7f0132a00640 1 -- 172.21.15.47:0/1158977338 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124009c10 con 0x7f01180039e0 2024-04-20T10:29:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.131+0000 7f0139800640 1 -- 172.21.15.47:0/1158977338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 msgr2=0x7f0118003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.131+0000 7f0139800640 1 --2- 172.21.15.47:0/1158977338 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f0118003e00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0124009f90 tx=0x7f0118005830 comp rx=0 tx=0).stop 2024-04-20T10:29:23.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.131+0000 7f0138e00640 1 -- 172.21.15.47:0/1158977338 reap_dead start 2024-04-20T10:29:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.131+0000 7f0139800640 1 -- 172.21.15.47:0/1158977338 shutdown_connections 2024-04-20T10:29:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.131+0000 7f0139800640 1 -- 172.21.15.47:0/1158977338 >> 172.21.15.47:0/1158977338 conn(0x7f01340b7570 msgr2=0x7f01340b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.131+0000 7f0139800640 1 -- 172.21.15.47:0/1158977338 shutdown_connections 2024-04-20T10:29:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.131+0000 7f0139800640 1 -- 172.21.15.47:0/1158977338 wait complete. 2024-04-20T10:29:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.133+0000 7f0139800640 1 Processor -- start 2024-04-20T10:29:23.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.133+0000 7f0139800640 1 -- start start 2024-04-20T10:29:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.134+0000 7f0139800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f01340cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.134+0000 7f0139800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01340b4be0 0x7f01340b6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.134+0000 7f0139800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01341f8650 0x7f013406b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.134+0000 7f0139800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f01340d4650 con 0x7f01180039e0 2024-04-20T10:29:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.134+0000 7f0139800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01340d44d0 con 0x7f01340b4be0 2024-04-20T10:29:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.134+0000 7f0139800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01340d2840 con 0x7f01341f8650 2024-04-20T10:29:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f01340cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f01340cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:58760/0 (socket says 172.21.15.47:58760) 2024-04-20T10:29:23.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 -- 172.21.15.47:0/2926161214 learned_addr learned my addr 172.21.15.47:0/2926161214 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0138e00640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01341f8650 0x7f013406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133400640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01340b4be0 0x7f01340b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 -- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01341f8650 msgr2=0x7f013406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:23.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01341f8650 0x7f013406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 -- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01340b4be0 msgr2=0x7f01340b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01340b4be0 0x7f01340b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 -- 172.21.15.47:0/2926161214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01340b50a0 con 0x7f01180039e0 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133400640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01340b4be0 0x7f01340b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.135+0000 7f0133e00640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f01340cacc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f012402c640 tx=0x7f0124009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.136+0000 7f012be00640 1 -- 172.21.15.47:0/2926161214 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124042020 con 0x7f01180039e0 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.136+0000 7f012be00640 1 -- 172.21.15.47:0/2926161214 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f012403a040 con 0x7f01180039e0 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.136+0000 7f012be00640 1 -- 172.21.15.47:0/2926161214 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0124039070 con 0x7f01180039e0 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.136+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01340b5380 con 0x7f01180039e0 2024-04-20T10:29:23.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.136+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01340b5940 con 0x7f01180039e0 2024-04-20T10:29:23.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.137+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01340b6680 con 0x7f01180039e0 2024-04-20T10:29:23.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.138+0000 7f012be00640 1 -- 172.21.15.47:0/2926161214 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0124005d10 con 0x7f01180039e0 2024-04-20T10:29:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.139+0000 7f012be00640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f011006c920 0x7f011006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.139+0000 7f012be00640 1 -- 172.21.15.47:0/2926161214 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 5249+0+0 (secure 0 0 0) 0x7f01240adfc0 con 0x7f01180039e0 2024-04-20T10:29:23.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.140+0000 7f0133400640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f011006c920 0x7f011006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:23.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.141+0000 7f0133400640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f011006c920 0x7f011006ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0134153940 tx=0x7f011c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:23.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.144+0000 7f012be00640 1 -- 172.21.15.47:0/2926161214 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f012404c030 con 0x7f01180039e0 2024-04-20T10:29:23.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.349+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0134075b30 con 0x7f01180039e0 2024-04-20T10:29:23.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.857+0000 7f012be00640 1 -- 172.21.15.47:0/2926161214 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v39) ==== 138+0+0 (secure 0 0 0) 0x7f012407c510 con 0x7f01180039e0 2024-04-20T10:29:23.858 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.861+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f011006c920 msgr2=0x7f011006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.861+0000 7f0139800640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f011006c920 0x7f011006ede0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0134153940 tx=0x7f011c009040 comp rx=0 tx=0).stop 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.861+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 msgr2=0x7f01340cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.861+0000 7f0139800640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f01340cacc0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f012402c640 tx=0x7f0124009750 comp rx=0 tx=0).stop 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 shutdown_connections 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01341f8650 0x7f013406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01340b4be0 0x7f01340b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f011006c920 0x7f011006ede0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 --2- 172.21.15.47:0/2926161214 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f01180039e0 0x7f01340cacc0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:23.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 >> 172.21.15.47:0/2926161214 conn(0x7f01340b7570 msgr2=0x7f01340cd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:23.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 shutdown_connections 2024-04-20T10:29:23.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:23.862+0000 7f0139800640 1 -- 172.21.15.47:0/2926161214 wait complete. 2024-04-20T10:29:23.887 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T10:29:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.087+0000 7fca55c00640 1 Processor -- start 2024-04-20T10:29:24.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.087+0000 7fca55c00640 1 -- start start 2024-04-20T10:29:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca55c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca50159e10 0x7fca50071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca55c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 0x7fca50071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca55c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fca5015cc90 con 0x7fca5006da90 2024-04-20T10:29:24.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca55c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fca50072900 con 0x7fca50159e10 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca55c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca5015ca40 con 0x7fca5015a1f0 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca4fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca50159e10 0x7fca50071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca4fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fca50159e10 0x7fca50071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34554/0 (socket says 172.21.15.47:34554) 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.089+0000 7fca4fe00640 1 -- 172.21.15.47:0/995811345 learned_addr learned my addr 172.21.15.47:0/995811345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.090+0000 7fca4f400640 1 --2- 172.21.15.47:0/995811345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 0x7fca50071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.090+0000 7fca4fe00640 1 -- 172.21.15.47:0/995811345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 msgr2=0x7fca50071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.090+0000 7fca4fe00640 1 --2- 172.21.15.47:0/995811345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 0x7fca50071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.090+0000 7fca4fe00640 1 -- 172.21.15.47:0/995811345 >> v1:172.21.15.175:6789/0 conn(0x7fca5006da90 legacy=0x7fca50160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:24.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.090+0000 7fca4fe00640 1 -- 172.21.15.47:0/995811345 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5015cf70 con 0x7fca50159e10 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.091+0000 7fca4fe00640 1 --2- 172.21.15.47:0/995811345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca50159e10 0x7fca50071a80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fca5015cbc0 tx=0x7fca4002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c42f0c96c90e00ba server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca4ea00640 1 -- 172.21.15.47:0/995811345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40035020 con 0x7fca50159e10 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca4ea00640 1 -- 172.21.15.47:0/995811345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca40031640 con 0x7fca50159e10 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca4ea00640 1 -- 172.21.15.47:0/995811345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca4002d040 con 0x7fca50159e10 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca55c00640 1 -- 172.21.15.47:0/995811345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca50159e10 msgr2=0x7fca50071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca55c00640 1 --2- 172.21.15.47:0/995811345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca50159e10 0x7fca50071a80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fca5015cbc0 tx=0x7fca4002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca55c00640 1 -- 172.21.15.47:0/995811345 shutdown_connections 2024-04-20T10:29:24.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca55c00640 1 --2- 172.21.15.47:0/995811345 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 0x7fca50071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.092+0000 7fca55c00640 1 --2- 172.21.15.47:0/995811345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca50159e10 0x7fca50071a80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.093+0000 7fca55c00640 1 -- 172.21.15.47:0/995811345 >> 172.21.15.47:0/995811345 conn(0x7fca500d0040 msgr2=0x7fca500d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.093+0000 7fca55c00640 1 -- 172.21.15.47:0/995811345 shutdown_connections 2024-04-20T10:29:24.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.093+0000 7fca55c00640 1 -- 172.21.15.47:0/995811345 wait complete. 2024-04-20T10:29:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.095+0000 7fca55c00640 1 Processor -- start 2024-04-20T10:29:24.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.095+0000 7fca55c00640 1 -- start start 2024-04-20T10:29:24.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.096+0000 7fca55c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca5006da90 0x7fca50156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca55c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca50159e10 0x7fca500c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca55c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 0x7fca500731d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca55c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fca5015d810 con 0x7fca5006da90 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca55c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca5015d530 con 0x7fca50159e10 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca55c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca5006b900 con 0x7fca5015a1f0 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca5006da90 0x7fca50156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca5006da90 0x7fca50156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34558/0 (socket says 172.21.15.47:34558) 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 -- 172.21.15.47:0/3496406550 learned_addr learned my addr 172.21.15.47:0/3496406550 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 -- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 msgr2=0x7fca500731d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 0x7fca500731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 -- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca50159e10 msgr2=0x7fca500c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4f400640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca50159e10 0x7fca500c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca50159e10 0x7fca500c92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.097+0000 7fca4fe00640 1 -- 172.21.15.47:0/3496406550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca500c9c40 con 0x7fca5006da90 2024-04-20T10:29:24.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.098+0000 7fca4fe00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca5006da90 0x7fca50156a70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.098+0000 7fca47e00640 1 -- 172.21.15.47:0/3496406550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40040070 con 0x7fca5006da90 2024-04-20T10:29:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.098+0000 7fca47e00640 1 -- 172.21.15.47:0/3496406550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fca400399f0 con 0x7fca5006da90 2024-04-20T10:29:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.098+0000 7fca47e00640 1 -- 172.21.15.47:0/3496406550 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40035040 con 0x7fca5006da90 2024-04-20T10:29:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.098+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca500c9f20 con 0x7fca5006da90 2024-04-20T10:29:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.098+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca500ca7c0 con 0x7fca5006da90 2024-04-20T10:29:24.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.100+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca50075b60 con 0x7fca5006da90 2024-04-20T10:29:24.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.101+0000 7fca47e00640 1 -- 172.21.15.47:0/3496406550 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fca4004b060 con 0x7fca5006da90 2024-04-20T10:29:24.102 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.102+0000 7fca47e00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:24.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.102+0000 7fca47e00640 1 -- 172.21.15.47:0/3496406550 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(39..39 src has 1..39) ==== 5619+0+0 (secure 0 0 0) 0x7fca400ad4e0 con 0x7fca5006da90 2024-04-20T10:29:24.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.103+0000 7fca4f400640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:24.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.104+0000 7fca4f400640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fca3006c8f0 0x7fca3006edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006fd0 tx=0x7fca3c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:24.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.108+0000 7fca47e00640 1 -- 172.21.15.47:0/3496406550 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fca4002f300 con 0x7fca5006da90 2024-04-20T10:29:24.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.312+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fca5015c820 con 0x7fca5006da90 2024-04-20T10:29:24.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.864+0000 7fca47e00640 1 -- 172.21.15.47:0/3496406550 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v40) ==== 170+0+0 (secure 0 0 0) 0x7fca4007b960 con 0x7fca5006da90 2024-04-20T10:29:24.864 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-20T10:29:24.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.868+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fca3006c8f0 msgr2=0x7fca3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:24.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.868+0000 7fca55c00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fca3006c8f0 0x7fca3006edb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fca3c006fd0 tx=0x7fca3c008040 comp rx=0 tx=0).stop 2024-04-20T10:29:24.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.868+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca5006da90 msgr2=0x7fca50156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:24.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.868+0000 7fca55c00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca5006da90 0x7fca50156a70 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fca40000c00 tx=0x7fca400046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.871+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 shutdown_connections 2024-04-20T10:29:24.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.871+0000 7fca55c00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca5015a1f0 0x7fca500731d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.871+0000 7fca55c00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca50159e10 0x7fca500c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.871+0000 7fca55c00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fca3006c8f0 0x7fca3006edb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.871+0000 7fca55c00640 1 --2- 172.21.15.47:0/3496406550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fca5006da90 0x7fca50156a70 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.871+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 >> 172.21.15.47:0/3496406550 conn(0x7fca500d0040 msgr2=0x7fca5006cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.872+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 shutdown_connections 2024-04-20T10:29:24.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:24.872+0000 7fca55c00640 1 -- 172.21.15.47:0/3496406550 wait complete. 2024-04-20T10:29:24.905 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T10:29:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.086+0000 7ff36d800640 1 Processor -- start 2024-04-20T10:29:25.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.087+0000 7ff36d800640 1 -- start start 2024-04-20T10:29:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.088+0000 7ff36d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff36806e2a0 0x7ff36815eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.088+0000 7ff36d800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff36815f960 con 0x7ff368158ca0 2024-04-20T10:29:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.088+0000 7ff36d800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff36806b9a0 con 0x7ff36815f0e0 2024-04-20T10:29:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.088+0000 7ff36d800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff36806bc80 con 0x7ff36806e2a0 2024-04-20T10:29:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.088+0000 7ff367e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff368158ca0 0x7ff36806db90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36588/0 (socket says 172.21.15.47:36588) 2024-04-20T10:29:25.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.088+0000 7ff367e00640 1 -- 172.21.15.47:0/1950354733 learned_addr learned my addr 172.21.15.47:0/1950354733 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.088+0000 7ff35f400640 1 --2- 172.21.15.47:0/1950354733 >> v2:172.21.15.47:3300/0 conn(0x7ff36806e2a0 0x7ff36815eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.089+0000 7ff367400640 1 -- 172.21.15.47:0/1950354733 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2579472773 0 0) 0x7ff36815f960 con 0x7ff368158ca0 2024-04-20T10:29:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.089+0000 7ff367400640 1 -- 172.21.15.47:0/1950354733 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff34c003520 con 0x7ff368158ca0 2024-04-20T10:29:25.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.089+0000 7ff35f400640 1 -- 172.21.15.47:0/1950354733 >> v1:172.21.15.175:6789/0 conn(0x7ff36815f0e0 legacy=0x7ff3681460f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.089+0000 7ff35f400640 1 -- 172.21.15.47:0/1950354733 >> v1:172.21.15.47:6789/0 conn(0x7ff368158ca0 legacy=0x7ff36806db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.089+0000 7ff35f400640 1 -- 172.21.15.47:0/1950354733 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36806bf60 con 0x7ff36806e2a0 2024-04-20T10:29:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.090+0000 7ff35f400640 1 --2- 172.21.15.47:0/1950354733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff36806e2a0 0x7ff36815eba0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff3500097a0 tx=0x7ff35002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd86434a3d4206c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.090+0000 7ff367400640 1 -- 172.21.15.47:0/1950354733 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350035020 con 0x7ff36806e2a0 2024-04-20T10:29:25.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.090+0000 7ff367400640 1 -- 172.21.15.47:0/1950354733 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff350031830 con 0x7ff36806e2a0 2024-04-20T10:29:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.090+0000 7ff367400640 1 -- 172.21.15.47:0/1950354733 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350031c00 con 0x7ff36806e2a0 2024-04-20T10:29:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.091+0000 7ff36d800640 1 -- 172.21.15.47:0/1950354733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff36806e2a0 msgr2=0x7ff36815eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.091+0000 7ff36d800640 1 --2- 172.21.15.47:0/1950354733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff36806e2a0 0x7ff36815eba0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff3500097a0 tx=0x7ff35002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.091+0000 7ff36d800640 1 -- 172.21.15.47:0/1950354733 shutdown_connections 2024-04-20T10:29:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.091+0000 7ff36d800640 1 --2- 172.21.15.47:0/1950354733 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff36806e2a0 0x7ff36815eba0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.091+0000 7ff36d800640 1 -- 172.21.15.47:0/1950354733 >> 172.21.15.47:0/1950354733 conn(0x7ff3680b5270 msgr2=0x7ff3680b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.091+0000 7ff36d800640 1 -- 172.21.15.47:0/1950354733 shutdown_connections 2024-04-20T10:29:25.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.091+0000 7ff36d800640 1 -- 172.21.15.47:0/1950354733 wait complete. 2024-04-20T10:29:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.093+0000 7ff36d800640 1 Processor -- start 2024-04-20T10:29:25.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.093+0000 7ff36d800640 1 -- start start 2024-04-20T10:29:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff36d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff36806e2a0 0x7ff3680ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff36d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368158ca0 0x7ff368153fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff36d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff36815f0e0 0x7ff3681531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:25.094 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff36d800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff36806c760 con 0x7ff368158ca0 2024-04-20T10:29:25.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff36d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff36815f510 con 0x7ff36806e2a0 2024-04-20T10:29:25.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff36d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff3681f7b00 con 0x7ff36815f0e0 2024-04-20T10:29:25.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff35f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368158ca0 0x7ff368153fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:25.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff35f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368158ca0 0x7ff368153fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34586/0 (socket says 172.21.15.47:34586) 2024-04-20T10:29:25.095 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.094+0000 7ff35f400640 1 -- 172.21.15.47:0/2241713714 learned_addr learned my addr 172.21.15.47:0/2241713714 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff367e00640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff36806e2a0 0x7ff3680ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff36ce00640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff36815f0e0 0x7ff3681531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff35f400640 1 -- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff36815f0e0 msgr2=0x7ff3681531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff35f400640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff36815f0e0 0x7ff3681531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff35f400640 1 -- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff36806e2a0 msgr2=0x7ff3680ad6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff35f400640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff36806e2a0 0x7ff3680ad6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.096 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff35f400640 1 -- 172.21.15.47:0/2241713714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3681f7de0 con 0x7ff368158ca0 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff367e00640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff36806e2a0 0x7ff3680ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff35f400640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368158ca0 0x7ff368153fc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff35002bee0 tx=0x7ff3500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.095+0000 7ff364c00640 1 -- 172.21.15.47:0/2241713714 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350041070 con 0x7ff368158ca0 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.096+0000 7ff364c00640 1 -- 172.21.15.47:0/2241713714 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff35003f070 con 0x7ff368158ca0 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.096+0000 7ff364c00640 1 -- 172.21.15.47:0/2241713714 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff350035040 con 0x7ff368158ca0 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.096+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3681f80c0 con 0x7ff368158ca0 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.096+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff368154500 con 0x7ff368158ca0 2024-04-20T10:29:25.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.097+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff368075b30 con 0x7ff368158ca0 2024-04-20T10:29:25.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.098+0000 7ff364c00640 1 -- 172.21.15.47:0/2241713714 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff350038020 con 0x7ff368158ca0 2024-04-20T10:29:25.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.099+0000 7ff364c00640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff34406c8f0 0x7ff34406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:25.099 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.099+0000 7ff364c00640 1 -- 172.21.15.47:0/2241713714 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 6001+0+0 (secure 0 0 0) 0x7ff3500ad920 con 0x7ff368158ca0 2024-04-20T10:29:25.100 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.100+0000 7ff367e00640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff34406c8f0 0x7ff34406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:25.101 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.102+0000 7ff367e00640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff34406c8f0 0x7ff34406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff358005880 tx=0x7ff358009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:25.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.105+0000 7ff364c00640 1 -- 172.21.15.47:0/2241713714 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff350031800 con 0x7ff368158ca0 2024-04-20T10:29:25.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.351+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff36806d170 con 0x7ff368158ca0 2024-04-20T10:29:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.908+0000 7ff364c00640 1 -- 172.21.15.47:0/2241713714 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v15) ==== 172+0+0 (secure 0 0 0) 0x7ff35007bc90 con 0x7ff368158ca0 2024-04-20T10:29:25.909 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 5 and data pool 6 2024-04-20T10:29:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.914+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff34406c8f0 msgr2=0x7ff34406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.914+0000 7ff36d800640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff34406c8f0 0x7ff34406edb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff358005880 tx=0x7ff358009040 comp rx=0 tx=0).stop 2024-04-20T10:29:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.914+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368158ca0 msgr2=0x7ff368153fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.914+0000 7ff36d800640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368158ca0 0x7ff368153fc0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff35002bee0 tx=0x7ff3500046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.916+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 shutdown_connections 2024-04-20T10:29:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.916+0000 7ff36d800640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff36815f0e0 0x7ff3681531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.916+0000 7ff36d800640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff34406c8f0 0x7ff34406edb0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.916+0000 7ff36d800640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff368158ca0 0x7ff368153fc0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.916+0000 7ff36d800640 1 --2- 172.21.15.47:0/2241713714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff36806e2a0 0x7ff3680ad6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:25.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.916+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 >> 172.21.15.47:0/2241713714 conn(0x7ff3680b5270 msgr2=0x7ff3680726f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.919+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 shutdown_connections 2024-04-20T10:29:25.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:25.919+0000 7ff36d800640 1 -- 172.21.15.47:0/2241713714 wait complete. 2024-04-20T10:29:25.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:26.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.131+0000 7f5107200640 1 Processor -- start 2024-04-20T10:29:26.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.131+0000 7f5107200640 1 -- start start 2024-04-20T10:29:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.132+0000 7f5107200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5100156c20 0x7f510006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.132+0000 7f5107200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f51000d0360 0x7f51000d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.132+0000 7f5107200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5100141c60 con 0x7f510015ca40 2024-04-20T10:29:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.132+0000 7f5107200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f51000adab0 con 0x7f51000d0360 2024-04-20T10:29:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.132+0000 7f5107200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f510015cf90 con 0x7f5100156c20 2024-04-20T10:29:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.133+0000 7f50ff400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f510015ca40 0x7f51000ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36590/0 (socket says 172.21.15.47:36590) 2024-04-20T10:29:26.133 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.133+0000 7f50ff400640 1 -- 172.21.15.47:0/2337157402 learned_addr learned my addr 172.21.15.47:0/2337157402 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:26.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.133+0000 7f50ffe00640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5100156c20 0x7f510006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.133+0000 7f5105200640 1 --2- 172.21.15.47:0/2337157402 >> v2:172.21.15.175:3300/0 conn(0x7f51000d0360 0x7f51000d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.133+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3090206600 0 0) 0x7f5100141c60 con 0x7f510015ca40 2024-04-20T10:29:26.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f50e4003520 con 0x7f510015ca40 2024-04-20T10:29:26.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1130052701 0 0) 0x7f50e4003520 con 0x7f510015ca40 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5100156c20 msgr2=0x7f510006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5100156c20 0x7f510006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 >> v2:172.21.15.175:3300/0 conn(0x7f51000d0360 msgr2=0x7f51000d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 --2- 172.21.15.47:0/2337157402 >> v2:172.21.15.175:3300/0 conn(0x7f51000d0360 0x7f51000d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510015d270 con 0x7f510015ca40 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f50ec001670 con 0x7f510015ca40 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 >> v1:172.21.15.47:6789/0 conn(0x7f510015ca40 legacy=0x7f51000ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5100156c20 0x7f50e40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4003f30 0x7f50e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50e4004d40 0x7f50e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f51000adab0 con 0x7f50e4004d40 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f510015cf90 con 0x7f50e4003f30 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f50e400b9c0 con 0x7f5100156c20 2024-04-20T10:29:26.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.134+0000 7f5105200640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50e4004d40 0x7f50e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.135+0000 7f50ff400640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4003f30 0x7f50e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.135+0000 7f5105200640 1 -- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5100156c20 msgr2=0x7f50e40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.135+0000 7f5105200640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5100156c20 0x7f50e40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.135+0000 7f5105200640 1 -- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4003f30 msgr2=0x7f50e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.135+0000 7f5105200640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4003f30 0x7f50e4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.135+0000 7f5105200640 1 -- 172.21.15.47:0/2337157402 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50e4003520 con 0x7f50e4004d40 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.136+0000 7f5105200640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50e4004d40 0x7f50e40051c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009fc0 tx=0x7f50e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6574519b5963981d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.136+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8039070 con 0x7f50e4004d40 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.136+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f802cc40 con 0x7f50e4004d40 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.136+0000 7f50fea00640 1 -- 172.21.15.47:0/2337157402 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8009c10 con 0x7f50e4004d40 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.137+0000 7f5107200640 1 -- 172.21.15.47:0/2337157402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50e4004d40 msgr2=0x7f50e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.137+0000 7f5107200640 1 --2- 172.21.15.47:0/2337157402 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50e4004d40 0x7f50e40051c0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f50f8009fc0 tx=0x7f50e4005830 comp rx=0 tx=0).stop 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.137+0000 7f5105200640 1 -- 172.21.15.47:0/2337157402 reap_dead start 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.137+0000 7f5107200640 1 -- 172.21.15.47:0/2337157402 shutdown_connections 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.137+0000 7f5107200640 1 -- 172.21.15.47:0/2337157402 >> 172.21.15.47:0/2337157402 conn(0x7f51000b3350 msgr2=0x7f51000b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.137+0000 7f5107200640 1 -- 172.21.15.47:0/2337157402 shutdown_connections 2024-04-20T10:29:26.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.137+0000 7f5107200640 1 -- 172.21.15.47:0/2337157402 wait complete. 2024-04-20T10:29:26.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.139+0000 7f5107200640 1 Processor -- start 2024-04-20T10:29:26.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.139+0000 7f5107200640 1 -- start start 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5107200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4004d40 0x7f51001f4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5107200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f510015b410 0x7f510015b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5107200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51000cd820 0x7f51000cdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5107200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f510006b900 con 0x7f51000cd820 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5107200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f510015d830 con 0x7f50e4004d40 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5107200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f510006ba80 con 0x7f510015b410 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5105200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51000cd820 0x7f51000cdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5105200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51000cd820 0x7f51000cdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34594/0 (socket says 172.21.15.47:34594) 2024-04-20T10:29:26.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.140+0000 7f5105200640 1 -- 172.21.15.47:0/911900937 learned_addr learned my addr 172.21.15.47:0/911900937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f50ffe00640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4004d40 0x7f51001f4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f50ff400640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f510015b410 0x7f510015b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f5105200640 1 -- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f510015b410 msgr2=0x7f510015b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f5105200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f510015b410 0x7f510015b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f5105200640 1 -- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4004d40 msgr2=0x7f51001f4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f5105200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4004d40 0x7f51001f4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f5105200640 1 -- 172.21.15.47:0/911900937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5100154360 con 0x7f51000cd820 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f50ffe00640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4004d40 0x7f51001f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f5105200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51000cd820 0x7f51000cdcb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f50f8002230 tx=0x7f50f8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:26.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f50f7e00640 1 -- 172.21.15.47:0/911900937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8039030 con 0x7f51000cd820 2024-04-20T10:29:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f50f7e00640 1 -- 172.21.15.47:0/911900937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f50f803a040 con 0x7f51000cd820 2024-04-20T10:29:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f50f7e00640 1 -- 172.21.15.47:0/911900937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f8003990 con 0x7f51000cd820 2024-04-20T10:29:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.141+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510006dce0 con 0x7f51000cd820 2024-04-20T10:29:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.142+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f510006e2a0 con 0x7f51000cd820 2024-04-20T10:29:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.143+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f510015bdd0 con 0x7f51000cd820 2024-04-20T10:29:26.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.144+0000 7f50f7e00640 1 -- 172.21.15.47:0/911900937 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f50f8045030 con 0x7f51000cd820 2024-04-20T10:29:26.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.145+0000 7f50f7e00640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f50e006c920 0x7f50e006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.145+0000 7f50f7e00640 1 -- 172.21.15.47:0/911900937 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7f50f80ad970 con 0x7f51000cd820 2024-04-20T10:29:26.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.145+0000 7f50ffe00640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f50e006c920 0x7f50e006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.147+0000 7f50ffe00640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f50e006c920 0x7f50e006ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51000694a0 tx=0x7f50f0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:26.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.150+0000 7f50f7e00640 1 -- 172.21.15.47:0/911900937 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f50f804c020 con 0x7f51000cd820 2024-04-20T10:29:26.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.355+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f51000742b0 con 0x7f51000cd820 2024-04-20T10:29:26.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.358+0000 7f50f7e00640 1 -- 172.21.15.47:0/911900937 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16967 (secure 0 0 0) 0x7f51000742b0 con 0x7f51000cd820 2024-04-20T10:29:26.358 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:26.359 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:25.353216+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f50e006c920 msgr2=0x7f50e006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f50e006c920 0x7f50e006ede0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f51000694a0 tx=0x7f50f0008040 comp rx=0 tx=0).stop 2024-04-20T10:29:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51000cd820 msgr2=0x7f51000cdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51000cd820 0x7f51000cdcb0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f50f8002230 tx=0x7f50f8009750 comp rx=0 tx=0).stop 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 shutdown_connections 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f50e006c920 0x7f50e006ede0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f51000cd820 0x7f51000cdcb0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f510015b410 0x7f510015b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 --2- 172.21.15.47:0/911900937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50e4004d40 0x7f51001f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 >> 172.21.15.47:0/911900937 conn(0x7f51000b3350 msgr2=0x7f51000ad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 shutdown_connections 2024-04-20T10:29:26.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.361+0000 7f5107200640 1 -- 172.21.15.47:0/911900937 wait complete. 2024-04-20T10:29:26.381 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.591+0000 7ff4e7200640 1 Processor -- start 2024-04-20T10:29:26.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.591+0000 7ff4e7200640 1 -- start start 2024-04-20T10:29:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e7200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4e8159e10 0x7ff4e8071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e815a1f0 0x7ff4e8071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e7200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff4e815cc90 con 0x7ff4e806da90 2024-04-20T10:29:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e7200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff4e8072900 con 0x7ff4e8159e10 2024-04-20T10:29:26.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e815ca40 con 0x7ff4e815a1f0 2024-04-20T10:29:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e5e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4e8159e10 0x7ff4e8071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e6800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff4e806da90 0x7ff4e8160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36600/0 (socket says 172.21.15.47:36600) 2024-04-20T10:29:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e5e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff4e8159e10 0x7ff4e8071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34598/0 (socket says 172.21.15.47:34598) 2024-04-20T10:29:26.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e6800640 1 -- 172.21.15.47:0/1049871395 learned_addr learned my addr 172.21.15.47:0/1049871395 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.593+0000 7ff4e5400640 1 --2- 172.21.15.47:0/1049871395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e815a1f0 0x7ff4e8071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.594+0000 7ff4e5e00640 1 -- 172.21.15.47:0/1049871395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e815a1f0 msgr2=0x7ff4e8071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.594+0000 7ff4e5e00640 1 --2- 172.21.15.47:0/1049871395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e815a1f0 0x7ff4e8071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.594+0000 7ff4e5e00640 1 -- 172.21.15.47:0/1049871395 >> v1:172.21.15.47:6789/0 conn(0x7ff4e806da90 legacy=0x7ff4e8160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.594+0000 7ff4e5e00640 1 -- 172.21.15.47:0/1049871395 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e815cf70 con 0x7ff4e8159e10 2024-04-20T10:29:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.595+0000 7ff4e5e00640 1 --2- 172.21.15.47:0/1049871395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e8159e10 0x7ff4e8071a80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff4e815cbc0 tx=0x7ff4d802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1520b6a7d8ed1be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.595+0000 7ff4e4a00640 1 -- 172.21.15.47:0/1049871395 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8035020 con 0x7ff4e8159e10 2024-04-20T10:29:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.595+0000 7ff4e4a00640 1 -- 172.21.15.47:0/1049871395 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4d8031810 con 0x7ff4e8159e10 2024-04-20T10:29:26.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.595+0000 7ff4e4a00640 1 -- 172.21.15.47:0/1049871395 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8031be0 con 0x7ff4e8159e10 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 -- 172.21.15.47:0/1049871395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e8159e10 msgr2=0x7ff4e8071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 --2- 172.21.15.47:0/1049871395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e8159e10 0x7ff4e8071a80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff4e815cbc0 tx=0x7ff4d802b8b0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 -- 172.21.15.47:0/1049871395 shutdown_connections 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 --2- 172.21.15.47:0/1049871395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e815a1f0 0x7ff4e8071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 --2- 172.21.15.47:0/1049871395 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e8159e10 0x7ff4e8071a80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 -- 172.21.15.47:0/1049871395 >> 172.21.15.47:0/1049871395 conn(0x7ff4e80d0040 msgr2=0x7ff4e80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 -- 172.21.15.47:0/1049871395 shutdown_connections 2024-04-20T10:29:26.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.596+0000 7ff4e7200640 1 -- 172.21.15.47:0/1049871395 wait complete. 2024-04-20T10:29:26.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.598+0000 7ff4e7200640 1 Processor -- start 2024-04-20T10:29:26.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.598+0000 7ff4e7200640 1 -- start start 2024-04-20T10:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e806da90 0x7ff4e8150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e8159e10 0x7ff4e8155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e815a1f0 0x7ff4e80cda30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e815d810 con 0x7ff4e815a1f0 2024-04-20T10:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e815d530 con 0x7ff4e806da90 2024-04-20T10:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e806b900 con 0x7ff4e8159e10 2024-04-20T10:29:26.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e815a1f0 0x7ff4e80cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e815a1f0 0x7ff4e80cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34600/0 (socket says 172.21.15.47:34600) 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.599+0000 7ff4e6800640 1 -- 172.21.15.47:0/4089447501 learned_addr learned my addr 172.21.15.47:0/4089447501 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e5e00640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e806da90 0x7ff4e8150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e5400640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e8159e10 0x7ff4e8155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e6800640 1 -- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e8159e10 msgr2=0x7ff4e8155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e6800640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e8159e10 0x7ff4e8155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e6800640 1 -- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e806da90 msgr2=0x7ff4e8150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e6800640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e806da90 0x7ff4e8150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e6800640 1 -- 172.21.15.47:0/4089447501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e81561d0 con 0x7ff4e815a1f0 2024-04-20T10:29:26.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e6800640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e815a1f0 0x7ff4e80cda30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000cbc0 tx=0x7ff4e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4de000640 1 -- 172.21.15.47:0/4089447501 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0017070 con 0x7ff4e815a1f0 2024-04-20T10:29:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4de000640 1 -- 172.21.15.47:0/4089447501 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff4e000ad40 con 0x7ff4e815a1f0 2024-04-20T10:29:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4de000640 1 -- 172.21.15.47:0/4089447501 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e0012600 con 0x7ff4e815a1f0 2024-04-20T10:29:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.600+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e81564b0 con 0x7ff4e815a1f0 2024-04-20T10:29:26.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.601+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e8156a70 con 0x7ff4e815a1f0 2024-04-20T10:29:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.602+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e8075b60 con 0x7ff4e815a1f0 2024-04-20T10:29:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.603+0000 7ff4de000640 1 -- 172.21.15.47:0/4089447501 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff4e001a070 con 0x7ff4e815a1f0 2024-04-20T10:29:26.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.604+0000 7ff4de000640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4c806c8f0 0x7ff4c806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.604+0000 7ff4de000640 1 -- 172.21.15.47:0/4089447501 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 6101+0+0 (secure 0 0 0) 0x7ff4e008ba50 con 0x7ff4e815a1f0 2024-04-20T10:29:26.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.604+0000 7ff4e5e00640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4c806c8f0 0x7ff4c806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:26.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.605+0000 7ff4e5e00640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4c806c8f0 0x7ff4c806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8036000 tx=0x7ff4d8008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:26.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.608+0000 7ff4de000640 1 -- 172.21.15.47:0/4089447501 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff4e0059600 con 0x7ff4e815a1f0 2024-04-20T10:29:26.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.814+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4e81593a0 con 0x7ff4e815a1f0 2024-04-20T10:29:26.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.816+0000 7ff4de000640 1 -- 172.21.15.47:0/4089447501 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) ==== 74+0+16967 (secure 0 0 0) 0x7ff4e005d900 con 0x7ff4e815a1f0 2024-04-20T10:29:26.817 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:26.817 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:25.353216+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.818+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4c806c8f0 msgr2=0x7ff4c806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.818+0000 7ff4e7200640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4c806c8f0 0x7ff4c806edb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff4d8036000 tx=0x7ff4d8008030 comp rx=0 tx=0).stop 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.818+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e815a1f0 msgr2=0x7ff4e80cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.818+0000 7ff4e7200640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e815a1f0 0x7ff4e80cda30 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000cbc0 tx=0x7ff4e000a430 comp rx=0 tx=0).stop 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 shutdown_connections 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff4c806c8f0 0x7ff4c806edb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff4e815a1f0 0x7ff4e80cda30 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4e8159e10 0x7ff4e8155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 --2- 172.21.15.47:0/4089447501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4e806da90 0x7ff4e8150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:26.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 >> 172.21.15.47:0/4089447501 conn(0x7ff4e80d0040 msgr2=0x7ff4e806c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:26.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 shutdown_connections 2024-04-20T10:29:26.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:26.819+0000 7ff4e7200640 1 -- 172.21.15.47:0/4089447501 wait complete. 2024-04-20T10:29:26.837 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T10:29:27.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.050+0000 7f41fa400640 1 Processor -- start 2024-04-20T10:29:27.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.050+0000 7f41fa400640 1 -- start start 2024-04-20T10:29:27.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41fa400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f4161a50 0x7f41f40b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:27.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41fa400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41f406c010 0x7f41f40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:27.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41fa400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f41f40d0980 con 0x7f41f40d3b20 2024-04-20T10:29:27.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41fa400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f41f40d0c60 con 0x7f41f406c010 2024-04-20T10:29:27.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41fa400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41f4162ae0 con 0x7f41f4161a50 2024-04-20T10:29:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41f9a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41f406c010 0x7f41f40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41f9a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f41f406c010 0x7f41f40ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34614/0 (socket says 172.21.15.47:34614) 2024-04-20T10:29:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.052+0000 7f41f9a00640 1 -- 172.21.15.47:0/1366192661 learned_addr learned my addr 172.21.15.47:0/1366192661 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.053+0000 7f41f9000640 1 --2- 172.21.15.47:0/1366192661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f4161a50 0x7f41f40b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:27.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.053+0000 7f41f9a00640 1 -- 172.21.15.47:0/1366192661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f4161a50 msgr2=0x7f41f40b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.053+0000 7f41f9a00640 1 --2- 172.21.15.47:0/1366192661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f4161a50 0x7f41f40b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.053+0000 7f41f9a00640 1 -- 172.21.15.47:0/1366192661 >> v1:172.21.15.175:6789/0 conn(0x7f41f40d3b20 legacy=0x7f41f406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.053+0000 7f41f9a00640 1 -- 172.21.15.47:0/1366192661 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f4162490 con 0x7f41f406c010 2024-04-20T10:29:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.054+0000 7f41f9a00640 1 --2- 172.21.15.47:0/1366192661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f40ceaa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009e50 tx=0x7f41ec02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcd587acda2a947d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:27.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.054+0000 7f41f3400640 1 -- 172.21.15.47:0/1366192661 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec02ca70 con 0x7f41f406c010 2024-04-20T10:29:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.054+0000 7f41f3400640 1 -- 172.21.15.47:0/1366192661 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41ec02cc10 con 0x7f41f406c010 2024-04-20T10:29:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41f3400640 1 -- 172.21.15.47:0/1366192661 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0343f0 con 0x7f41f406c010 2024-04-20T10:29:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 -- 172.21.15.47:0/1366192661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 msgr2=0x7f41f40ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:27.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 --2- 172.21.15.47:0/1366192661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f40ceaa0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009e50 tx=0x7f41ec02bed0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 -- 172.21.15.47:0/1366192661 shutdown_connections 2024-04-20T10:29:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 --2- 172.21.15.47:0/1366192661 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f40ceaa0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 --2- 172.21.15.47:0/1366192661 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f4161a50 0x7f41f40b3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 -- 172.21.15.47:0/1366192661 >> 172.21.15.47:0/1366192661 conn(0x7f41f40d0ed0 msgr2=0x7f41f40d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 -- 172.21.15.47:0/1366192661 shutdown_connections 2024-04-20T10:29:27.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.055+0000 7f41fa400640 1 -- 172.21.15.47:0/1366192661 wait complete. 2024-04-20T10:29:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.057+0000 7f41fa400640 1 Processor -- start 2024-04-20T10:29:27.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.057+0000 7f41fa400640 1 -- start start 2024-04-20T10:29:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41fa400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f41e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41fa400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41f4161a50 0x7f41f40cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41fa400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f40cdaa0 0x7f41f40b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:27.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41fa400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40d0600 con 0x7f41f406c010 2024-04-20T10:29:27.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41fa400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41f40d0480 con 0x7f41f4161a50 2024-04-20T10:29:27.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41fa400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41f4161e30 con 0x7f41f40cdaa0 2024-04-20T10:29:27.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f41e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:27.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f41e98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34622/0 (socket says 172.21.15.47:34622) 2024-04-20T10:29:27.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.058+0000 7f41f9000640 1 -- 172.21.15.47:0/3827334823 learned_addr learned my addr 172.21.15.47:0/3827334823 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f9000640 1 -- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f40cdaa0 msgr2=0x7f41f40b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f9000640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f40cdaa0 0x7f41f40b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f9000640 1 -- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41f4161a50 msgr2=0x7f41f40cbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f9000640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41f4161a50 0x7f41f40cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f9000640 1 -- 172.21.15.47:0/3827334823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41f41f48f0 con 0x7f41f406c010 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f9000640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f41e98a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41e800d5f0 tx=0x7f41e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f0c00640 1 -- 172.21.15.47:0/3827334823 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e80125f0 con 0x7f41f406c010 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f0c00640 1 -- 172.21.15.47:0/3827334823 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f41e8019070 con 0x7f41f406c010 2024-04-20T10:29:27.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41f0c00640 1 -- 172.21.15.47:0/3827334823 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41e80129d0 con 0x7f41f406c010 2024-04-20T10:29:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.059+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41f40cc630 con 0x7f41f406c010 2024-04-20T10:29:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.060+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41f40ccbf0 con 0x7f41f406c010 2024-04-20T10:29:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.061+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41f40b3010 con 0x7f41f406c010 2024-04-20T10:29:27.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.062+0000 7f41f0c00640 1 -- 172.21.15.47:0/3827334823 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f41e8012b70 con 0x7f41f406c010 2024-04-20T10:29:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.062+0000 7f41f0c00640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:27.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.063+0000 7f41f0c00640 1 -- 172.21.15.47:0/3827334823 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f41e808c1d0 con 0x7f41f406c010 2024-04-20T10:29:27.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.063+0000 7f41f3e00640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:27.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.064+0000 7f41f3e00640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41d406c8f0 0x7f41d406edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41f40cd4d0 tx=0x7f41e00079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:27.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.067+0000 7f41f0c00640 1 -- 172.21.15.47:0/3827334823 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f41f40b3010 con 0x7f41f406c010 2024-04-20T10:29:27.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.319+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f41f40cf930 con 0x7f41f406c010 2024-04-20T10:29:27.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.940+0000 7f41f0c00640 1 -- 172.21.15.47:0/3827334823 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f41f41f48f0 con 0x7f41f406c010 2024-04-20T10:29:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.943+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41d406c8f0 msgr2=0x7f41d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41d406c8f0 0x7f41d406edb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41f40cd4d0 tx=0x7f41e00079b0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 msgr2=0x7f41f41e98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f41e98a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f41e800d5f0 tx=0x7f41e800dab0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 shutdown_connections 2024-04-20T10:29:27.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41f40cdaa0 0x7f41f40b2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41f4161a50 0x7f41f40cbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f41d406c8f0 0x7f41d406edb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 --2- 172.21.15.47:0/3827334823 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f41f406c010 0x7f41f41e98a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 >> 172.21.15.47:0/3827334823 conn(0x7f41f40d0ed0 msgr2=0x7f41f40b5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 shutdown_connections 2024-04-20T10:29:27.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:27.944+0000 7f41fa400640 1 -- 172.21.15.47:0/3827334823 wait complete. 2024-04-20T10:29:27.967 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T10:29:27.968 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T10:29:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.177+0000 7f2b00e00640 1 Processor -- start 2024-04-20T10:29:28.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.177+0000 7f2b00e00640 1 -- start start 2024-04-20T10:29:28.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2b00e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc15b310 0x7f2afc0cf750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:28.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2b00e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2afc15f5a0 0x7f2afc0b5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:28.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2b00e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2afc0d0a40 con 0x7f2afc0cfc90 2024-04-20T10:29:28.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2b00e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2afc15c700 con 0x7f2afc15f5a0 2024-04-20T10:29:28.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2b00e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc0d01a0 con 0x7f2afc15b310 2024-04-20T10:29:28.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2afbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2afc15f5a0 0x7f2afc0b5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:28.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2afbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2afc15f5a0 0x7f2afc0b5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34638/0 (socket says 172.21.15.47:34638) 2024-04-20T10:29:28.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.179+0000 7f2afbe00640 1 -- 172.21.15.47:0/3540449069 learned_addr learned my addr 172.21.15.47:0/3540449069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:28.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.180+0000 7f2afbe00640 1 -- 172.21.15.47:0/3540449069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc15b310 msgr2=0x7f2afc0cf750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:28.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.180+0000 7f2afbe00640 1 --2- 172.21.15.47:0/3540449069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc15b310 0x7f2afc0cf750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.180+0000 7f2afbe00640 1 -- 172.21.15.47:0/3540449069 >> v1:172.21.15.175:6789/0 conn(0x7f2afc0cfc90 legacy=0x7f2afc15ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.180+0000 7f2afbe00640 1 -- 172.21.15.47:0/3540449069 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc0d0480 con 0x7f2afc15f5a0 2024-04-20T10:29:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.181+0000 7f2afbe00640 1 --2- 172.21.15.47:0/3540449069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15f5a0 0x7f2afc0b5800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0d1050 tx=0x7f2af402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3de879dfdbc6713 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.181+0000 7f2afa000640 1 -- 172.21.15.47:0/3540449069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af402ca50 con 0x7f2afc15f5a0 2024-04-20T10:29:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.181+0000 7f2afa000640 1 -- 172.21.15.47:0/3540449069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2af402cbf0 con 0x7f2afc15f5a0 2024-04-20T10:29:28.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.182+0000 7f2afa000640 1 -- 172.21.15.47:0/3540449069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af40343f0 con 0x7f2afc15f5a0 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.182+0000 7f2b00e00640 1 -- 172.21.15.47:0/3540449069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15f5a0 msgr2=0x7f2afc0b5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.182+0000 7f2b00e00640 1 --2- 172.21.15.47:0/3540449069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15f5a0 0x7f2afc0b5800 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f2afc0d1050 tx=0x7f2af402beb0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.182+0000 7f2b00e00640 1 -- 172.21.15.47:0/3540449069 shutdown_connections 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.182+0000 7f2b00e00640 1 --2- 172.21.15.47:0/3540449069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15f5a0 0x7f2afc0b5800 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.182+0000 7f2b00e00640 1 --2- 172.21.15.47:0/3540449069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc15b310 0x7f2afc0cf750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.182+0000 7f2b00e00640 1 -- 172.21.15.47:0/3540449069 >> 172.21.15.47:0/3540449069 conn(0x7f2afc070090 msgr2=0x7f2afc0704a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.183+0000 7f2b00e00640 1 -- 172.21.15.47:0/3540449069 shutdown_connections 2024-04-20T10:29:28.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.183+0000 7f2b00e00640 1 -- 172.21.15.47:0/3540449069 wait complete. 2024-04-20T10:29:28.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.185+0000 7f2b00e00640 1 Processor -- start 2024-04-20T10:29:28.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.186+0000 7f2b00e00640 1 -- start start 2024-04-20T10:29:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.187+0000 7f2b00e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0cfc90 0x7f2afc157680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.187+0000 7f2b00e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15b310 0x7f2afc1e6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.187+0000 7f2b00e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc15f5a0 0x7f2afc1e7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.187+0000 7f2b00e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc15fba0 con 0x7f2afc15b310 2024-04-20T10:29:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.187+0000 7f2b00e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc0b52e0 con 0x7f2afc15f5a0 2024-04-20T10:29:28.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.187+0000 7f2b00e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2afc15fd20 con 0x7f2afc0cfc90 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15b310 0x7f2afc1e6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15b310 0x7f2afc1e6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34650/0 (socket says 172.21.15.47:34650) 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 -- 172.21.15.47:0/2593329549 learned_addr learned my addr 172.21.15.47:0/2593329549 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afb400640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0cfc90 0x7f2afc157680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afbe00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc15f5a0 0x7f2afc1e7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 -- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0cfc90 msgr2=0x7f2afc157680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0cfc90 0x7f2afc157680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 -- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc15f5a0 msgr2=0x7f2afc1e7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc15f5a0 0x7f2afc1e7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 -- 172.21.15.47:0/2593329549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc1f6130 con 0x7f2afc15b310 2024-04-20T10:29:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.188+0000 7f2afaa00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15b310 0x7f2afc1e6df0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c400 tx=0x7f2ae800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.189+0000 7f2af3400640 1 -- 172.21.15.47:0/2593329549 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8011790 con 0x7f2afc15b310 2024-04-20T10:29:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.189+0000 7f2af3400640 1 -- 172.21.15.47:0/2593329549 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ae8017070 con 0x7f2afc15b310 2024-04-20T10:29:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.189+0000 7f2af3400640 1 -- 172.21.15.47:0/2593329549 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8011bb0 con 0x7f2afc15b310 2024-04-20T10:29:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.189+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc1f6410 con 0x7f2afc15b310 2024-04-20T10:29:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.189+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc1f69d0 con 0x7f2afc15b310 2024-04-20T10:29:28.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.190+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc075b30 con 0x7f2afc15b310 2024-04-20T10:29:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.191+0000 7f2af3400640 1 -- 172.21.15.47:0/2593329549 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ae801c460 con 0x7f2afc15b310 2024-04-20T10:29:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.192+0000 7f2af3400640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2adc06c8f0 0x7f2adc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:28.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.192+0000 7f2af3400640 1 -- 172.21.15.47:0/2593329549 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f2ae808bc20 con 0x7f2afc15b310 2024-04-20T10:29:28.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.193+0000 7f2afb400640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2adc06c8f0 0x7f2adc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:28.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.194+0000 7f2afb400640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2adc06c8f0 0x7f2adc06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009890 tx=0x7f2aec008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:28.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.197+0000 7f2af3400640 1 -- 172.21.15.47:0/2593329549 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ae8059e20 con 0x7f2afc15b310 2024-04-20T10:29:28.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.438+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f2afc1f6ba0 con 0x7f2afc15b310 2024-04-20T10:29:28.497 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:28.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:28.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.943+0000 7f2af3400640 1 -- 172.21.15.47:0/2593329549 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v19) ==== 122+0+0 (secure 0 0 0) 0x7f2ae8059720 con 0x7f2afc15b310 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.946+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2adc06c8f0 msgr2=0x7f2adc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.946+0000 7f2b00e00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2adc06c8f0 0x7f2adc06edb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009890 tx=0x7f2aec008040 comp rx=0 tx=0).stop 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.946+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15b310 msgr2=0x7f2afc1e6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.946+0000 7f2b00e00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15b310 0x7f2afc1e6df0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c400 tx=0x7f2ae800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 shutdown_connections 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2afc15f5a0 0x7f2afc1e7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2adc06c8f0 0x7f2adc06edb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2afc15b310 0x7f2afc1e6df0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 --2- 172.21.15.47:0/2593329549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2afc0cfc90 0x7f2afc157680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 >> 172.21.15.47:0/2593329549 conn(0x7f2afc070090 msgr2=0x7f2afc0b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 shutdown_connections 2024-04-20T10:29:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:28.947+0000 7f2b00e00640 1 -- 172.21.15.47:0/2593329549 wait complete. 2024-04-20T10:29:28.966 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T10:29:28.967 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:29.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.142+0000 7ff0b1400640 1 Processor -- start 2024-04-20T10:29:29.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.142+0000 7ff0b1400640 1 -- start start 2024-04-20T10:29:29.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.143+0000 7ff0b1400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0ac161d50 0x7ff0ac0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.143+0000 7ff0b1400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ac06c190 con 0x7ff0ac06cde0 2024-04-20T10:29:29.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.143+0000 7ff0b1400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff0ac06c470 con 0x7ff0ac0b3330 2024-04-20T10:29:29.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.143+0000 7ff0b1400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7ff0ac162130 con 0x7ff0ac161d50 2024-04-20T10:29:29.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.144+0000 7ff0abe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7ff0ac161d50 0x7ff0ac0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.144+0000 7ff0b0a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff0ac06cde0 0x7ff0ac0b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36612/0 (socket says 172.21.15.47:36612) 2024-04-20T10:29:29.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.144+0000 7ff0b0a00640 1 -- 172.21.15.47:0/308237208 learned_addr learned my addr 172.21.15.47:0/308237208 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:29.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.144+0000 7ff0abe00640 1 -- 172.21.15.47:0/308237208 >> v1:172.21.15.175:6789/0 conn(0x7ff0ac0b3330 legacy=0x7ff0ac06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:29.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.144+0000 7ff0abe00640 1 -- 172.21.15.47:0/308237208 >> v1:172.21.15.47:6789/0 conn(0x7ff0ac06cde0 legacy=0x7ff0ac0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.144+0000 7ff0abe00640 1 -- 172.21.15.47:0/308237208 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac0d2e00 con 0x7ff0ac161d50 2024-04-20T10:29:29.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.144+0000 7ff0aaa00640 1 -- 172.21.15.47:0/308237208 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1530537885 0 0) 0x7ff0ac06c190 con 0x7ff0ac06cde0 2024-04-20T10:29:29.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.145+0000 7ff0abe00640 1 --2- 172.21.15.47:0/308237208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac161d50 0x7ff0ac0d38e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40097a0 tx=0x7ff0a402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483d44273b96f8f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:29.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0aaa00640 1 -- 172.21.15.47:0/308237208 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4035020 con 0x7ff0ac161d50 2024-04-20T10:29:29.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0aaa00640 1 -- 172.21.15.47:0/308237208 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff0a4031830 con 0x7ff0ac161d50 2024-04-20T10:29:29.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0aaa00640 1 -- 172.21.15.47:0/308237208 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0a4031c00 con 0x7ff0ac161d50 2024-04-20T10:29:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0b1400640 1 -- 172.21.15.47:0/308237208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac161d50 msgr2=0x7ff0ac0d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0b1400640 1 --2- 172.21.15.47:0/308237208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac161d50 0x7ff0ac0d38e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff0a40097a0 tx=0x7ff0a402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0b1400640 1 -- 172.21.15.47:0/308237208 shutdown_connections 2024-04-20T10:29:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0b1400640 1 --2- 172.21.15.47:0/308237208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac161d50 0x7ff0ac0d38e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0b1400640 1 -- 172.21.15.47:0/308237208 >> 172.21.15.47:0/308237208 conn(0x7ff0ac0d4450 msgr2=0x7ff0ac0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0b1400640 1 -- 172.21.15.47:0/308237208 shutdown_connections 2024-04-20T10:29:29.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.146+0000 7ff0b1400640 1 -- 172.21.15.47:0/308237208 wait complete. 2024-04-20T10:29:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.148+0000 7ff0b1400640 1 Processor -- start 2024-04-20T10:29:29.148 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.148+0000 7ff0b1400640 1 -- start start 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0b1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac06cde0 0x7ff0ac14f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0b1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac0b3330 0x7ff0ac15d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0b1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0ac161d50 0x7ff0ac15d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0b1400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0b3880 con 0x7ff0ac0b3330 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0b1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0ac0d33c0 con 0x7ff0ac06cde0 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0b1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff0ac0b3a00 con 0x7ff0ac161d50 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0ab400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac0b3330 0x7ff0ac15d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0ab400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac0b3330 0x7ff0ac15d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34674/0 (socket says 172.21.15.47:34674) 2024-04-20T10:29:29.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.149+0000 7ff0ab400640 1 -- 172.21.15.47:0/2759222208 learned_addr learned my addr 172.21.15.47:0/2759222208 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0abe00640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac06cde0 0x7ff0ac14f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0b0a00640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0ac161d50 0x7ff0ac15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0ab400640 1 -- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0ac161d50 msgr2=0x7ff0ac15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0ab400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0ac161d50 0x7ff0ac15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0ab400640 1 -- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac06cde0 msgr2=0x7ff0ac14f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0ab400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac06cde0 0x7ff0ac14f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0ab400640 1 -- 172.21.15.47:0/2759222208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0ac1f4680 con 0x7ff0ac0b3330 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0b0a00640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0ac161d50 0x7ff0ac15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0abe00640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac06cde0 0x7ff0ac14f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0ab400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac0b3330 0x7ff0ac15d1e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c430 tx=0x7ff09400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0a3e00640 1 -- 172.21.15.47:0/2759222208 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094011790 con 0x7ff0ac0b3330 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0a3e00640 1 -- 172.21.15.47:0/2759222208 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff094017070 con 0x7ff0ac0b3330 2024-04-20T10:29:29.152 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.150+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0ac1f4920 con 0x7ff0ac0b3330 2024-04-20T10:29:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.151+0000 7ff0a3e00640 1 -- 172.21.15.47:0/2759222208 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff094011bb0 con 0x7ff0ac0b3330 2024-04-20T10:29:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.151+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0ac1f4ee0 con 0x7ff0ac0b3330 2024-04-20T10:29:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.152+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0ac075b60 con 0x7ff0ac0b3330 2024-04-20T10:29:29.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.153+0000 7ff0a3e00640 1 -- 172.21.15.47:0/2759222208 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff09401c460 con 0x7ff0ac0b3330 2024-04-20T10:29:29.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.154+0000 7ff0a3e00640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.154 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.154+0000 7ff0a3e00640 1 -- 172.21.15.47:0/2759222208 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7ff09408bca0 con 0x7ff0ac0b3330 2024-04-20T10:29:29.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.155+0000 7ff0abe00640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.156+0000 7ff0abe00640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff08806c8f0 0x7ff08806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4036000 tx=0x7ff0a4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:29.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.159+0000 7ff0a3e00640 1 -- 172.21.15.47:0/2759222208 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff094059ea0 con 0x7ff0ac0b3330 2024-04-20T10:29:29.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.397+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0ac1f50b0 con 0x7ff0ac0b3330 2024-04-20T10:29:29.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.398+0000 7ff0a3e00640 1 -- 172.21.15.47:0/2759222208 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5205 (secure 0 0 0) 0x7ff0940597a0 con 0x7ff0ac0b3330 2024-04-20T10:29:29.398 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:29.399 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:28.440368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/1013167846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1013167846},{"type":"v1","addr":"172.21.15.47:6835","nonce":1013167846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5481,"qdb_cluster":[5481]},"id":2}]} 2024-04-20T10:29:29.399 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-20T10:29:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff08806c8f0 msgr2=0x7ff08806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff08806c8f0 0x7ff08806edb0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7ff0a4036000 tx=0x7ff0a4008030 comp rx=0 tx=0).stop 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac0b3330 msgr2=0x7ff0ac15d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac0b3330 0x7ff0ac15d1e0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7ff09400c430 tx=0x7ff09400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 shutdown_connections 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0ac161d50 0x7ff0ac15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff08806c8f0 0x7ff08806edb0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff0ac0b3330 0x7ff0ac15d1e0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 --2- 172.21.15.47:0/2759222208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff0ac06cde0 0x7ff0ac14f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 >> 172.21.15.47:0/2759222208 conn(0x7ff0ac0d4450 msgr2=0x7ff0ac160a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 shutdown_connections 2024-04-20T10:29:29.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.401+0000 7ff0b1400640 1 -- 172.21.15.47:0/2759222208 wait complete. 2024-04-20T10:29:29.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:29.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.609+0000 7f8445c00640 1 Processor -- start 2024-04-20T10:29:29.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.609+0000 7f8445c00640 1 -- start start 2024-04-20T10:29:29.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.610+0000 7f8445c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8440161d50 0x7f84400d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.610+0000 7f8445c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.610+0000 7f8445c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f844006c190 con 0x7f84400b3330 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.610+0000 7f8445c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8440162130 con 0x7f8440161d50 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.610+0000 7f8445c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f844006c470 con 0x7f844006cde0 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f843fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8440161d50 0x7f84400d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f843fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8440161d50 0x7f84400d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34682/0 (socket says 172.21.15.47:34682) 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f843fe00640 1 -- 172.21.15.47:0/1644332856 learned_addr learned my addr 172.21.15.47:0/1644332856 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f8445200640 1 --2- 172.21.15.47:0/1644332856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f843fe00640 1 -- 172.21.15.47:0/1644332856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 msgr2=0x7f84400b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f843fe00640 1 --2- 172.21.15.47:0/1644332856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f843fe00640 1 -- 172.21.15.47:0/1644332856 >> v1:172.21.15.175:6789/0 conn(0x7f84400b3330 legacy=0x7f844006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:29.612 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.611+0000 7f843fe00640 1 -- 172.21.15.47:0/1644332856 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84400d2e00 con 0x7f8440161d50 2024-04-20T10:29:29.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.612+0000 7f843fe00640 1 --2- 172.21.15.47:0/1644332856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8440161d50 0x7f84400d38e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f84400b6f50 tx=0x7f843002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ef82d2b72dcbfae server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:29.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.613+0000 7f843f400640 1 -- 172.21.15.47:0/1644332856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430035020 con 0x7f8440161d50 2024-04-20T10:29:29.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.613+0000 7f843f400640 1 -- 172.21.15.47:0/1644332856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8430031830 con 0x7f8440161d50 2024-04-20T10:29:29.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.613+0000 7f843f400640 1 -- 172.21.15.47:0/1644332856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430031c00 con 0x7f8440161d50 2024-04-20T10:29:29.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.613+0000 7f8445c00640 1 -- 172.21.15.47:0/1644332856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8440161d50 msgr2=0x7f84400d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.613+0000 7f8445c00640 1 --2- 172.21.15.47:0/1644332856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8440161d50 0x7f84400d38e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f84400b6f50 tx=0x7f843002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.614+0000 7f8445c00640 1 -- 172.21.15.47:0/1644332856 shutdown_connections 2024-04-20T10:29:29.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.614+0000 7f8445c00640 1 --2- 172.21.15.47:0/1644332856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.614+0000 7f8445c00640 1 --2- 172.21.15.47:0/1644332856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8440161d50 0x7f84400d38e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.614+0000 7f8445c00640 1 -- 172.21.15.47:0/1644332856 >> 172.21.15.47:0/1644332856 conn(0x7f84400d4450 msgr2=0x7f84400d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:29.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.614+0000 7f8445c00640 1 -- 172.21.15.47:0/1644332856 shutdown_connections 2024-04-20T10:29:29.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.614+0000 7f8445c00640 1 -- 172.21.15.47:0/1644332856 wait complete. 2024-04-20T10:29:29.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.615+0000 7f8445c00640 1 Processor -- start 2024-04-20T10:29:29.616 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.616+0000 7f8445c00640 1 -- start start 2024-04-20T10:29:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.616+0000 7f8445c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400cde50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.616+0000 7f8445c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84400b3330 0x7f84401f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.616+0000 7f8445c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8440161d50 0x7f84401f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8445c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f84400b7660 con 0x7f84400b3330 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8445c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f84400d33c0 con 0x7f8440161d50 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8445c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f84400b77e0 con 0x7f844006cde0 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84400b3330 0x7f84401f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84400b3330 0x7f84401f7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34696/0 (socket says 172.21.15.47:34696) 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 -- 172.21.15.47:0/2404493080 learned_addr learned my addr 172.21.15.47:0/2404493080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f843fe00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8445200640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8440161d50 0x7f84401f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 -- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 msgr2=0x7f84400cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 -- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8440161d50 msgr2=0x7f84401f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8440161d50 0x7f84401f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f8437e00640 1 -- 172.21.15.47:0/2404493080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84401f4660 con 0x7f84400b3330 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.617+0000 7f843fe00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.618+0000 7f8437e00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84400b3330 0x7f84401f7d60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f842800c460 tx=0x7f842800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.618+0000 7f843cc00640 1 -- 172.21.15.47:0/2404493080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428016020 con 0x7f84400b3330 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.618+0000 7f843cc00640 1 -- 172.21.15.47:0/2404493080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8428012870 con 0x7f84400b3330 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.618+0000 7f843cc00640 1 -- 172.21.15.47:0/2404493080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8428012c60 con 0x7f84400b3330 2024-04-20T10:29:29.620 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.618+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84400b3d50 con 0x7f84400b3330 2024-04-20T10:29:29.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.618+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84400b4310 con 0x7f84400b3330 2024-04-20T10:29:29.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.619+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8440075b60 con 0x7f84400b3330 2024-04-20T10:29:29.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.620+0000 7f843cc00640 1 -- 172.21.15.47:0/2404493080 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8428019020 con 0x7f84400b3330 2024-04-20T10:29:29.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.621+0000 7f843cc00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f841c06c920 0x7f841c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:29.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.621+0000 7f843cc00640 1 -- 172.21.15.47:0/2404493080 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f842808b770 con 0x7f84400b3330 2024-04-20T10:29:29.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.622+0000 7f843fe00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f841c06c920 0x7f841c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:29.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.623+0000 7f843fe00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f841c06c920 0x7f841c06ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8430036000 tx=0x7f843003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:29.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.626+0000 7f843cc00640 1 -- 172.21.15.47:0/2404493080 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8428020070 con 0x7f84400b3330 2024-04-20T10:29:29.835 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.835+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f84400d4a80 con 0x7f84400b3330 2024-04-20T10:29:29.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.838+0000 7f843cc00640 1 -- 172.21.15.47:0/2404493080 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16954 (secure 0 0 0) 0x7f8428059a00 con 0x7f84400b3330 2024-04-20T10:29:29.838 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:29.839 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:26.904749+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.840+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f841c06c920 msgr2=0x7f841c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.840+0000 7f8445c00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f841c06c920 0x7f841c06ede0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8430036000 tx=0x7f843003c040 comp rx=0 tx=0).stop 2024-04-20T10:29:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.840+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84400b3330 msgr2=0x7f84401f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84400b3330 0x7f84401f7d60 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f842800c460 tx=0x7f842800c920 comp rx=0 tx=0).stop 2024-04-20T10:29:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 shutdown_connections 2024-04-20T10:29:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8440161d50 0x7f84401f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f841c06c920 0x7f841c06ede0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f84400b3330 0x7f84401f7d60 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 --2- 172.21.15.47:0/2404493080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f844006cde0 0x7f84400cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 >> 172.21.15.47:0/2404493080 conn(0x7f84400d4450 msgr2=0x7f84400d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 shutdown_connections 2024-04-20T10:29:29.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:29.841+0000 7f8445c00640 1 -- 172.21.15.47:0/2404493080 wait complete. 2024-04-20T10:29:29.859 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T10:29:29.859 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:29:30.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.067+0000 7f985fe00640 1 Processor -- start 2024-04-20T10:29:30.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.067+0000 7f985fe00640 1 -- start start 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.069+0000 7f985fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9860072190 0x7f98600735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.069+0000 7f985fe00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986015cc90 con 0x7f986015a1f0 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.069+0000 7f985fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f986015cf70 con 0x7f9860159e10 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.069+0000 7f985fe00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9860072900 con 0x7f9860072190 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.069+0000 7f985f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9860072190 0x7f98600735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.069+0000 7f985e000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f986015a1f0 0x7f9860072ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36628/0 (socket says 172.21.15.47:36628) 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.069+0000 7f985e000640 1 -- 172.21.15.47:0/2653877941 learned_addr learned my addr 172.21.15.47:0/2653877941 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.070+0000 7f985f400640 1 -- 172.21.15.47:0/2653877941 >> v1:172.21.15.175:6789/0 conn(0x7f9860159e10 legacy=0x7f9860071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.070+0000 7f985f400640 1 -- 172.21.15.47:0/2653877941 >> v1:172.21.15.47:6789/0 conn(0x7f986015a1f0 legacy=0x7f9860072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.070+0000 7f985f400640 1 -- 172.21.15.47:0/2653877941 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986015d250 con 0x7f9860072190 2024-04-20T10:29:30.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.070+0000 7f985d600640 1 -- 172.21.15.47:0/2653877941 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2342100335 0 0) 0x7f986015cc90 con 0x7f986015a1f0 2024-04-20T10:29:30.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.071+0000 7f985f400640 1 --2- 172.21.15.47:0/2653877941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860072190 0x7f98600735f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9858009d90 tx=0x7f985802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16de04243884209 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:30.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.071+0000 7f985d600640 1 -- 172.21.15.47:0/2653877941 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858039070 con 0x7f9860072190 2024-04-20T10:29:30.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.071+0000 7f985d600640 1 -- 172.21.15.47:0/2653877941 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985802cc60 con 0x7f9860072190 2024-04-20T10:29:30.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.071+0000 7f985d600640 1 -- 172.21.15.47:0/2653877941 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858034660 con 0x7f9860072190 2024-04-20T10:29:30.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.072+0000 7f985fe00640 1 -- 172.21.15.47:0/2653877941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860072190 msgr2=0x7f98600735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.072+0000 7f985fe00640 1 --2- 172.21.15.47:0/2653877941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860072190 0x7f98600735f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9858009d90 tx=0x7f985802beb0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.072+0000 7f985fe00640 1 -- 172.21.15.47:0/2653877941 shutdown_connections 2024-04-20T10:29:30.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.072+0000 7f985fe00640 1 --2- 172.21.15.47:0/2653877941 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860072190 0x7f98600735f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.072+0000 7f985fe00640 1 -- 172.21.15.47:0/2653877941 >> 172.21.15.47:0/2653877941 conn(0x7f98600d0040 msgr2=0x7f98600d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:30.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.072+0000 7f985fe00640 1 -- 172.21.15.47:0/2653877941 shutdown_connections 2024-04-20T10:29:30.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.072+0000 7f985fe00640 1 -- 172.21.15.47:0/2653877941 wait complete. 2024-04-20T10:29:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.074+0000 7f985fe00640 1 Processor -- start 2024-04-20T10:29:30.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.074+0000 7f985fe00640 1 -- start start 2024-04-20T10:29:30.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9860072190 0x7f986015da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860159e10 0x7f986014ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986015a1f0 0x7f986015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f986015d810 con 0x7f9860159e10 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f986015ca40 con 0x7f9860072190 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9860073b30 con 0x7f986015a1f0 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860159e10 0x7f986014ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860159e10 0x7f986014ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34714/0 (socket says 172.21.15.47:34714) 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985e000640 1 -- 172.21.15.47:0/1281885603 learned_addr learned my addr 172.21.15.47:0/1281885603 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985e000640 1 -- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986015a1f0 msgr2=0x7f986015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.075+0000 7f985e000640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986015a1f0 0x7f986015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f985e000640 1 -- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9860072190 msgr2=0x7f986015da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f985e000640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9860072190 0x7f986015da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f985e000640 1 -- 172.21.15.47:0/1281885603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986015fce0 con 0x7f9860159e10 2024-04-20T10:29:30.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f985e000640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860159e10 0x7f986014ab70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c440 tx=0x7f984c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f9857400640 1 -- 172.21.15.47:0/1281885603 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c0074e0 con 0x7f9860159e10 2024-04-20T10:29:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f9857400640 1 -- 172.21.15.47:0/1281885603 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c0047e0 con 0x7f9860159e10 2024-04-20T10:29:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f9857400640 1 -- 172.21.15.47:0/1281885603 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c004bd0 con 0x7f9860159e10 2024-04-20T10:29:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986015e030 con 0x7f9860159e10 2024-04-20T10:29:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.076+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98600ca080 con 0x7f9860159e10 2024-04-20T10:29:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.078+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9860075b60 con 0x7f9860159e10 2024-04-20T10:29:30.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.078+0000 7f9857400640 1 -- 172.21.15.47:0/1281885603 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f984c011070 con 0x7f9860159e10 2024-04-20T10:29:30.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.079+0000 7f9857400640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c8f0 0x7f984006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.079+0000 7f9857400640 1 -- 172.21.15.47:0/1281885603 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f984c08b8e0 con 0x7f9860159e10 2024-04-20T10:29:30.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.080+0000 7f985ea00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c8f0 0x7f984006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.081+0000 7f985ea00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c8f0 0x7f984006edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9850003cc0 tx=0x7f98500079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:30.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.084+0000 7f9857400640 1 -- 172.21.15.47:0/1281885603 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f984c059b70 con 0x7f9860159e10 2024-04-20T10:29:30.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.284+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f986006bb50 con 0x7f984006c8f0 2024-04-20T10:29:30.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.304+0000 7f9857400640 1 -- 172.21.15.47:0/1281885603 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245403 (secure 0 0 0) 0x7f986006bb50 con 0x7f984006c8f0 2024-04-20T10:29:30.306 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:30.315 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c8f0 msgr2=0x7f984006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c8f0 0x7f984006edb0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9850003cc0 tx=0x7f98500079b0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860159e10 msgr2=0x7f986014ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860159e10 0x7f986014ab70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f984c00c440 tx=0x7f984c00c900 comp rx=0 tx=0).stop 2024-04-20T10:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 shutdown_connections 2024-04-20T10:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986015a1f0 0x7f986015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c8f0 0x7f984006edb0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9860159e10 0x7f986014ab70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 --2- 172.21.15.47:0/1281885603 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9860072190 0x7f986015da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 >> 172.21.15.47:0/1281885603 conn(0x7f98600d0040 msgr2=0x7f986006cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 shutdown_connections 2024-04-20T10:29:30.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.311+0000 7f985fe00640 1 -- 172.21.15.47:0/1281885603 wait complete. 2024-04-20T10:29:30.331 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-20T10:29:29.108233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237416,"kb_used_data":4616,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737304,"statfs":{"total":773094113280,"available":772850999296,"internally_reserved":0,"allocated":4726784,"data_stored":2659210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.246888"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.876319+0000","last_change":"2024-04-20T10:29:24.876319+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.876319+0000","last_undegraded":"2024-04-20T10:29:24.876319+0000","last_fullsized":"2024-04-20T10:29:24.876319+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.891026+0000","last_change":"2024-04-20T10:29:24.891026+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.891026+0000","last_undegraded":"2024-04-20T10:29:24.891026+0000","last_fullsized":"2024-04-20T10:29:24.891026+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.875150+0000","last_change":"2024-04-20T10:29:24.875150+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.875150+0000","last_undegraded":"2024-04-20T10:29:24.875150+0000","last_fullsized":"2024-04-20T10:29:24.875150+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.872026+0000","last_change":"2024-04-20T10:29:24.872026+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.872026+0000","last_undegraded":"2024-04-20T10:29:24.872026+0000","last_fullsized":"2024-04-20T10:29:24.872026+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.890571+0000","last_change":"2024-04-20T10:29:24.890571+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.890571+0000","last_undegraded":"2024-04-20T10:29:24.890571+0000","last_fullsized":"2024-04-20T10:29:24.890571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.875344+0000","last_change":"2024-04-20T10:29:24.875344+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.875344+0000","last_undegraded":"2024-04-20T10:29:24.875344+0000","last_fullsized":"2024-04-20T10:29:24.875344+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.882398+0000","last_change":"2024-04-20T10:29:24.882398+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.882398+0000","last_undegraded":"2024-04-20T10:29:24.882398+0000","last_fullsized":"2024-04-20T10:29:24.882398+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.893280+0000","last_change":"2024-04-20T10:29:24.893280+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.893280+0000","last_undegraded":"2024-04-20T10:29:24.893280+0000","last_fullsized":"2024-04-20T10:29:24.893280+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.873641+0000","last_change":"2024-04-20T10:29:24.873641+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.873641+0000","last_undegraded":"2024-04-20T10:29:24.873641+0000","last_fullsized":"2024-04-20T10:29:24.873641+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880255+0000","last_change":"2024-04-20T10:29:24.880255+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880255+0000","last_undegraded":"2024-04-20T10:29:24.880255+0000","last_fullsized":"2024-04-20T10:29:24.880255+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.888129+0000","last_change":"2024-04-20T10:29:24.888129+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.888129+0000","last_undegraded":"2024-04-20T10:29:24.888129+0000","last_fullsized":"2024-04-20T10:29:24.888129+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.873062+0000","last_change":"2024-04-20T10:29:24.873062+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.873062+0000","last_undegraded":"2024-04-20T10:29:24.873062+0000","last_fullsized":"2024-04-20T10:29:24.873062+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.888078+0000","last_change":"2024-04-20T10:29:24.888078+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.888078+0000","last_undegraded":"2024-04-20T10:29:24.888078+0000","last_fullsized":"2024-04-20T10:29:24.888078+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.885480+0000","last_change":"2024-04-20T10:29:24.885480+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.885480+0000","last_undegraded":"2024-04-20T10:29:24.885480+0000","last_fullsized":"2024-04-20T10:29:24.885480+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.891109+0000","last_change":"2024-04-20T10:29:24.891109+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.891109+0000","last_undegraded":"2024-04-20T10:29:24.891109+0000","last_fullsized":"2024-04-20T10:29:24.891109+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880217+0000","last_change":"2024-04-20T10:29:24.880217+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880217+0000","last_undegraded":"2024-04-20T10:29:24.880217+0000","last_fullsized":"2024-04-20T10:29:24.880217+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879298+0000","last_change":"2024-04-20T10:29:24.879298+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879298+0000","last_undegraded":"2024-04-20T10:29:24.879298+0000","last_fullsized":"2024-04-20T10:29:24.879298+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.881134+0000","last_change":"2024-04-20T10:29:24.881134+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.881134+0000","last_undegraded":"2024-04-20T10:29:24.881134+0000","last_fullsized":"2024-04-20T10:29:24.881134+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.884948+0000","last_change":"2024-04-20T10:29:24.884948+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.884948+0000","last_undegraded":"2024-04-20T10:29:24.884948+0000","last_fullsized":"2024-04-20T10:29:24.884948+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.884353+0000","last_change":"2024-04-20T10:29:24.884353+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.884353+0000","last_undegraded":"2024-04-20T10:29:24.884353+0000","last_fullsized":"2024-04-20T10:29:24.884353+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.881905+0000","last_change":"2024-04-20T10:29:24.881905+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.881905+0000","last_undegraded":"2024-04-20T10:29:24.881905+0000","last_fullsized":"2024-04-20T10:29:24.881905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874807+0000","last_change":"2024-04-20T10:29:24.874807+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874807+0000","last_undegraded":"2024-04-20T10:29:24.874807+0000","last_fullsized":"2024-04-20T10:29:24.874807+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.885364+0000","last_change":"2024-04-20T10:29:24.885364+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.885364+0000","last_undegraded":"2024-04-20T10:29:24.885364+0000","last_fullsized":"2024-04-20T10:29:24.885364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.883718+0000","last_change":"2024-04-20T10:29:24.883718+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.883718+0000","last_undegraded":"2024-04-20T10:29:24.883718+0000","last_fullsized":"2024-04-20T10:29:24.883718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.871306+0000","last_change":"2024-04-20T10:29:24.871306+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.871306+0000","last_undegraded":"2024-04-20T10:29:24.871306+0000","last_fullsized":"2024-04-20T10:29:24.871306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880191+0000","last_change":"2024-04-20T10:29:24.880191+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880191+0000","last_undegraded":"2024-04-20T10:29:24.880191+0000","last_fullsized":"2024-04-20T10:29:24.880191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.904385+0000","last_change":"2024-04-20T10:29:24.902599+0000","last_active":"2024-04-20T10:29:24.904385+0000","last_peered":"2024-04-20T10:29:24.904385+0000","last_clean":"2024-04-20T10:29:24.904385+0000","last_became_active":"2024-04-20T10:29:24.902243+0000","last_became_peered":"2024-04-20T10:29:24.902243+0000","last_unstale":"2024-04-20T10:29:24.904385+0000","last_undegraded":"2024-04-20T10:29:24.904385+0000","last_fullsized":"2024-04-20T10:29:24.904385+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:27.152060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874424+0000","last_change":"2024-04-20T10:29:24.874424+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874424+0000","last_undegraded":"2024-04-20T10:29:24.874424+0000","last_fullsized":"2024-04-20T10:29:24.874424+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.900314+0000","last_change":"2024-04-20T10:29:24.895695+0000","last_active":"2024-04-20T10:29:24.900314+0000","last_peered":"2024-04-20T10:29:24.900314+0000","last_clean":"2024-04-20T10:29:24.900314+0000","last_became_active":"2024-04-20T10:29:24.895319+0000","last_became_peered":"2024-04-20T10:29:24.895319+0000","last_unstale":"2024-04-20T10:29:24.900314+0000","last_undegraded":"2024-04-20T10:29:24.900314+0000","last_fullsized":"2024-04-20T10:29:24.900314+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:12.664563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880516+0000","last_change":"2024-04-20T10:29:24.880516+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880516+0000","last_undegraded":"2024-04-20T10:29:24.880516+0000","last_fullsized":"2024-04-20T10:29:24.880516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.867151+0000","last_change":"2024-04-20T10:29:23.867151+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.867151+0000","last_undegraded":"2024-04-20T10:29:23.867151+0000","last_fullsized":"2024-04-20T10:29:23.867151+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874247+0000","last_change":"2024-04-20T10:29:24.874247+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874247+0000","last_undegraded":"2024-04-20T10:29:24.874247+0000","last_fullsized":"2024-04-20T10:29:24.874247+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.909342+0000","last_change":"2024-04-20T10:29:24.908508+0000","last_active":"2024-04-20T10:29:24.909342+0000","last_peered":"2024-04-20T10:29:24.909342+0000","last_clean":"2024-04-20T10:29:24.909342+0000","last_became_active":"2024-04-20T10:29:24.908247+0000","last_became_peered":"2024-04-20T10:29:24.908247+0000","last_unstale":"2024-04-20T10:29:24.909342+0000","last_undegraded":"2024-04-20T10:29:24.909342+0000","last_fullsized":"2024-04-20T10:29:24.909342+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:47.895831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.904345+0000","last_change":"2024-04-20T10:29:24.902051+0000","last_active":"2024-04-20T10:29:24.904345+0000","last_peered":"2024-04-20T10:29:24.904345+0000","last_clean":"2024-04-20T10:29:24.904345+0000","last_became_active":"2024-04-20T10:29:24.899974+0000","last_became_peered":"2024-04-20T10:29:24.899974+0000","last_unstale":"2024-04-20T10:29:24.904345+0000","last_undegraded":"2024-04-20T10:29:24.904345+0000","last_fullsized":"2024-04-20T10:29:24.904345+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:22.781056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879206+0000","last_change":"2024-04-20T10:29:24.879206+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879206+0000","last_undegraded":"2024-04-20T10:29:24.879206+0000","last_fullsized":"2024-04-20T10:29:24.879206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.907797+0000","last_change":"2024-04-20T10:29:24.906546+0000","last_active":"2024-04-20T10:29:24.907797+0000","last_peered":"2024-04-20T10:29:24.907797+0000","last_clean":"2024-04-20T10:29:24.907797+0000","last_became_active":"2024-04-20T10:29:24.906232+0000","last_became_peered":"2024-04-20T10:29:24.906232+0000","last_unstale":"2024-04-20T10:29:24.907797+0000","last_undegraded":"2024-04-20T10:29:24.907797+0000","last_fullsized":"2024-04-20T10:29:24.907797+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:58:20.925379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879048+0000","last_change":"2024-04-20T10:29:24.879048+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879048+0000","last_undegraded":"2024-04-20T10:29:24.879048+0000","last_fullsized":"2024-04-20T10:29:24.879048+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.866791+0000","last_change":"2024-04-20T10:29:23.866791+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.866791+0000","last_undegraded":"2024-04-20T10:29:23.866791+0000","last_fullsized":"2024-04-20T10:29:23.866791+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.877137+0000","last_change":"2024-04-20T10:29:24.877137+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.877137+0000","last_undegraded":"2024-04-20T10:29:24.877137+0000","last_fullsized":"2024-04-20T10:29:24.877137+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.892731+0000","last_change":"2024-04-20T10:29:24.885719+0000","last_active":"2024-04-20T10:29:24.892731+0000","last_peered":"2024-04-20T10:29:24.892731+0000","last_clean":"2024-04-20T10:29:24.892731+0000","last_became_active":"2024-04-20T10:29:24.885350+0000","last_became_peered":"2024-04-20T10:29:24.885350+0000","last_unstale":"2024-04-20T10:29:24.892731+0000","last_undegraded":"2024-04-20T10:29:24.892731+0000","last_fullsized":"2024-04-20T10:29:24.892731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:44.587858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.867312+0000","last_change":"2024-04-20T10:29:23.867312+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.867312+0000","last_undegraded":"2024-04-20T10:29:23.867312+0000","last_fullsized":"2024-04-20T10:29:23.867312+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.883364+0000","last_change":"2024-04-20T10:29:24.883364+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.883364+0000","last_undegraded":"2024-04-20T10:29:24.883364+0000","last_fullsized":"2024-04-20T10:29:24.883364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.894252+0000","last_change":"2024-04-20T10:29:24.891333+0000","last_active":"2024-04-20T10:29:24.894252+0000","last_peered":"2024-04-20T10:29:24.894252+0000","last_clean":"2024-04-20T10:29:24.894252+0000","last_became_active":"2024-04-20T10:29:24.890971+0000","last_became_peered":"2024-04-20T10:29:24.890971+0000","last_unstale":"2024-04-20T10:29:24.894252+0000","last_undegraded":"2024-04-20T10:29:24.894252+0000","last_fullsized":"2024-04-20T10:29:24.894252+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:27.931008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.907457+0000","last_change":"2024-04-20T10:29:24.906234+0000","last_active":"2024-04-20T10:29:24.907457+0000","last_peered":"2024-04-20T10:29:24.907457+0000","last_clean":"2024-04-20T10:29:24.907457+0000","last_became_active":"2024-04-20T10:29:24.905929+0000","last_became_peered":"2024-04-20T10:29:24.905929+0000","last_unstale":"2024-04-20T10:29:24.907457+0000","last_undegraded":"2024-04-20T10:29:24.907457+0000","last_fullsized":"2024-04-20T10:29:24.907457+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:35.896543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874744+0000","last_change":"2024-04-20T10:29:24.874744+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874744+0000","last_undegraded":"2024-04-20T10:29:24.874744+0000","last_fullsized":"2024-04-20T10:29:24.874744+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.866904+0000","last_change":"2024-04-20T10:29:23.866904+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.866904+0000","last_undegraded":"2024-04-20T10:29:23.866904+0000","last_fullsized":"2024-04-20T10:29:23.866904+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.868505+0000","last_change":"2024-04-20T10:29:23.868505+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.868505+0000","last_undegraded":"2024-04-20T10:29:23.868505+0000","last_fullsized":"2024-04-20T10:29:23.868505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878228+0000","last_change":"2024-04-20T10:29:24.878228+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878228+0000","last_undegraded":"2024-04-20T10:29:24.878228+0000","last_fullsized":"2024-04-20T10:29:24.878228+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.898029+0000","last_change":"2024-04-20T10:29:24.893579+0000","last_active":"2024-04-20T10:29:24.898029+0000","last_peered":"2024-04-20T10:29:24.898029+0000","last_clean":"2024-04-20T10:29:24.898029+0000","last_became_active":"2024-04-20T10:29:24.893233+0000","last_became_peered":"2024-04-20T10:29:24.893233+0000","last_unstale":"2024-04-20T10:29:24.898029+0000","last_undegraded":"2024-04-20T10:29:24.898029+0000","last_fullsized":"2024-04-20T10:29:24.898029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:47.895831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879529+0000","last_change":"2024-04-20T10:29:24.879529+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879529+0000","last_undegraded":"2024-04-20T10:29:24.879529+0000","last_fullsized":"2024-04-20T10:29:24.879529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.895311+0000","last_change":"2024-04-20T10:29:24.891939+0000","last_active":"2024-04-20T10:29:24.895311+0000","last_peered":"2024-04-20T10:29:24.895311+0000","last_clean":"2024-04-20T10:29:24.895311+0000","last_became_active":"2024-04-20T10:29:24.891666+0000","last_became_peered":"2024-04-20T10:29:24.891666+0000","last_unstale":"2024-04-20T10:29:24.895311+0000","last_undegraded":"2024-04-20T10:29:24.895311+0000","last_fullsized":"2024-04-20T10:29:24.895311+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:11.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.886729+0000","last_change":"2024-04-20T10:29:24.886729+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.886729+0000","last_undegraded":"2024-04-20T10:29:24.886729+0000","last_fullsized":"2024-04-20T10:29:24.886729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.893680+0000","last_change":"2024-04-20T10:29:24.890425+0000","last_active":"2024-04-20T10:29:24.893680+0000","last_peered":"2024-04-20T10:29:24.893680+0000","last_clean":"2024-04-20T10:29:24.893680+0000","last_became_active":"2024-04-20T10:29:24.890115+0000","last_became_peered":"2024-04-20T10:29:24.890115+0000","last_unstale":"2024-04-20T10:29:24.893680+0000","last_undegraded":"2024-04-20T10:29:24.893680+0000","last_fullsized":"2024-04-20T10:29:24.893680+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:14.922861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874106+0000","last_change":"2024-04-20T10:29:24.874106+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874106+0000","last_undegraded":"2024-04-20T10:29:24.874106+0000","last_fullsized":"2024-04-20T10:29:24.874106+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.893006+0000","last_change":"2024-04-20T10:29:24.886285+0000","last_active":"2024-04-20T10:29:24.893006+0000","last_peered":"2024-04-20T10:29:24.893006+0000","last_clean":"2024-04-20T10:29:24.893006+0000","last_became_active":"2024-04-20T10:29:24.886016+0000","last_became_peered":"2024-04-20T10:29:24.886016+0000","last_unstale":"2024-04-20T10:29:24.893006+0000","last_undegraded":"2024-04-20T10:29:24.893006+0000","last_fullsized":"2024-04-20T10:29:24.893006+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:43.826357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874212+0000","last_change":"2024-04-20T10:29:24.874212+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874212+0000","last_undegraded":"2024-04-20T10:29:24.874212+0000","last_fullsized":"2024-04-20T10:29:24.874212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.908875+0000","last_change":"2024-04-20T10:29:24.907760+0000","last_active":"2024-04-20T10:29:24.908875+0000","last_peered":"2024-04-20T10:29:24.908875+0000","last_clean":"2024-04-20T10:29:24.908875+0000","last_became_active":"2024-04-20T10:29:24.907189+0000","last_became_peered":"2024-04-20T10:29:24.907189+0000","last_unstale":"2024-04-20T10:29:24.908875+0000","last_undegraded":"2024-04-20T10:29:24.908875+0000","last_fullsized":"2024-04-20T10:29:24.908875+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:04.160335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874054+0000","last_change":"2024-04-20T10:29:24.874054+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874054+0000","last_undegraded":"2024-04-20T10:29:24.874054+0000","last_fullsized":"2024-04-20T10:29:24.874054+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.906364+0000","last_change":"2024-04-20T10:29:24.902160+0000","last_active":"2024-04-20T10:29:24.906364+0000","last_peered":"2024-04-20T10:29:24.906364+0000","last_clean":"2024-04-20T10:29:24.906364+0000","last_became_active":"2024-04-20T10:29:24.900595+0000","last_became_peered":"2024-04-20T10:29:24.900595+0000","last_unstale":"2024-04-20T10:29:24.906364+0000","last_undegraded":"2024-04-20T10:29:24.906364+0000","last_fullsized":"2024-04-20T10:29:24.906364+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:01.416574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.887029+0000","last_change":"2024-04-20T10:29:24.887029+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.887029+0000","last_undegraded":"2024-04-20T10:29:24.887029+0000","last_fullsized":"2024-04-20T10:29:24.887029+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.904132+0000","last_change":"2024-04-20T10:29:24.896677+0000","last_active":"2024-04-20T10:29:24.904132+0000","last_peered":"2024-04-20T10:29:24.904132+0000","last_clean":"2024-04-20T10:29:24.904132+0000","last_became_active":"2024-04-20T10:29:24.896023+0000","last_became_peered":"2024-04-20T10:29:24.896023+0000","last_unstale":"2024-04-20T10:29:24.904132+0000","last_undegraded":"2024-04-20T10:29:24.904132+0000","last_fullsized":"2024-04-20T10:29:24.904132+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:50.540035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879132+0000","last_change":"2024-04-20T10:29:24.879132+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879132+0000","last_undegraded":"2024-04-20T10:29:24.879132+0000","last_fullsized":"2024-04-20T10:29:24.879132+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.905295+0000","last_change":"2024-04-20T10:29:24.903532+0000","last_active":"2024-04-20T10:29:24.905295+0000","last_peered":"2024-04-20T10:29:24.905295+0000","last_clean":"2024-04-20T10:29:24.905295+0000","last_became_active":"2024-04-20T10:29:24.902068+0000","last_became_peered":"2024-04-20T10:29:24.902068+0000","last_unstale":"2024-04-20T10:29:24.905295+0000","last_undegraded":"2024-04-20T10:29:24.905295+0000","last_fullsized":"2024-04-20T10:29:24.905295+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:45.052148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.898005+0000","last_change":"2024-04-20T10:29:24.892163+0000","last_active":"2024-04-20T10:29:24.898005+0000","last_peered":"2024-04-20T10:29:24.898005+0000","last_clean":"2024-04-20T10:29:24.898005+0000","last_became_active":"2024-04-20T10:29:24.891419+0000","last_became_peered":"2024-04-20T10:29:24.891419+0000","last_unstale":"2024-04-20T10:29:24.898005+0000","last_undegraded":"2024-04-20T10:29:24.898005+0000","last_fullsized":"2024-04-20T10:29:24.898005+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:27.152060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880682+0000","last_change":"2024-04-20T10:29:24.880682+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880682+0000","last_undegraded":"2024-04-20T10:29:24.880682+0000","last_fullsized":"2024-04-20T10:29:24.880682+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.886815+0000","last_change":"2024-04-20T10:29:24.883849+0000","last_active":"2024-04-20T10:29:24.886815+0000","last_peered":"2024-04-20T10:29:24.886815+0000","last_clean":"2024-04-20T10:29:24.886815+0000","last_became_active":"2024-04-20T10:29:24.883436+0000","last_became_peered":"2024-04-20T10:29:24.883436+0000","last_unstale":"2024-04-20T10:29:24.886815+0000","last_undegraded":"2024-04-20T10:29:24.886815+0000","last_fullsized":"2024-04-20T10:29:24.886815+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:43.911327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.905653+0000","last_change":"2024-04-20T10:29:24.902461+0000","last_active":"2024-04-20T10:29:24.905653+0000","last_peered":"2024-04-20T10:29:24.905653+0000","last_clean":"2024-04-20T10:29:24.905653+0000","last_became_active":"2024-04-20T10:29:24.901505+0000","last_became_peered":"2024-04-20T10:29:24.901505+0000","last_unstale":"2024-04-20T10:29:24.905653+0000","last_undegraded":"2024-04-20T10:29:24.905653+0000","last_fullsized":"2024-04-20T10:29:24.905653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:50.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.886859+0000","last_change":"2024-04-20T10:29:24.886859+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.886859+0000","last_undegraded":"2024-04-20T10:29:24.886859+0000","last_fullsized":"2024-04-20T10:29:24.886859+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.901076+0000","last_change":"2024-04-20T10:29:24.899608+0000","last_active":"2024-04-20T10:29:24.901076+0000","last_peered":"2024-04-20T10:29:24.901076+0000","last_clean":"2024-04-20T10:29:24.901076+0000","last_became_active":"2024-04-20T10:29:24.899329+0000","last_became_peered":"2024-04-20T10:29:24.899329+0000","last_unstale":"2024-04-20T10:29:24.901076+0000","last_undegraded":"2024-04-20T10:29:24.901076+0000","last_fullsized":"2024-04-20T10:29:24.901076+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:09.312548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.881471+0000","last_change":"2024-04-20T10:29:24.881471+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.881471+0000","last_undegraded":"2024-04-20T10:29:24.881471+0000","last_fullsized":"2024-04-20T10:29:24.881471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.906175+0000","last_change":"2024-04-20T10:29:24.901189+0000","last_active":"2024-04-20T10:29:24.906175+0000","last_peered":"2024-04-20T10:29:24.906175+0000","last_clean":"2024-04-20T10:29:24.906175+0000","last_became_active":"2024-04-20T10:29:24.900748+0000","last_became_peered":"2024-04-20T10:29:24.900748+0000","last_unstale":"2024-04-20T10:29:24.906175+0000","last_undegraded":"2024-04-20T10:29:24.906175+0000","last_fullsized":"2024-04-20T10:29:24.906175+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:33.581245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.890653+0000","last_change":"2024-04-20T10:29:24.890653+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.890653+0000","last_undegraded":"2024-04-20T10:29:24.890653+0000","last_fullsized":"2024-04-20T10:29:24.890653+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.910618+0000","last_change":"2024-04-20T10:29:24.907398+0000","last_active":"2024-04-20T10:29:24.910618+0000","last_peered":"2024-04-20T10:29:24.910618+0000","last_clean":"2024-04-20T10:29:24.910618+0000","last_became_active":"2024-04-20T10:29:24.907118+0000","last_became_peered":"2024-04-20T10:29:24.907118+0000","last_unstale":"2024-04-20T10:29:24.910618+0000","last_undegraded":"2024-04-20T10:29:24.910618+0000","last_fullsized":"2024-04-20T10:29:24.910618+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:03.281492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.884473+0000","last_change":"2024-04-20T10:29:24.884473+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.884473+0000","last_undegraded":"2024-04-20T10:29:24.884473+0000","last_fullsized":"2024-04-20T10:29:24.884473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.867282+0000","last_change":"2024-04-20T10:29:23.867282+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.867282+0000","last_undegraded":"2024-04-20T10:29:23.867282+0000","last_fullsized":"2024-04-20T10:29:23.867282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.875165+0000","last_change":"2024-04-20T10:29:24.875165+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.875165+0000","last_undegraded":"2024-04-20T10:29:24.875165+0000","last_fullsized":"2024-04-20T10:29:24.875165+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.903516+0000","last_change":"2024-04-20T10:29:24.900706+0000","last_active":"2024-04-20T10:29:24.903516+0000","last_peered":"2024-04-20T10:29:24.903516+0000","last_clean":"2024-04-20T10:29:24.903516+0000","last_became_active":"2024-04-20T10:29:24.900256+0000","last_became_peered":"2024-04-20T10:29:24.900256+0000","last_unstale":"2024-04-20T10:29:24.903516+0000","last_undegraded":"2024-04-20T10:29:24.903516+0000","last_fullsized":"2024-04-20T10:29:24.903516+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878515+0000","last_change":"2024-04-20T10:29:24.878515+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878515+0000","last_undegraded":"2024-04-20T10:29:24.878515+0000","last_fullsized":"2024-04-20T10:29:24.878515+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.902396+0000","last_change":"2024-04-20T10:29:24.901083+0000","last_active":"2024-04-20T10:29:24.902396+0000","last_peered":"2024-04-20T10:29:24.902396+0000","last_clean":"2024-04-20T10:29:24.902396+0000","last_became_active":"2024-04-20T10:29:24.899838+0000","last_became_peered":"2024-04-20T10:29:24.899838+0000","last_unstale":"2024-04-20T10:29:24.902396+0000","last_undegraded":"2024-04-20T10:29:24.902396+0000","last_fullsized":"2024-04-20T10:29:24.902396+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:29.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878718+0000","last_change":"2024-04-20T10:29:24.878718+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878718+0000","last_undegraded":"2024-04-20T10:29:24.878718+0000","last_fullsized":"2024-04-20T10:29:24.878718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.866844+0000","last_change":"2024-04-20T10:29:23.866844+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.866844+0000","last_undegraded":"2024-04-20T10:29:23.866844+0000","last_fullsized":"2024-04-20T10:29:23.866844+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878611+0000","last_change":"2024-04-20T10:29:24.878611+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878611+0000","last_undegraded":"2024-04-20T10:29:24.878611+0000","last_fullsized":"2024-04-20T10:29:24.878611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.903563+0000","last_change":"2024-04-20T10:29:24.900850+0000","last_active":"2024-04-20T10:29:24.903563+0000","last_peered":"2024-04-20T10:29:24.903563+0000","last_clean":"2024-04-20T10:29:24.903563+0000","last_became_active":"2024-04-20T10:29:24.898940+0000","last_became_peered":"2024-04-20T10:29:24.898940+0000","last_unstale":"2024-04-20T10:29:24.903563+0000","last_undegraded":"2024-04-20T10:29:24.903563+0000","last_fullsized":"2024-04-20T10:29:24.903563+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":204,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.870383+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:29:24.870383+0000","last_peered":"2024-04-20T10:29:24.870383+0000","last_clean":"2024-04-20T10:29:24.870383+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:29:24.870383+0000","last_undegraded":"2024-04-20T10:29:24.870383+0000","last_fullsized":"2024-04-20T10:29:24.870383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":775120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":191419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":775120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":178543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:29:30.335 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:29:30.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.582+0000 7fbaeb200640 1 Processor -- start 2024-04-20T10:29:30.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.582+0000 7fbaeb200640 1 -- start start 2024-04-20T10:29:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.584+0000 7fbaeb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbaec161d50 0x7fbaec0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.584+0000 7fbaeb200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec06c190 con 0x7fbaec0b3330 2024-04-20T10:29:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.584+0000 7fbaeb200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbaec06c470 con 0x7fbaec06cde0 2024-04-20T10:29:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.584+0000 7fbaeb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbaec162130 con 0x7fbaec161d50 2024-04-20T10:29:30.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.584+0000 7fbae3e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fbaec0b3330 0x7fbaec06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36630/0 (socket says 172.21.15.47:36630) 2024-04-20T10:29:30.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.584+0000 7fbae3e00640 1 -- 172.21.15.47:0/4142929194 learned_addr learned my addr 172.21.15.47:0/4142929194 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9e00640 1 --2- 172.21.15.47:0/4142929194 >> v2:172.21.15.175:3300/0 conn(0x7fbaec161d50 0x7fbaec0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3884094603 0 0) 0x7fbaec06c190 con 0x7fbaec0b3330 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fbad0003520 con 0x7fbaec0b3330 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2287319487 0 0) 0x7fbad0003520 con 0x7fbaec0b3330 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 >> v2:172.21.15.175:3300/0 conn(0x7fbaec161d50 msgr2=0x7fbaec0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 --2- 172.21.15.47:0/4142929194 >> v2:172.21.15.175:3300/0 conn(0x7fbaec161d50 0x7fbaec0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 >> v1:172.21.15.175:6789/0 conn(0x7fbaec06cde0 legacy=0x7fbaec0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec0d2e00 con 0x7fbaec0b3330 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fbad4001670 con 0x7fbaec0b3330 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.585+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 >> v1:172.21.15.47:6789/0 conn(0x7fbaec0b3330 legacy=0x7fbaec06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae9400640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbad00039b0 0x7fbad0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae9400640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad0004800 0x7fbad0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae9400640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad00051c0 0x7fbad000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbad000c030 con 0x7fbad00051c0 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbad000c1e0 con 0x7fbad0004800 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbad000c3e0 con 0x7fbad00039b0 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbaea800640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad00051c0 0x7fbad000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae3e00640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad0004800 0x7fbad0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbae9e00640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbad00039b0 0x7fbad0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbaea800640 1 -- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbad00039b0 msgr2=0x7fbad0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbaea800640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbad00039b0 0x7fbad0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbaea800640 1 -- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad0004800 msgr2=0x7fbad0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbaea800640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad0004800 0x7fbad0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.586+0000 7fbaea800640 1 -- 172.21.15.47:0/4142929194 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbad0003520 con 0x7fbad00051c0 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.587+0000 7fbaea800640 1 -- 172.21.15.47:0/4142929194 reap_dead start 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.588+0000 7fbaea800640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad00051c0 0x7fbad000b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbae4001e50 tx=0x7fbad0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f2f318c0a2227f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.588+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae4001a70 con 0x7fbad00051c0 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.588+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbae4003bb0 con 0x7fbad00051c0 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.588+0000 7fbae9400640 1 -- 172.21.15.47:0/4142929194 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbae40050d0 con 0x7fbad00051c0 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.588+0000 7fbaeb200640 1 -- 172.21.15.47:0/4142929194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad00051c0 msgr2=0x7fbad000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.588+0000 7fbaeb200640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad00051c0 0x7fbad000b9c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbae4001e50 tx=0x7fbad0005830 comp rx=0 tx=0).stop 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.589+0000 7fbaeb200640 1 -- 172.21.15.47:0/4142929194 shutdown_connections 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.589+0000 7fbaeb200640 1 --2- 172.21.15.47:0/4142929194 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbad00051c0 0x7fbad000b9c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.589+0000 7fbaeb200640 1 -- 172.21.15.47:0/4142929194 >> 172.21.15.47:0/4142929194 conn(0x7fbaec0d4450 msgr2=0x7fbaec0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:30.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.589+0000 7fbaeb200640 1 -- 172.21.15.47:0/4142929194 shutdown_connections 2024-04-20T10:29:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.589+0000 7fbaeb200640 1 -- 172.21.15.47:0/4142929194 wait complete. 2024-04-20T10:29:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.590+0000 7fbaeb200640 1 Processor -- start 2024-04-20T10:29:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.591+0000 7fbaeb200640 1 -- start start 2024-04-20T10:29:30.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.591+0000 7fbaeb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad00051c0 0x7fbaec0c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.591+0000 7fbaeb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaec0b88e0 0x7fbaec14b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.591+0000 7fbaeb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaec0b8d60 0x7fbaec14bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.591+0000 7fbaeb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0d38e0 con 0x7fbaec0b88e0 2024-04-20T10:29:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.591+0000 7fbaeb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaec0d33c0 con 0x7fbad00051c0 2024-04-20T10:29:30.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.591+0000 7fbaeb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbaec0d3a60 con 0x7fbaec0b8d60 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaec0b88e0 0x7fbaec14b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaec0b88e0 0x7fbaec14b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34718/0 (socket says 172.21.15.47:34718) 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 -- 172.21.15.47:0/2473731028 learned_addr learned my addr 172.21.15.47:0/2473731028 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae9e00640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad00051c0 0x7fbaec0c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbaea800640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaec0b8d60 0x7fbaec14bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 -- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaec0b8d60 msgr2=0x7fbaec14bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaec0b8d60 0x7fbaec14bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 -- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad00051c0 msgr2=0x7fbaec0c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad00051c0 0x7fbaec0c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae3e00640 1 -- 172.21.15.47:0/2473731028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaec14ef90 con 0x7fbaec0b88e0 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.592+0000 7fbae9e00640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad00051c0 0x7fbaec0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.593+0000 7fbae3e00640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaec0b88e0 0x7fbaec14b680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbad400e780 tx=0x7fbad400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:30.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.593+0000 7fbae2000640 1 -- 172.21.15.47:0/2473731028 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad400ee90 con 0x7fbaec0b88e0 2024-04-20T10:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.593+0000 7fbae2000640 1 -- 172.21.15.47:0/2473731028 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbad4018070 con 0x7fbaec0b88e0 2024-04-20T10:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.593+0000 7fbae2000640 1 -- 172.21.15.47:0/2473731028 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad400ca90 con 0x7fbaec0b88e0 2024-04-20T10:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.593+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaec0c8090 con 0x7fbaec0b88e0 2024-04-20T10:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.593+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaec14e340 con 0x7fbaec0b88e0 2024-04-20T10:29:30.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.594+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaec075b60 con 0x7fbaec0b88e0 2024-04-20T10:29:30.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.596+0000 7fbae2000640 1 -- 172.21.15.47:0/2473731028 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbad400cc30 con 0x7fbaec0b88e0 2024-04-20T10:29:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.597+0000 7fbae2000640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbac806ca90 0x7fbac806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.597+0000 7fbae2000640 1 -- 172.21.15.47:0/2473731028 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fbad408bb10 con 0x7fbaec0b88e0 2024-04-20T10:29:30.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.597+0000 7fbae9e00640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbac806ca90 0x7fbac806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:30.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.599+0000 7fbae9e00640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbac806ca90 0x7fbac806ef50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0099e0 tx=0x7fbadc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:30.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.601+0000 7fbae2000640 1 -- 172.21.15.47:0/2473731028 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbad4059de0 con 0x7fbaec0b88e0 2024-04-20T10:29:30.802 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.802+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbaec14f270 con 0x7fbac806ca90 2024-04-20T10:29:30.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.820+0000 7fbae2000640 1 -- 172.21.15.47:0/2473731028 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245403 (secure 0 0 0) 0x7fbaec14f270 con 0x7fbac806ca90 2024-04-20T10:29:30.823 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:30.825 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.826+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbac806ca90 msgr2=0x7fbac806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.826+0000 7fbaeb200640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbac806ca90 0x7fbac806ef50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbadc0099e0 tx=0x7fbadc008040 comp rx=0 tx=0).stop 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.826+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaec0b88e0 msgr2=0x7fbaec14b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.826+0000 7fbaeb200640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaec0b88e0 0x7fbaec14b680 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fbad400e780 tx=0x7fbad400ec40 comp rx=0 tx=0).stop 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.826+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 shutdown_connections 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.826+0000 7fbaeb200640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaec0b8d60 0x7fbaec14bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.827+0000 7fbaeb200640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbac806ca90 0x7fbac806ef50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.827+0000 7fbaeb200640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbaec0b88e0 0x7fbaec14b680 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.827+0000 7fbaeb200640 1 --2- 172.21.15.47:0/2473731028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbad00051c0 0x7fbaec0c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.827+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 >> 172.21.15.47:0/2473731028 conn(0x7fbaec0d4450 msgr2=0x7fbaec15f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.827+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 shutdown_connections 2024-04-20T10:29:30.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:30.827+0000 7fbaeb200640 1 -- 172.21.15.47:0/2473731028 wait complete. 2024-04-20T10:29:30.846 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-04-20T10:29:29.108233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":168,"acting":168,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237416,"kb_used_data":4616,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737304,"statfs":{"total":773094113280,"available":772850999296,"internally_reserved":0,"allocated":4726784,"data_stored":2659210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.246888"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.876319+0000","last_change":"2024-04-20T10:29:24.876319+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.876319+0000","last_undegraded":"2024-04-20T10:29:24.876319+0000","last_fullsized":"2024-04-20T10:29:24.876319+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.891026+0000","last_change":"2024-04-20T10:29:24.891026+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.891026+0000","last_undegraded":"2024-04-20T10:29:24.891026+0000","last_fullsized":"2024-04-20T10:29:24.891026+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.875150+0000","last_change":"2024-04-20T10:29:24.875150+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.875150+0000","last_undegraded":"2024-04-20T10:29:24.875150+0000","last_fullsized":"2024-04-20T10:29:24.875150+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.872026+0000","last_change":"2024-04-20T10:29:24.872026+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.872026+0000","last_undegraded":"2024-04-20T10:29:24.872026+0000","last_fullsized":"2024-04-20T10:29:24.872026+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.890571+0000","last_change":"2024-04-20T10:29:24.890571+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.890571+0000","last_undegraded":"2024-04-20T10:29:24.890571+0000","last_fullsized":"2024-04-20T10:29:24.890571+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.875344+0000","last_change":"2024-04-20T10:29:24.875344+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.875344+0000","last_undegraded":"2024-04-20T10:29:24.875344+0000","last_fullsized":"2024-04-20T10:29:24.875344+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.882398+0000","last_change":"2024-04-20T10:29:24.882398+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.882398+0000","last_undegraded":"2024-04-20T10:29:24.882398+0000","last_fullsized":"2024-04-20T10:29:24.882398+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.893280+0000","last_change":"2024-04-20T10:29:24.893280+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.893280+0000","last_undegraded":"2024-04-20T10:29:24.893280+0000","last_fullsized":"2024-04-20T10:29:24.893280+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.873641+0000","last_change":"2024-04-20T10:29:24.873641+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.873641+0000","last_undegraded":"2024-04-20T10:29:24.873641+0000","last_fullsized":"2024-04-20T10:29:24.873641+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880255+0000","last_change":"2024-04-20T10:29:24.880255+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880255+0000","last_undegraded":"2024-04-20T10:29:24.880255+0000","last_fullsized":"2024-04-20T10:29:24.880255+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.888129+0000","last_change":"2024-04-20T10:29:24.888129+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.888129+0000","last_undegraded":"2024-04-20T10:29:24.888129+0000","last_fullsized":"2024-04-20T10:29:24.888129+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.873062+0000","last_change":"2024-04-20T10:29:24.873062+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.873062+0000","last_undegraded":"2024-04-20T10:29:24.873062+0000","last_fullsized":"2024-04-20T10:29:24.873062+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.888078+0000","last_change":"2024-04-20T10:29:24.888078+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.888078+0000","last_undegraded":"2024-04-20T10:29:24.888078+0000","last_fullsized":"2024-04-20T10:29:24.888078+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.885480+0000","last_change":"2024-04-20T10:29:24.885480+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.885480+0000","last_undegraded":"2024-04-20T10:29:24.885480+0000","last_fullsized":"2024-04-20T10:29:24.885480+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.891109+0000","last_change":"2024-04-20T10:29:24.891109+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.891109+0000","last_undegraded":"2024-04-20T10:29:24.891109+0000","last_fullsized":"2024-04-20T10:29:24.891109+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880217+0000","last_change":"2024-04-20T10:29:24.880217+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880217+0000","last_undegraded":"2024-04-20T10:29:24.880217+0000","last_fullsized":"2024-04-20T10:29:24.880217+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879298+0000","last_change":"2024-04-20T10:29:24.879298+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879298+0000","last_undegraded":"2024-04-20T10:29:24.879298+0000","last_fullsized":"2024-04-20T10:29:24.879298+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.881134+0000","last_change":"2024-04-20T10:29:24.881134+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.881134+0000","last_undegraded":"2024-04-20T10:29:24.881134+0000","last_fullsized":"2024-04-20T10:29:24.881134+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.884948+0000","last_change":"2024-04-20T10:29:24.884948+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.884948+0000","last_undegraded":"2024-04-20T10:29:24.884948+0000","last_fullsized":"2024-04-20T10:29:24.884948+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.884353+0000","last_change":"2024-04-20T10:29:24.884353+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.884353+0000","last_undegraded":"2024-04-20T10:29:24.884353+0000","last_fullsized":"2024-04-20T10:29:24.884353+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.881905+0000","last_change":"2024-04-20T10:29:24.881905+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.881905+0000","last_undegraded":"2024-04-20T10:29:24.881905+0000","last_fullsized":"2024-04-20T10:29:24.881905+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874807+0000","last_change":"2024-04-20T10:29:24.874807+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874807+0000","last_undegraded":"2024-04-20T10:29:24.874807+0000","last_fullsized":"2024-04-20T10:29:24.874807+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.885364+0000","last_change":"2024-04-20T10:29:24.885364+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.885364+0000","last_undegraded":"2024-04-20T10:29:24.885364+0000","last_fullsized":"2024-04-20T10:29:24.885364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.883718+0000","last_change":"2024-04-20T10:29:24.883718+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.883718+0000","last_undegraded":"2024-04-20T10:29:24.883718+0000","last_fullsized":"2024-04-20T10:29:24.883718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.871306+0000","last_change":"2024-04-20T10:29:24.871306+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.871306+0000","last_undegraded":"2024-04-20T10:29:24.871306+0000","last_fullsized":"2024-04-20T10:29:24.871306+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880191+0000","last_change":"2024-04-20T10:29:24.880191+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880191+0000","last_undegraded":"2024-04-20T10:29:24.880191+0000","last_fullsized":"2024-04-20T10:29:24.880191+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.904385+0000","last_change":"2024-04-20T10:29:24.902599+0000","last_active":"2024-04-20T10:29:24.904385+0000","last_peered":"2024-04-20T10:29:24.904385+0000","last_clean":"2024-04-20T10:29:24.904385+0000","last_became_active":"2024-04-20T10:29:24.902243+0000","last_became_peered":"2024-04-20T10:29:24.902243+0000","last_unstale":"2024-04-20T10:29:24.904385+0000","last_undegraded":"2024-04-20T10:29:24.904385+0000","last_fullsized":"2024-04-20T10:29:24.904385+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:27.152060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874424+0000","last_change":"2024-04-20T10:29:24.874424+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874424+0000","last_undegraded":"2024-04-20T10:29:24.874424+0000","last_fullsized":"2024-04-20T10:29:24.874424+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.900314+0000","last_change":"2024-04-20T10:29:24.895695+0000","last_active":"2024-04-20T10:29:24.900314+0000","last_peered":"2024-04-20T10:29:24.900314+0000","last_clean":"2024-04-20T10:29:24.900314+0000","last_became_active":"2024-04-20T10:29:24.895319+0000","last_became_peered":"2024-04-20T10:29:24.895319+0000","last_unstale":"2024-04-20T10:29:24.900314+0000","last_undegraded":"2024-04-20T10:29:24.900314+0000","last_fullsized":"2024-04-20T10:29:24.900314+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:12.664563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880516+0000","last_change":"2024-04-20T10:29:24.880516+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880516+0000","last_undegraded":"2024-04-20T10:29:24.880516+0000","last_fullsized":"2024-04-20T10:29:24.880516+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.867151+0000","last_change":"2024-04-20T10:29:23.867151+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.867151+0000","last_undegraded":"2024-04-20T10:29:23.867151+0000","last_fullsized":"2024-04-20T10:29:23.867151+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874247+0000","last_change":"2024-04-20T10:29:24.874247+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874247+0000","last_undegraded":"2024-04-20T10:29:24.874247+0000","last_fullsized":"2024-04-20T10:29:24.874247+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.909342+0000","last_change":"2024-04-20T10:29:24.908508+0000","last_active":"2024-04-20T10:29:24.909342+0000","last_peered":"2024-04-20T10:29:24.909342+0000","last_clean":"2024-04-20T10:29:24.909342+0000","last_became_active":"2024-04-20T10:29:24.908247+0000","last_became_peered":"2024-04-20T10:29:24.908247+0000","last_unstale":"2024-04-20T10:29:24.909342+0000","last_undegraded":"2024-04-20T10:29:24.909342+0000","last_fullsized":"2024-04-20T10:29:24.909342+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:47.895831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.904345+0000","last_change":"2024-04-20T10:29:24.902051+0000","last_active":"2024-04-20T10:29:24.904345+0000","last_peered":"2024-04-20T10:29:24.904345+0000","last_clean":"2024-04-20T10:29:24.904345+0000","last_became_active":"2024-04-20T10:29:24.899974+0000","last_became_peered":"2024-04-20T10:29:24.899974+0000","last_unstale":"2024-04-20T10:29:24.904345+0000","last_undegraded":"2024-04-20T10:29:24.904345+0000","last_fullsized":"2024-04-20T10:29:24.904345+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:22.781056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879206+0000","last_change":"2024-04-20T10:29:24.879206+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879206+0000","last_undegraded":"2024-04-20T10:29:24.879206+0000","last_fullsized":"2024-04-20T10:29:24.879206+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.907797+0000","last_change":"2024-04-20T10:29:24.906546+0000","last_active":"2024-04-20T10:29:24.907797+0000","last_peered":"2024-04-20T10:29:24.907797+0000","last_clean":"2024-04-20T10:29:24.907797+0000","last_became_active":"2024-04-20T10:29:24.906232+0000","last_became_peered":"2024-04-20T10:29:24.906232+0000","last_unstale":"2024-04-20T10:29:24.907797+0000","last_undegraded":"2024-04-20T10:29:24.907797+0000","last_fullsized":"2024-04-20T10:29:24.907797+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:58:20.925379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879048+0000","last_change":"2024-04-20T10:29:24.879048+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879048+0000","last_undegraded":"2024-04-20T10:29:24.879048+0000","last_fullsized":"2024-04-20T10:29:24.879048+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.866791+0000","last_change":"2024-04-20T10:29:23.866791+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.866791+0000","last_undegraded":"2024-04-20T10:29:23.866791+0000","last_fullsized":"2024-04-20T10:29:23.866791+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.877137+0000","last_change":"2024-04-20T10:29:24.877137+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.877137+0000","last_undegraded":"2024-04-20T10:29:24.877137+0000","last_fullsized":"2024-04-20T10:29:24.877137+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.892731+0000","last_change":"2024-04-20T10:29:24.885719+0000","last_active":"2024-04-20T10:29:24.892731+0000","last_peered":"2024-04-20T10:29:24.892731+0000","last_clean":"2024-04-20T10:29:24.892731+0000","last_became_active":"2024-04-20T10:29:24.885350+0000","last_became_peered":"2024-04-20T10:29:24.885350+0000","last_unstale":"2024-04-20T10:29:24.892731+0000","last_undegraded":"2024-04-20T10:29:24.892731+0000","last_fullsized":"2024-04-20T10:29:24.892731+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:30:44.587858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.867312+0000","last_change":"2024-04-20T10:29:23.867312+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.867312+0000","last_undegraded":"2024-04-20T10:29:23.867312+0000","last_fullsized":"2024-04-20T10:29:23.867312+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.883364+0000","last_change":"2024-04-20T10:29:24.883364+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.883364+0000","last_undegraded":"2024-04-20T10:29:24.883364+0000","last_fullsized":"2024-04-20T10:29:24.883364+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.894252+0000","last_change":"2024-04-20T10:29:24.891333+0000","last_active":"2024-04-20T10:29:24.894252+0000","last_peered":"2024-04-20T10:29:24.894252+0000","last_clean":"2024-04-20T10:29:24.894252+0000","last_became_active":"2024-04-20T10:29:24.890971+0000","last_became_peered":"2024-04-20T10:29:24.890971+0000","last_unstale":"2024-04-20T10:29:24.894252+0000","last_undegraded":"2024-04-20T10:29:24.894252+0000","last_fullsized":"2024-04-20T10:29:24.894252+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:27.931008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.907457+0000","last_change":"2024-04-20T10:29:24.906234+0000","last_active":"2024-04-20T10:29:24.907457+0000","last_peered":"2024-04-20T10:29:24.907457+0000","last_clean":"2024-04-20T10:29:24.907457+0000","last_became_active":"2024-04-20T10:29:24.905929+0000","last_became_peered":"2024-04-20T10:29:24.905929+0000","last_unstale":"2024-04-20T10:29:24.907457+0000","last_undegraded":"2024-04-20T10:29:24.907457+0000","last_fullsized":"2024-04-20T10:29:24.907457+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:35.896543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874744+0000","last_change":"2024-04-20T10:29:24.874744+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874744+0000","last_undegraded":"2024-04-20T10:29:24.874744+0000","last_fullsized":"2024-04-20T10:29:24.874744+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.866904+0000","last_change":"2024-04-20T10:29:23.866904+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.866904+0000","last_undegraded":"2024-04-20T10:29:23.866904+0000","last_fullsized":"2024-04-20T10:29:23.866904+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.868505+0000","last_change":"2024-04-20T10:29:23.868505+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.868505+0000","last_undegraded":"2024-04-20T10:29:23.868505+0000","last_fullsized":"2024-04-20T10:29:23.868505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878228+0000","last_change":"2024-04-20T10:29:24.878228+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878228+0000","last_undegraded":"2024-04-20T10:29:24.878228+0000","last_fullsized":"2024-04-20T10:29:24.878228+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.898029+0000","last_change":"2024-04-20T10:29:24.893579+0000","last_active":"2024-04-20T10:29:24.898029+0000","last_peered":"2024-04-20T10:29:24.898029+0000","last_clean":"2024-04-20T10:29:24.898029+0000","last_became_active":"2024-04-20T10:29:24.893233+0000","last_became_peered":"2024-04-20T10:29:24.893233+0000","last_unstale":"2024-04-20T10:29:24.898029+0000","last_undegraded":"2024-04-20T10:29:24.898029+0000","last_fullsized":"2024-04-20T10:29:24.898029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:47.895831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879529+0000","last_change":"2024-04-20T10:29:24.879529+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879529+0000","last_undegraded":"2024-04-20T10:29:24.879529+0000","last_fullsized":"2024-04-20T10:29:24.879529+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.895311+0000","last_change":"2024-04-20T10:29:24.891939+0000","last_active":"2024-04-20T10:29:24.895311+0000","last_peered":"2024-04-20T10:29:24.895311+0000","last_clean":"2024-04-20T10:29:24.895311+0000","last_became_active":"2024-04-20T10:29:24.891666+0000","last_became_peered":"2024-04-20T10:29:24.891666+0000","last_unstale":"2024-04-20T10:29:24.895311+0000","last_undegraded":"2024-04-20T10:29:24.895311+0000","last_fullsized":"2024-04-20T10:29:24.895311+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:11.246884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.886729+0000","last_change":"2024-04-20T10:29:24.886729+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.886729+0000","last_undegraded":"2024-04-20T10:29:24.886729+0000","last_fullsized":"2024-04-20T10:29:24.886729+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.893680+0000","last_change":"2024-04-20T10:29:24.890425+0000","last_active":"2024-04-20T10:29:24.893680+0000","last_peered":"2024-04-20T10:29:24.893680+0000","last_clean":"2024-04-20T10:29:24.893680+0000","last_became_active":"2024-04-20T10:29:24.890115+0000","last_became_peered":"2024-04-20T10:29:24.890115+0000","last_unstale":"2024-04-20T10:29:24.893680+0000","last_undegraded":"2024-04-20T10:29:24.893680+0000","last_fullsized":"2024-04-20T10:29:24.893680+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:14.922861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874106+0000","last_change":"2024-04-20T10:29:24.874106+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874106+0000","last_undegraded":"2024-04-20T10:29:24.874106+0000","last_fullsized":"2024-04-20T10:29:24.874106+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.893006+0000","last_change":"2024-04-20T10:29:24.886285+0000","last_active":"2024-04-20T10:29:24.893006+0000","last_peered":"2024-04-20T10:29:24.893006+0000","last_clean":"2024-04-20T10:29:24.893006+0000","last_became_active":"2024-04-20T10:29:24.886016+0000","last_became_peered":"2024-04-20T10:29:24.886016+0000","last_unstale":"2024-04-20T10:29:24.893006+0000","last_undegraded":"2024-04-20T10:29:24.893006+0000","last_fullsized":"2024-04-20T10:29:24.893006+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:43.826357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874212+0000","last_change":"2024-04-20T10:29:24.874212+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874212+0000","last_undegraded":"2024-04-20T10:29:24.874212+0000","last_fullsized":"2024-04-20T10:29:24.874212+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.908875+0000","last_change":"2024-04-20T10:29:24.907760+0000","last_active":"2024-04-20T10:29:24.908875+0000","last_peered":"2024-04-20T10:29:24.908875+0000","last_clean":"2024-04-20T10:29:24.908875+0000","last_became_active":"2024-04-20T10:29:24.907189+0000","last_became_peered":"2024-04-20T10:29:24.907189+0000","last_unstale":"2024-04-20T10:29:24.908875+0000","last_undegraded":"2024-04-20T10:29:24.908875+0000","last_fullsized":"2024-04-20T10:29:24.908875+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:04.160335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.874054+0000","last_change":"2024-04-20T10:29:24.874054+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.874054+0000","last_undegraded":"2024-04-20T10:29:24.874054+0000","last_fullsized":"2024-04-20T10:29:24.874054+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.906364+0000","last_change":"2024-04-20T10:29:24.902160+0000","last_active":"2024-04-20T10:29:24.906364+0000","last_peered":"2024-04-20T10:29:24.906364+0000","last_clean":"2024-04-20T10:29:24.906364+0000","last_became_active":"2024-04-20T10:29:24.900595+0000","last_became_peered":"2024-04-20T10:29:24.900595+0000","last_unstale":"2024-04-20T10:29:24.906364+0000","last_undegraded":"2024-04-20T10:29:24.906364+0000","last_fullsized":"2024-04-20T10:29:24.906364+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:01.416574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.887029+0000","last_change":"2024-04-20T10:29:24.887029+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.887029+0000","last_undegraded":"2024-04-20T10:29:24.887029+0000","last_fullsized":"2024-04-20T10:29:24.887029+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.904132+0000","last_change":"2024-04-20T10:29:24.896677+0000","last_active":"2024-04-20T10:29:24.904132+0000","last_peered":"2024-04-20T10:29:24.904132+0000","last_clean":"2024-04-20T10:29:24.904132+0000","last_became_active":"2024-04-20T10:29:24.896023+0000","last_became_peered":"2024-04-20T10:29:24.896023+0000","last_unstale":"2024-04-20T10:29:24.904132+0000","last_undegraded":"2024-04-20T10:29:24.904132+0000","last_fullsized":"2024-04-20T10:29:24.904132+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:50.540035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.879132+0000","last_change":"2024-04-20T10:29:24.879132+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.879132+0000","last_undegraded":"2024-04-20T10:29:24.879132+0000","last_fullsized":"2024-04-20T10:29:24.879132+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.905295+0000","last_change":"2024-04-20T10:29:24.903532+0000","last_active":"2024-04-20T10:29:24.905295+0000","last_peered":"2024-04-20T10:29:24.905295+0000","last_clean":"2024-04-20T10:29:24.905295+0000","last_became_active":"2024-04-20T10:29:24.902068+0000","last_became_peered":"2024-04-20T10:29:24.902068+0000","last_unstale":"2024-04-20T10:29:24.905295+0000","last_undegraded":"2024-04-20T10:29:24.905295+0000","last_fullsized":"2024-04-20T10:29:24.905295+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:45.052148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.898005+0000","last_change":"2024-04-20T10:29:24.892163+0000","last_active":"2024-04-20T10:29:24.898005+0000","last_peered":"2024-04-20T10:29:24.898005+0000","last_clean":"2024-04-20T10:29:24.898005+0000","last_became_active":"2024-04-20T10:29:24.891419+0000","last_became_peered":"2024-04-20T10:29:24.891419+0000","last_unstale":"2024-04-20T10:29:24.898005+0000","last_undegraded":"2024-04-20T10:29:24.898005+0000","last_fullsized":"2024-04-20T10:29:24.898005+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:27.152060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.880682+0000","last_change":"2024-04-20T10:29:24.880682+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.880682+0000","last_undegraded":"2024-04-20T10:29:24.880682+0000","last_fullsized":"2024-04-20T10:29:24.880682+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.886815+0000","last_change":"2024-04-20T10:29:24.883849+0000","last_active":"2024-04-20T10:29:24.886815+0000","last_peered":"2024-04-20T10:29:24.886815+0000","last_clean":"2024-04-20T10:29:24.886815+0000","last_became_active":"2024-04-20T10:29:24.883436+0000","last_became_peered":"2024-04-20T10:29:24.883436+0000","last_unstale":"2024-04-20T10:29:24.886815+0000","last_undegraded":"2024-04-20T10:29:24.886815+0000","last_fullsized":"2024-04-20T10:29:24.886815+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:43.911327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:29:24.858281+0000","last_change":"2024-04-20T10:29:24.858281+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.858281+0000","last_undegraded":"2024-04-20T10:29:24.858281+0000","last_fullsized":"2024-04-20T10:29:24.858281+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.905653+0000","last_change":"2024-04-20T10:29:24.902461+0000","last_active":"2024-04-20T10:29:24.905653+0000","last_peered":"2024-04-20T10:29:24.905653+0000","last_clean":"2024-04-20T10:29:24.905653+0000","last_became_active":"2024-04-20T10:29:24.901505+0000","last_became_peered":"2024-04-20T10:29:24.901505+0000","last_unstale":"2024-04-20T10:29:24.905653+0000","last_undegraded":"2024-04-20T10:29:24.905653+0000","last_fullsized":"2024-04-20T10:29:24.905653+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:50.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.886859+0000","last_change":"2024-04-20T10:29:24.886859+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.886859+0000","last_undegraded":"2024-04-20T10:29:24.886859+0000","last_fullsized":"2024-04-20T10:29:24.886859+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.901076+0000","last_change":"2024-04-20T10:29:24.899608+0000","last_active":"2024-04-20T10:29:24.901076+0000","last_peered":"2024-04-20T10:29:24.901076+0000","last_clean":"2024-04-20T10:29:24.901076+0000","last_became_active":"2024-04-20T10:29:24.899329+0000","last_became_peered":"2024-04-20T10:29:24.899329+0000","last_unstale":"2024-04-20T10:29:24.901076+0000","last_undegraded":"2024-04-20T10:29:24.901076+0000","last_fullsized":"2024-04-20T10:29:24.901076+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:09.312548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.881471+0000","last_change":"2024-04-20T10:29:24.881471+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.881471+0000","last_undegraded":"2024-04-20T10:29:24.881471+0000","last_fullsized":"2024-04-20T10:29:24.881471+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.906175+0000","last_change":"2024-04-20T10:29:24.901189+0000","last_active":"2024-04-20T10:29:24.906175+0000","last_peered":"2024-04-20T10:29:24.906175+0000","last_clean":"2024-04-20T10:29:24.906175+0000","last_became_active":"2024-04-20T10:29:24.900748+0000","last_became_peered":"2024-04-20T10:29:24.900748+0000","last_unstale":"2024-04-20T10:29:24.906175+0000","last_undegraded":"2024-04-20T10:29:24.906175+0000","last_fullsized":"2024-04-20T10:29:24.906175+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:33.581245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.890653+0000","last_change":"2024-04-20T10:29:24.890653+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.890653+0000","last_undegraded":"2024-04-20T10:29:24.890653+0000","last_fullsized":"2024-04-20T10:29:24.890653+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.910618+0000","last_change":"2024-04-20T10:29:24.907398+0000","last_active":"2024-04-20T10:29:24.910618+0000","last_peered":"2024-04-20T10:29:24.910618+0000","last_clean":"2024-04-20T10:29:24.910618+0000","last_became_active":"2024-04-20T10:29:24.907118+0000","last_became_peered":"2024-04-20T10:29:24.907118+0000","last_unstale":"2024-04-20T10:29:24.910618+0000","last_undegraded":"2024-04-20T10:29:24.910618+0000","last_fullsized":"2024-04-20T10:29:24.910618+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:12:03.281492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.884473+0000","last_change":"2024-04-20T10:29:24.884473+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.884473+0000","last_undegraded":"2024-04-20T10:29:24.884473+0000","last_fullsized":"2024-04-20T10:29:24.884473+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.867282+0000","last_change":"2024-04-20T10:29:23.867282+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.867282+0000","last_undegraded":"2024-04-20T10:29:23.867282+0000","last_fullsized":"2024-04-20T10:29:23.867282+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.875165+0000","last_change":"2024-04-20T10:29:24.875165+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.875165+0000","last_undegraded":"2024-04-20T10:29:24.875165+0000","last_fullsized":"2024-04-20T10:29:24.875165+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.903516+0000","last_change":"2024-04-20T10:29:24.900706+0000","last_active":"2024-04-20T10:29:24.903516+0000","last_peered":"2024-04-20T10:29:24.903516+0000","last_clean":"2024-04-20T10:29:24.903516+0000","last_became_active":"2024-04-20T10:29:24.900256+0000","last_became_peered":"2024-04-20T10:29:24.900256+0000","last_unstale":"2024-04-20T10:29:24.903516+0000","last_undegraded":"2024-04-20T10:29:24.903516+0000","last_fullsized":"2024-04-20T10:29:24.903516+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878515+0000","last_change":"2024-04-20T10:29:24.878515+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878515+0000","last_undegraded":"2024-04-20T10:29:24.878515+0000","last_fullsized":"2024-04-20T10:29:24.878515+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.902396+0000","last_change":"2024-04-20T10:29:24.901083+0000","last_active":"2024-04-20T10:29:24.902396+0000","last_peered":"2024-04-20T10:29:24.902396+0000","last_clean":"2024-04-20T10:29:24.902396+0000","last_became_active":"2024-04-20T10:29:24.899838+0000","last_became_peered":"2024-04-20T10:29:24.899838+0000","last_unstale":"2024-04-20T10:29:24.902396+0000","last_undegraded":"2024-04-20T10:29:24.902396+0000","last_fullsized":"2024-04-20T10:29:24.902396+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:29.764776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878718+0000","last_change":"2024-04-20T10:29:24.878718+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878718+0000","last_undegraded":"2024-04-20T10:29:24.878718+0000","last_fullsized":"2024-04-20T10:29:24.878718+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":1,"reported_epoch":39,"state":"creating+peering","last_fresh":"2024-04-20T10:29:23.866844+0000","last_change":"2024-04-20T10:29:23.866844+0000","last_active":"2024-04-20T10:29:23.844512+0000","last_peered":"2024-04-20T10:29:23.844512+0000","last_clean":"2024-04-20T10:29:23.844512+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:23.866844+0000","last_undegraded":"2024-04-20T10:29:23.866844+0000","last_fullsized":"2024-04-20T10:29:23.866844+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":40,"state":"creating+peering","last_fresh":"2024-04-20T10:29:24.878611+0000","last_change":"2024-04-20T10:29:24.878611+0000","last_active":"2024-04-20T10:29:24.858281+0000","last_peered":"2024-04-20T10:29:24.858281+0000","last_clean":"2024-04-20T10:29:24.858281+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:29:24.878611+0000","last_undegraded":"2024-04-20T10:29:24.878611+0000","last_fullsized":"2024-04-20T10:29:24.878611+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":9,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.903563+0000","last_change":"2024-04-20T10:29:24.900850+0000","last_active":"2024-04-20T10:29:24.903563+0000","last_peered":"2024-04-20T10:29:24.903563+0000","last_clean":"2024-04-20T10:29:24.903563+0000","last_became_active":"2024-04-20T10:29:24.898940+0000","last_became_peered":"2024-04-20T10:29:24.898940+0000","last_unstale":"2024-04-20T10:29:24.903563+0000","last_undegraded":"2024-04-20T10:29:24.903563+0000","last_fullsized":"2024-04-20T10:29:24.903563+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":204,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-20T10:29:24.870383+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:29:24.870383+0000","last_peered":"2024-04-20T10:29:24.870383+0000","last_clean":"2024-04-20T10:29:24.870383+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:29:24.870383+0000","last_undegraded":"2024-04-20T10:29:24.870383+0000","last_fullsized":"2024-04-20T10:29:24.870383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":102,"acting":102,"num_store_stats":7},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509468,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":775120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509468,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31624,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340216,"statfs":{"total":96636764160,"available":96604381184,"internally_reserved":0,"allocated":454656,"data_stored":191419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509468,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339648,"statfs":{"total":96636764160,"available":96603799552,"internally_reserved":0,"allocated":1036288,"data_stored":775120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509468,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31612,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340228,"statfs":{"total":96636764160,"available":96604393472,"internally_reserved":0,"allocated":442368,"data_stored":184752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":420,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":430080,"data_stored":178543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:29:30.850 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:29:31.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.103+0000 7fa8a6000640 1 Processor -- start 2024-04-20T10:29:31.103 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.103+0000 7fa8a6000640 1 -- start start 2024-04-20T10:29:31.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.104+0000 7fa8a6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a006cde0 0x7fa8a00d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.104+0000 7fa8a6000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa8a00b3330 0x7fa8a00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.104+0000 7fa8a6000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8a006c190 con 0x7fa8a0161d50 2024-04-20T10:29:31.104 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.104+0000 7fa8a6000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa8a0162130 con 0x7fa8a00b3330 2024-04-20T10:29:31.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.104+0000 7fa8a6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a006c470 con 0x7fa8a006cde0 2024-04-20T10:29:31.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.105+0000 7fa8a5600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa8a00b3330 0x7fa8a00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.105+0000 7fa89fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a006cde0 0x7fa8a00d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.105+0000 7fa8a4c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa8a0161d50 0x7fa8a006c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36632/0 (socket says 172.21.15.47:36632) 2024-04-20T10:29:31.105 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.105+0000 7fa8a4c00640 1 -- 172.21.15.47:0/3459128895 learned_addr learned my addr 172.21.15.47:0/3459128895 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:31.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.106+0000 7fa89f400640 1 -- 172.21.15.47:0/3459128895 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3253275395 0 0) 0x7fa8a006c190 con 0x7fa8a0161d50 2024-04-20T10:29:31.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.106+0000 7fa89f400640 1 -- 172.21.15.47:0/3459128895 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa884003520 con 0x7fa8a0161d50 2024-04-20T10:29:31.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.106+0000 7fa8a5600640 1 -- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a006cde0 msgr2=0x7fa8a00d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:31.106 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.106+0000 7fa8a5600640 1 --2- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a006cde0 0x7fa8a00d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.106+0000 7fa8a5600640 1 -- 172.21.15.47:0/3459128895 >> v1:172.21.15.47:6789/0 conn(0x7fa8a0161d50 legacy=0x7fa8a006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.106+0000 7fa8a5600640 1 -- 172.21.15.47:0/3459128895 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a00d2e00 con 0x7fa8a00b3330 2024-04-20T10:29:31.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.106+0000 7fa89fe00640 1 --2- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a006cde0 0x7fa8a00d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:29:31.107 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.107+0000 7fa8a5600640 1 --2- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 0x7fa8a00b3880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa898009d90 tx=0x7fa89802bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a075997291de75 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:31.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.108+0000 7fa89f400640 1 -- 172.21.15.47:0/3459128895 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898039070 con 0x7fa8a00b3330 2024-04-20T10:29:31.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.108+0000 7fa89f400640 1 -- 172.21.15.47:0/3459128895 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa89802cc50 con 0x7fa8a00b3330 2024-04-20T10:29:31.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.108+0000 7fa89f400640 1 -- 172.21.15.47:0/3459128895 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa898034440 con 0x7fa8a00b3330 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 -- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 msgr2=0x7fa8a00b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 --2- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 0x7fa8a00b3880 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa898009d90 tx=0x7fa89802bea0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 -- 172.21.15.47:0/3459128895 shutdown_connections 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 --2- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 0x7fa8a00b3880 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 --2- 172.21.15.47:0/3459128895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a006cde0 0x7fa8a00d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 -- 172.21.15.47:0/3459128895 >> 172.21.15.47:0/3459128895 conn(0x7fa8a00d4450 msgr2=0x7fa8a00d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 -- 172.21.15.47:0/3459128895 shutdown_connections 2024-04-20T10:29:31.109 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.109+0000 7fa8a6000640 1 -- 172.21.15.47:0/3459128895 wait complete. 2024-04-20T10:29:31.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.110+0000 7fa8a6000640 1 Processor -- start 2024-04-20T10:29:31.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.111+0000 7fa8a6000640 1 -- start start 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8a006cde0 0x7fa8a014a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 0x7fa8a00cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a0161d50 0x7fa8a01f7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a6000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a00b7660 con 0x7fa8a006cde0 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a00d33c0 con 0x7fa8a00b3330 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8a00b77e0 con 0x7fa8a0161d50 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8a006cde0 0x7fa8a014a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8a006cde0 0x7fa8a014a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34728/0 (socket says 172.21.15.47:34728) 2024-04-20T10:29:31.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 -- 172.21.15.47:0/2048287723 learned_addr learned my addr 172.21.15.47:0/2048287723 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 -- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a0161d50 msgr2=0x7fa8a01f7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a0161d50 0x7fa8a01f7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 -- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 msgr2=0x7fa8a00cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 0x7fa8a00cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.112+0000 7fa8a4c00640 1 -- 172.21.15.47:0/2048287723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a00cbc10 con 0x7fa8a006cde0 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.113+0000 7fa8a4c00640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8a006cde0 0x7fa8a014a1b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c3c0 tx=0x7fa89000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.113+0000 7fa89cc00640 1 -- 172.21.15.47:0/2048287723 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa890017070 con 0x7fa8a006cde0 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.113+0000 7fa89cc00640 1 -- 172.21.15.47:0/2048287723 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa890005ce0 con 0x7fa8a006cde0 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.113+0000 7fa89cc00640 1 -- 172.21.15.47:0/2048287723 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8900072f0 con 0x7fa8a006cde0 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.113+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a00cd1e0 con 0x7fa8a006cde0 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.113+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a00cd780 con 0x7fa8a006cde0 2024-04-20T10:29:31.114 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.114+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a0075b60 con 0x7fa8a006cde0 2024-04-20T10:29:31.115 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.115+0000 7fa89cc00640 1 -- 172.21.15.47:0/2048287723 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa89001a070 con 0x7fa8a006cde0 2024-04-20T10:29:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.116+0000 7fa89cc00640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88006c940 0x7fa88006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.116+0000 7fa89cc00640 1 -- 172.21.15.47:0/2048287723 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fa89008b060 con 0x7fa8a006cde0 2024-04-20T10:29:31.116 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.117+0000 7fa89fe00640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88006c940 0x7fa88006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.118+0000 7fa89fe00640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88006c940 0x7fa88006ee00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006e80 tx=0x7fa88c006dd0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:31.120 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.120+0000 7fa89cc00640 1 -- 172.21.15.47:0/2048287723 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa890059370 con 0x7fa8a006cde0 2024-04-20T10:29:31.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.322+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8a00cf770 con 0x7fa88006c940 2024-04-20T10:29:31.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.341+0000 7fa89cc00640 1 -- 172.21.15.47:0/2048287723 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251966 (secure 0 0 0) 0x7fa8a00cf770 con 0x7fa88006c940 2024-04-20T10:29:31.345 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:31.346 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:29:31.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88006c940 msgr2=0x7fa88006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88006c940 0x7fa88006ee00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006e80 tx=0x7fa88c006dd0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8a006cde0 msgr2=0x7fa8a014a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8a006cde0 0x7fa8a014a1b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa89000c3c0 tx=0x7fa89000c880 comp rx=0 tx=0).stop 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 shutdown_connections 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8a0161d50 0x7fa8a01f7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8a00b3330 0x7fa8a00cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa88006c940 0x7fa88006ee00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 --2- 172.21.15.47:0/2048287723 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa8a006cde0 0x7fa8a014a1b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.346+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 >> 172.21.15.47:0/2048287723 conn(0x7fa8a00d4450 msgr2=0x7fa8a0160be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.347+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 shutdown_connections 2024-04-20T10:29:31.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.347+0000 7fa8a6000640 1 -- 172.21.15.47:0/2048287723 wait complete. 2024-04-20T10:29:31.366 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-20T10:29:31.108884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237800,"kb_used_data":4968,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736920,"statfs":{"total":773094113280,"available":772850606080,"internally_reserved":0,"allocated":5087232,"data_stored":2827780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.241224"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946274+0000","last_change":"2024-04-20T10:29:25.932997+0000","last_active":"2024-04-20T10:29:26.946274+0000","last_peered":"2024-04-20T10:29:26.946274+0000","last_clean":"2024-04-20T10:29:26.946274+0000","last_became_active":"2024-04-20T10:29:25.932397+0000","last_became_peered":"2024-04-20T10:29:25.932397+0000","last_unstale":"2024-04-20T10:29:26.946274+0000","last_undegraded":"2024-04-20T10:29:26.946274+0000","last_fullsized":"2024-04-20T10:29:26.946274+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:06.846350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945063+0000","last_change":"2024-04-20T10:29:25.915562+0000","last_active":"2024-04-20T10:29:26.945063+0000","last_peered":"2024-04-20T10:29:26.945063+0000","last_clean":"2024-04-20T10:29:26.945063+0000","last_became_active":"2024-04-20T10:29:25.914716+0000","last_became_peered":"2024-04-20T10:29:25.914716+0000","last_unstale":"2024-04-20T10:29:26.945063+0000","last_undegraded":"2024-04-20T10:29:26.945063+0000","last_fullsized":"2024-04-20T10:29:26.945063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:28.944777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946847+0000","last_change":"2024-04-20T10:29:25.943381+0000","last_active":"2024-04-20T10:29:26.946847+0000","last_peered":"2024-04-20T10:29:26.946847+0000","last_clean":"2024-04-20T10:29:26.946847+0000","last_became_active":"2024-04-20T10:29:25.943065+0000","last_became_peered":"2024-04-20T10:29:25.943065+0000","last_unstale":"2024-04-20T10:29:26.946847+0000","last_undegraded":"2024-04-20T10:29:26.946847+0000","last_fullsized":"2024-04-20T10:29:26.946847+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:39.697615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946286+0000","last_change":"2024-04-20T10:29:25.927762+0000","last_active":"2024-04-20T10:29:26.946286+0000","last_peered":"2024-04-20T10:29:26.946286+0000","last_clean":"2024-04-20T10:29:26.946286+0000","last_became_active":"2024-04-20T10:29:25.927221+0000","last_became_peered":"2024-04-20T10:29:25.927221+0000","last_unstale":"2024-04-20T10:29:26.946286+0000","last_undegraded":"2024-04-20T10:29:26.946286+0000","last_fullsized":"2024-04-20T10:29:26.946286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:14:22.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917869+0000","last_change":"2024-04-20T10:29:25.942229+0000","last_active":"2024-04-20T10:29:26.917869+0000","last_peered":"2024-04-20T10:29:26.917869+0000","last_clean":"2024-04-20T10:29:26.917869+0000","last_became_active":"2024-04-20T10:29:25.941934+0000","last_became_peered":"2024-04-20T10:29:25.941934+0000","last_unstale":"2024-04-20T10:29:26.917869+0000","last_undegraded":"2024-04-20T10:29:26.917869+0000","last_fullsized":"2024-04-20T10:29:26.917869+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:48.909601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944030+0000","last_change":"2024-04-20T10:29:25.923700+0000","last_active":"2024-04-20T10:29:26.944030+0000","last_peered":"2024-04-20T10:29:26.944030+0000","last_clean":"2024-04-20T10:29:26.944030+0000","last_became_active":"2024-04-20T10:29:25.922823+0000","last_became_peered":"2024-04-20T10:29:25.922823+0000","last_unstale":"2024-04-20T10:29:26.944030+0000","last_undegraded":"2024-04-20T10:29:26.944030+0000","last_fullsized":"2024-04-20T10:29:26.944030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:18.438200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915101+0000","last_change":"2024-04-20T10:29:25.915804+0000","last_active":"2024-04-20T10:29:26.915101+0000","last_peered":"2024-04-20T10:29:26.915101+0000","last_clean":"2024-04-20T10:29:26.915101+0000","last_became_active":"2024-04-20T10:29:25.915493+0000","last_became_peered":"2024-04-20T10:29:25.915493+0000","last_unstale":"2024-04-20T10:29:26.915101+0000","last_undegraded":"2024-04-20T10:29:26.915101+0000","last_fullsized":"2024-04-20T10:29:26.915101+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:28.165829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944198+0000","last_change":"2024-04-20T10:29:25.930111+0000","last_active":"2024-04-20T10:29:26.944198+0000","last_peered":"2024-04-20T10:29:26.944198+0000","last_clean":"2024-04-20T10:29:26.944198+0000","last_became_active":"2024-04-20T10:29:25.929086+0000","last_became_peered":"2024-04-20T10:29:25.929086+0000","last_unstale":"2024-04-20T10:29:26.944198+0000","last_undegraded":"2024-04-20T10:29:26.944198+0000","last_fullsized":"2024-04-20T10:29:26.944198+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:05.706072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918351+0000","last_change":"2024-04-20T10:29:25.938030+0000","last_active":"2024-04-20T10:29:26.918351+0000","last_peered":"2024-04-20T10:29:26.918351+0000","last_clean":"2024-04-20T10:29:26.918351+0000","last_became_active":"2024-04-20T10:29:25.937705+0000","last_became_peered":"2024-04-20T10:29:25.937705+0000","last_unstale":"2024-04-20T10:29:26.918351+0000","last_undegraded":"2024-04-20T10:29:26.918351+0000","last_fullsized":"2024-04-20T10:29:26.918351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:51.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915834+0000","last_change":"2024-04-20T10:29:25.921186+0000","last_active":"2024-04-20T10:29:26.915834+0000","last_peered":"2024-04-20T10:29:26.915834+0000","last_clean":"2024-04-20T10:29:26.915834+0000","last_became_active":"2024-04-20T10:29:25.920862+0000","last_became_peered":"2024-04-20T10:29:25.920862+0000","last_unstale":"2024-04-20T10:29:26.915834+0000","last_undegraded":"2024-04-20T10:29:26.915834+0000","last_fullsized":"2024-04-20T10:29:26.915834+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:45.007288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942466+0000","last_change":"2024-04-20T10:29:25.928586+0000","last_active":"2024-04-20T10:29:26.942466+0000","last_peered":"2024-04-20T10:29:26.942466+0000","last_clean":"2024-04-20T10:29:26.942466+0000","last_became_active":"2024-04-20T10:29:25.925435+0000","last_became_peered":"2024-04-20T10:29:25.925435+0000","last_unstale":"2024-04-20T10:29:26.942466+0000","last_undegraded":"2024-04-20T10:29:26.942466+0000","last_fullsized":"2024-04-20T10:29:26.942466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:45:38.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917226+0000","last_change":"2024-04-20T10:29:25.931669+0000","last_active":"2024-04-20T10:29:26.917226+0000","last_peered":"2024-04-20T10:29:26.917226+0000","last_clean":"2024-04-20T10:29:26.917226+0000","last_became_active":"2024-04-20T10:29:25.931356+0000","last_became_peered":"2024-04-20T10:29:25.931356+0000","last_unstale":"2024-04-20T10:29:26.917226+0000","last_undegraded":"2024-04-20T10:29:26.917226+0000","last_fullsized":"2024-04-20T10:29:26.917226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:15.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914837+0000","last_change":"2024-04-20T10:29:25.931125+0000","last_active":"2024-04-20T10:29:26.914837+0000","last_peered":"2024-04-20T10:29:26.914837+0000","last_clean":"2024-04-20T10:29:26.914837+0000","last_became_active":"2024-04-20T10:29:25.930841+0000","last_became_peered":"2024-04-20T10:29:25.930841+0000","last_unstale":"2024-04-20T10:29:26.914837+0000","last_undegraded":"2024-04-20T10:29:26.914837+0000","last_fullsized":"2024-04-20T10:29:26.914837+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:23.794826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940863+0000","last_change":"2024-04-20T10:29:25.929516+0000","last_active":"2024-04-20T10:29:26.940863+0000","last_peered":"2024-04-20T10:29:26.940863+0000","last_clean":"2024-04-20T10:29:26.940863+0000","last_became_active":"2024-04-20T10:29:25.929218+0000","last_became_peered":"2024-04-20T10:29:25.929218+0000","last_unstale":"2024-04-20T10:29:26.940863+0000","last_undegraded":"2024-04-20T10:29:26.940863+0000","last_fullsized":"2024-04-20T10:29:26.940863+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:25.861823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.911735+0000","last_change":"2024-04-20T10:29:25.912156+0000","last_active":"2024-04-20T10:29:26.911735+0000","last_peered":"2024-04-20T10:29:26.911735+0000","last_clean":"2024-04-20T10:29:26.911735+0000","last_became_active":"2024-04-20T10:29:25.911810+0000","last_became_peered":"2024-04-20T10:29:25.911810+0000","last_unstale":"2024-04-20T10:29:26.911735+0000","last_undegraded":"2024-04-20T10:29:26.911735+0000","last_fullsized":"2024-04-20T10:29:26.911735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:03.872640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914609+0000","last_change":"2024-04-20T10:29:25.927858+0000","last_active":"2024-04-20T10:29:26.914609+0000","last_peered":"2024-04-20T10:29:26.914609+0000","last_clean":"2024-04-20T10:29:26.914609+0000","last_became_active":"2024-04-20T10:29:25.927563+0000","last_became_peered":"2024-04-20T10:29:25.927563+0000","last_unstale":"2024-04-20T10:29:26.914609+0000","last_undegraded":"2024-04-20T10:29:26.914609+0000","last_fullsized":"2024-04-20T10:29:26.914609+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:12.564631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913527+0000","last_change":"2024-04-20T10:29:25.929945+0000","last_active":"2024-04-20T10:29:26.913527+0000","last_peered":"2024-04-20T10:29:26.913527+0000","last_clean":"2024-04-20T10:29:26.913527+0000","last_became_active":"2024-04-20T10:29:25.929631+0000","last_became_peered":"2024-04-20T10:29:25.929631+0000","last_unstale":"2024-04-20T10:29:26.913527+0000","last_undegraded":"2024-04-20T10:29:26.913527+0000","last_fullsized":"2024-04-20T10:29:26.913527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:26:28.673759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913374+0000","last_change":"2024-04-20T10:29:25.944916+0000","last_active":"2024-04-20T10:29:26.913374+0000","last_peered":"2024-04-20T10:29:26.913374+0000","last_clean":"2024-04-20T10:29:26.913374+0000","last_became_active":"2024-04-20T10:29:25.944362+0000","last_became_peered":"2024-04-20T10:29:25.944362+0000","last_unstale":"2024-04-20T10:29:26.913374+0000","last_undegraded":"2024-04-20T10:29:26.913374+0000","last_fullsized":"2024-04-20T10:29:26.913374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:36:07.459320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942644+0000","last_change":"2024-04-20T10:29:25.936202+0000","last_active":"2024-04-20T10:29:26.942644+0000","last_peered":"2024-04-20T10:29:26.942644+0000","last_clean":"2024-04-20T10:29:26.942644+0000","last_became_active":"2024-04-20T10:29:25.935861+0000","last_became_peered":"2024-04-20T10:29:25.935861+0000","last_unstale":"2024-04-20T10:29:26.942644+0000","last_undegraded":"2024-04-20T10:29:26.942644+0000","last_fullsized":"2024-04-20T10:29:26.942644+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:31:01.813606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942442+0000","last_change":"2024-04-20T10:29:25.940311+0000","last_active":"2024-04-20T10:29:26.942442+0000","last_peered":"2024-04-20T10:29:26.942442+0000","last_clean":"2024-04-20T10:29:26.942442+0000","last_became_active":"2024-04-20T10:29:25.939941+0000","last_became_peered":"2024-04-20T10:29:25.939941+0000","last_unstale":"2024-04-20T10:29:26.942442+0000","last_undegraded":"2024-04-20T10:29:26.942442+0000","last_fullsized":"2024-04-20T10:29:26.942442+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:05.174105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942072+0000","last_change":"2024-04-20T10:29:25.937804+0000","last_active":"2024-04-20T10:29:26.942072+0000","last_peered":"2024-04-20T10:29:26.942072+0000","last_clean":"2024-04-20T10:29:26.942072+0000","last_became_active":"2024-04-20T10:29:25.937537+0000","last_became_peered":"2024-04-20T10:29:25.937537+0000","last_unstale":"2024-04-20T10:29:26.942072+0000","last_undegraded":"2024-04-20T10:29:26.942072+0000","last_fullsized":"2024-04-20T10:29:26.942072+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:10.326318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941472+0000","last_change":"2024-04-20T10:29:25.938440+0000","last_active":"2024-04-20T10:29:26.941472+0000","last_peered":"2024-04-20T10:29:26.941472+0000","last_clean":"2024-04-20T10:29:26.941472+0000","last_became_active":"2024-04-20T10:29:25.936330+0000","last_became_peered":"2024-04-20T10:29:25.936330+0000","last_unstale":"2024-04-20T10:29:26.941472+0000","last_undegraded":"2024-04-20T10:29:26.941472+0000","last_fullsized":"2024-04-20T10:29:26.941472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:28.944777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912104+0000","last_change":"2024-04-20T10:29:25.921848+0000","last_active":"2024-04-20T10:29:26.912104+0000","last_peered":"2024-04-20T10:29:26.912104+0000","last_clean":"2024-04-20T10:29:26.912104+0000","last_became_active":"2024-04-20T10:29:25.921530+0000","last_became_peered":"2024-04-20T10:29:25.921530+0000","last_unstale":"2024-04-20T10:29:26.912104+0000","last_undegraded":"2024-04-20T10:29:26.912104+0000","last_fullsized":"2024-04-20T10:29:26.912104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:14.691988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944601+0000","last_change":"2024-04-20T10:29:25.937828+0000","last_active":"2024-04-20T10:29:26.944601+0000","last_peered":"2024-04-20T10:29:26.944601+0000","last_clean":"2024-04-20T10:29:26.944601+0000","last_became_active":"2024-04-20T10:29:25.937289+0000","last_became_peered":"2024-04-20T10:29:25.937289+0000","last_unstale":"2024-04-20T10:29:26.944601+0000","last_undegraded":"2024-04-20T10:29:26.944601+0000","last_fullsized":"2024-04-20T10:29:26.944601+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:44.872739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941542+0000","last_change":"2024-04-20T10:29:25.931338+0000","last_active":"2024-04-20T10:29:26.941542+0000","last_peered":"2024-04-20T10:29:26.941542+0000","last_clean":"2024-04-20T10:29:26.941542+0000","last_became_active":"2024-04-20T10:29:25.931042+0000","last_became_peered":"2024-04-20T10:29:25.931042+0000","last_unstale":"2024-04-20T10:29:26.941542+0000","last_undegraded":"2024-04-20T10:29:26.941542+0000","last_fullsized":"2024-04-20T10:29:26.941542+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:11.620620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944692+0000","last_change":"2024-04-20T10:29:25.932131+0000","last_active":"2024-04-20T10:29:26.944692+0000","last_peered":"2024-04-20T10:29:26.944692+0000","last_clean":"2024-04-20T10:29:26.944692+0000","last_became_active":"2024-04-20T10:29:25.931787+0000","last_became_peered":"2024-04-20T10:29:25.931787+0000","last_unstale":"2024-04-20T10:29:26.944692+0000","last_undegraded":"2024-04-20T10:29:26.944692+0000","last_fullsized":"2024-04-20T10:29:26.944692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:34:32.114848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942556+0000","last_change":"2024-04-20T10:29:25.935814+0000","last_active":"2024-04-20T10:29:26.942556+0000","last_peered":"2024-04-20T10:29:26.942556+0000","last_clean":"2024-04-20T10:29:26.942556+0000","last_became_active":"2024-04-20T10:29:25.935537+0000","last_became_peered":"2024-04-20T10:29:25.935537+0000","last_unstale":"2024-04-20T10:29:26.942556+0000","last_undegraded":"2024-04-20T10:29:26.942556+0000","last_fullsized":"2024-04-20T10:29:26.942556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:58.666030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914952+0000","last_change":"2024-04-20T10:29:25.933134+0000","last_active":"2024-04-20T10:29:26.914952+0000","last_peered":"2024-04-20T10:29:26.914952+0000","last_clean":"2024-04-20T10:29:26.914952+0000","last_became_active":"2024-04-20T10:29:25.932739+0000","last_became_peered":"2024-04-20T10:29:25.932739+0000","last_unstale":"2024-04-20T10:29:26.914952+0000","last_undegraded":"2024-04-20T10:29:26.914952+0000","last_fullsized":"2024-04-20T10:29:26.914952+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:15.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912565+0000","last_change":"2024-04-20T10:29:25.931473+0000","last_active":"2024-04-20T10:29:26.912565+0000","last_peered":"2024-04-20T10:29:26.912565+0000","last_clean":"2024-04-20T10:29:26.912565+0000","last_became_active":"2024-04-20T10:29:25.927277+0000","last_became_peered":"2024-04-20T10:29:25.927277+0000","last_unstale":"2024-04-20T10:29:26.912565+0000","last_undegraded":"2024-04-20T10:29:26.912565+0000","last_fullsized":"2024-04-20T10:29:26.912565+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:57.452559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912631+0000","last_change":"2024-04-20T10:29:25.932232+0000","last_active":"2024-04-20T10:29:26.912631+0000","last_peered":"2024-04-20T10:29:26.912631+0000","last_clean":"2024-04-20T10:29:26.912631+0000","last_became_active":"2024-04-20T10:29:25.931639+0000","last_became_peered":"2024-04-20T10:29:25.931639+0000","last_unstale":"2024-04-20T10:29:26.912631+0000","last_undegraded":"2024-04-20T10:29:26.912631+0000","last_fullsized":"2024-04-20T10:29:26.912631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:04.208340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915170+0000","last_change":"2024-04-20T10:29:25.930526+0000","last_active":"2024-04-20T10:29:26.915170+0000","last_peered":"2024-04-20T10:29:26.915170+0000","last_clean":"2024-04-20T10:29:26.915170+0000","last_became_active":"2024-04-20T10:29:25.930226+0000","last_became_peered":"2024-04-20T10:29:25.930226+0000","last_unstale":"2024-04-20T10:29:26.915170+0000","last_undegraded":"2024-04-20T10:29:26.915170+0000","last_fullsized":"2024-04-20T10:29:26.915170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:48.909601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945043+0000","last_change":"2024-04-20T10:29:25.929491+0000","last_active":"2024-04-20T10:29:26.945043+0000","last_peered":"2024-04-20T10:29:26.945043+0000","last_clean":"2024-04-20T10:29:26.945043+0000","last_became_active":"2024-04-20T10:29:25.928653+0000","last_became_peered":"2024-04-20T10:29:25.928653+0000","last_unstale":"2024-04-20T10:29:26.945043+0000","last_undegraded":"2024-04-20T10:29:26.945043+0000","last_fullsized":"2024-04-20T10:29:26.945043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:24:26.609508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913097+0000","last_change":"2024-04-20T10:29:25.924820+0000","last_active":"2024-04-20T10:29:26.913097+0000","last_peered":"2024-04-20T10:29:26.913097+0000","last_clean":"2024-04-20T10:29:26.913097+0000","last_became_active":"2024-04-20T10:29:25.923609+0000","last_became_peered":"2024-04-20T10:29:25.923609+0000","last_unstale":"2024-04-20T10:29:26.913097+0000","last_undegraded":"2024-04-20T10:29:26.913097+0000","last_fullsized":"2024-04-20T10:29:26.913097+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:24:16.621728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"41'4","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941497+0000","last_change":"2024-04-20T10:29:24.902599+0000","last_active":"2024-04-20T10:29:26.941497+0000","last_peered":"2024-04-20T10:29:26.941497+0000","last_clean":"2024-04-20T10:29:26.941497+0000","last_became_active":"2024-04-20T10:29:24.902243+0000","last_became_peered":"2024-04-20T10:29:24.902243+0000","last_unstale":"2024-04-20T10:29:26.941497+0000","last_undegraded":"2024-04-20T10:29:26.941497+0000","last_fullsized":"2024-04-20T10:29:26.941497+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:12.664563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944052+0000","last_change":"2024-04-20T10:29:25.935762+0000","last_active":"2024-04-20T10:29:26.944052+0000","last_peered":"2024-04-20T10:29:26.944052+0000","last_clean":"2024-04-20T10:29:26.944052+0000","last_became_active":"2024-04-20T10:29:25.935453+0000","last_became_peered":"2024-04-20T10:29:25.935453+0000","last_unstale":"2024-04-20T10:29:26.944052+0000","last_undegraded":"2024-04-20T10:29:26.944052+0000","last_fullsized":"2024-04-20T10:29:26.944052+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:12:39.385668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946081+0000","last_change":"2024-04-20T10:29:24.895695+0000","last_active":"2024-04-20T10:29:26.946081+0000","last_peered":"2024-04-20T10:29:26.946081+0000","last_clean":"2024-04-20T10:29:26.946081+0000","last_became_active":"2024-04-20T10:29:24.895319+0000","last_became_peered":"2024-04-20T10:29:24.895319+0000","last_unstale":"2024-04-20T10:29:26.946081+0000","last_undegraded":"2024-04-20T10:29:26.946081+0000","last_fullsized":"2024-04-20T10:29:26.946081+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:25.116742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913617+0000","last_change":"2024-04-20T10:29:25.934817+0000","last_active":"2024-04-20T10:29:26.913617+0000","last_peered":"2024-04-20T10:29:26.913617+0000","last_clean":"2024-04-20T10:29:26.913617+0000","last_became_active":"2024-04-20T10:29:25.934543+0000","last_became_peered":"2024-04-20T10:29:25.934543+0000","last_unstale":"2024-04-20T10:29:26.913617+0000","last_undegraded":"2024-04-20T10:29:26.913617+0000","last_fullsized":"2024-04-20T10:29:26.913617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:29.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946937+0000","last_change":"2024-04-20T10:29:24.908735+0000","last_active":"2024-04-20T10:29:26.946937+0000","last_peered":"2024-04-20T10:29:26.946937+0000","last_clean":"2024-04-20T10:29:26.946937+0000","last_became_active":"2024-04-20T10:29:24.908390+0000","last_became_peered":"2024-04-20T10:29:24.908390+0000","last_unstale":"2024-04-20T10:29:26.946937+0000","last_undegraded":"2024-04-20T10:29:26.946937+0000","last_fullsized":"2024-04-20T10:29:26.946937+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:12.598913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945696+0000","last_change":"2024-04-20T10:29:25.937482+0000","last_active":"2024-04-20T10:29:26.945696+0000","last_peered":"2024-04-20T10:29:26.945696+0000","last_clean":"2024-04-20T10:29:26.945696+0000","last_became_active":"2024-04-20T10:29:25.937257+0000","last_became_peered":"2024-04-20T10:29:25.937257+0000","last_unstale":"2024-04-20T10:29:26.945696+0000","last_undegraded":"2024-04-20T10:29:26.945696+0000","last_fullsized":"2024-04-20T10:29:26.945696+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:30.778546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941060+0000","last_change":"2024-04-20T10:29:24.908508+0000","last_active":"2024-04-20T10:29:26.941060+0000","last_peered":"2024-04-20T10:29:26.941060+0000","last_clean":"2024-04-20T10:29:26.941060+0000","last_became_active":"2024-04-20T10:29:24.908247+0000","last_became_peered":"2024-04-20T10:29:24.908247+0000","last_unstale":"2024-04-20T10:29:26.941060+0000","last_undegraded":"2024-04-20T10:29:26.941060+0000","last_fullsized":"2024-04-20T10:29:26.941060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945286+0000","last_change":"2024-04-20T10:29:25.930392+0000","last_active":"2024-04-20T10:29:26.945286+0000","last_peered":"2024-04-20T10:29:26.945286+0000","last_clean":"2024-04-20T10:29:26.945286+0000","last_became_active":"2024-04-20T10:29:25.929589+0000","last_became_peered":"2024-04-20T10:29:25.929589+0000","last_unstale":"2024-04-20T10:29:26.945286+0000","last_undegraded":"2024-04-20T10:29:26.945286+0000","last_fullsized":"2024-04-20T10:29:26.945286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:51.314342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941152+0000","last_change":"2024-04-20T10:29:24.902051+0000","last_active":"2024-04-20T10:29:26.941152+0000","last_peered":"2024-04-20T10:29:26.941152+0000","last_clean":"2024-04-20T10:29:26.941152+0000","last_became_active":"2024-04-20T10:29:24.899974+0000","last_became_peered":"2024-04-20T10:29:24.899974+0000","last_unstale":"2024-04-20T10:29:26.941152+0000","last_undegraded":"2024-04-20T10:29:26.941152+0000","last_fullsized":"2024-04-20T10:29:26.941152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941292+0000","last_change":"2024-04-20T10:29:25.938506+0000","last_active":"2024-04-20T10:29:26.941292+0000","last_peered":"2024-04-20T10:29:26.941292+0000","last_clean":"2024-04-20T10:29:26.941292+0000","last_became_active":"2024-04-20T10:29:25.930725+0000","last_became_peered":"2024-04-20T10:29:25.930725+0000","last_unstale":"2024-04-20T10:29:26.941292+0000","last_undegraded":"2024-04-20T10:29:26.941292+0000","last_fullsized":"2024-04-20T10:29:26.941292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:25.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914740+0000","last_change":"2024-04-20T10:29:24.906546+0000","last_active":"2024-04-20T10:29:26.914740+0000","last_peered":"2024-04-20T10:29:26.914740+0000","last_clean":"2024-04-20T10:29:26.914740+0000","last_became_active":"2024-04-20T10:29:24.906232+0000","last_became_peered":"2024-04-20T10:29:24.906232+0000","last_unstale":"2024-04-20T10:29:26.914740+0000","last_undegraded":"2024-04-20T10:29:26.914740+0000","last_fullsized":"2024-04-20T10:29:26.914740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:55:18.848874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912573+0000","last_change":"2024-04-20T10:29:25.931931+0000","last_active":"2024-04-20T10:29:26.912573+0000","last_peered":"2024-04-20T10:29:26.912573+0000","last_clean":"2024-04-20T10:29:26.912573+0000","last_became_active":"2024-04-20T10:29:25.930641+0000","last_became_peered":"2024-04-20T10:29:25.930641+0000","last_unstale":"2024-04-20T10:29:26.912573+0000","last_undegraded":"2024-04-20T10:29:26.912573+0000","last_fullsized":"2024-04-20T10:29:26.912573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:29.219376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944374+0000","last_change":"2024-04-20T10:29:24.904088+0000","last_active":"2024-04-20T10:29:26.944374+0000","last_peered":"2024-04-20T10:29:26.944374+0000","last_clean":"2024-04-20T10:29:26.944374+0000","last_became_active":"2024-04-20T10:29:24.903794+0000","last_became_peered":"2024-04-20T10:29:24.903794+0000","last_unstale":"2024-04-20T10:29:26.944374+0000","last_undegraded":"2024-04-20T10:29:26.944374+0000","last_fullsized":"2024-04-20T10:29:26.944374+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:22:48.209250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945188+0000","last_change":"2024-04-20T10:29:25.926414+0000","last_active":"2024-04-20T10:29:26.945188+0000","last_peered":"2024-04-20T10:29:26.945188+0000","last_clean":"2024-04-20T10:29:26.945188+0000","last_became_active":"2024-04-20T10:29:25.925823+0000","last_became_peered":"2024-04-20T10:29:25.925823+0000","last_unstale":"2024-04-20T10:29:26.945188+0000","last_undegraded":"2024-04-20T10:29:26.945188+0000","last_fullsized":"2024-04-20T10:29:26.945188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:10.326318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914612+0000","last_change":"2024-04-20T10:29:24.885719+0000","last_active":"2024-04-20T10:29:26.914612+0000","last_peered":"2024-04-20T10:29:26.914612+0000","last_clean":"2024-04-20T10:29:26.914612+0000","last_became_active":"2024-04-20T10:29:24.885350+0000","last_became_peered":"2024-04-20T10:29:24.885350+0000","last_unstale":"2024-04-20T10:29:26.914612+0000","last_undegraded":"2024-04-20T10:29:26.914612+0000","last_fullsized":"2024-04-20T10:29:26.914612+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:23.752255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946847+0000","last_change":"2024-04-20T10:29:25.930977+0000","last_active":"2024-04-20T10:29:26.946847+0000","last_peered":"2024-04-20T10:29:26.946847+0000","last_clean":"2024-04-20T10:29:26.946847+0000","last_became_active":"2024-04-20T10:29:25.930670+0000","last_became_peered":"2024-04-20T10:29:25.930670+0000","last_unstale":"2024-04-20T10:29:26.946847+0000","last_undegraded":"2024-04-20T10:29:26.946847+0000","last_fullsized":"2024-04-20T10:29:26.946847+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:52:04.011278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946370+0000","last_change":"2024-04-20T10:29:24.908349+0000","last_active":"2024-04-20T10:29:26.946370+0000","last_peered":"2024-04-20T10:29:26.946370+0000","last_clean":"2024-04-20T10:29:26.946370+0000","last_became_active":"2024-04-20T10:29:24.908024+0000","last_became_peered":"2024-04-20T10:29:24.908024+0000","last_unstale":"2024-04-20T10:29:26.946370+0000","last_undegraded":"2024-04-20T10:29:26.946370+0000","last_fullsized":"2024-04-20T10:29:26.946370+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:32.664127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940678+0000","last_change":"2024-04-20T10:29:25.948527+0000","last_active":"2024-04-20T10:29:26.940678+0000","last_peered":"2024-04-20T10:29:26.940678+0000","last_clean":"2024-04-20T10:29:26.940678+0000","last_became_active":"2024-04-20T10:29:25.947562+0000","last_became_peered":"2024-04-20T10:29:25.947562+0000","last_unstale":"2024-04-20T10:29:26.940678+0000","last_undegraded":"2024-04-20T10:29:26.940678+0000","last_fullsized":"2024-04-20T10:29:26.940678+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:46.065918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913390+0000","last_change":"2024-04-20T10:29:24.891333+0000","last_active":"2024-04-20T10:29:26.913390+0000","last_peered":"2024-04-20T10:29:26.913390+0000","last_clean":"2024-04-20T10:29:26.913390+0000","last_became_active":"2024-04-20T10:29:24.890971+0000","last_became_peered":"2024-04-20T10:29:24.890971+0000","last_unstale":"2024-04-20T10:29:26.913390+0000","last_undegraded":"2024-04-20T10:29:26.913390+0000","last_fullsized":"2024-04-20T10:29:26.913390+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:56.588805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945083+0000","last_change":"2024-04-20T10:29:25.924543+0000","last_active":"2024-04-20T10:29:26.945083+0000","last_peered":"2024-04-20T10:29:26.945083+0000","last_clean":"2024-04-20T10:29:26.945083+0000","last_became_active":"2024-04-20T10:29:25.924057+0000","last_became_peered":"2024-04-20T10:29:25.924057+0000","last_unstale":"2024-04-20T10:29:26.945083+0000","last_undegraded":"2024-04-20T10:29:26.945083+0000","last_fullsized":"2024-04-20T10:29:26.945083+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:16:48.171394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"41'3","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912856+0000","last_change":"2024-04-20T10:29:24.906234+0000","last_active":"2024-04-20T10:29:26.912856+0000","last_peered":"2024-04-20T10:29:26.912856+0000","last_clean":"2024-04-20T10:29:26.912856+0000","last_became_active":"2024-04-20T10:29:24.905929+0000","last_became_peered":"2024-04-20T10:29:24.905929+0000","last_unstale":"2024-04-20T10:29:26.912856+0000","last_undegraded":"2024-04-20T10:29:26.912856+0000","last_fullsized":"2024-04-20T10:29:26.912856+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:51.866665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915813+0000","last_change":"2024-04-20T10:29:25.924240+0000","last_active":"2024-04-20T10:29:26.915813+0000","last_peered":"2024-04-20T10:29:26.915813+0000","last_clean":"2024-04-20T10:29:26.915813+0000","last_became_active":"2024-04-20T10:29:25.923953+0000","last_became_peered":"2024-04-20T10:29:25.923953+0000","last_unstale":"2024-04-20T10:29:26.915813+0000","last_undegraded":"2024-04-20T10:29:26.915813+0000","last_fullsized":"2024-04-20T10:29:26.915813+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:50.706951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"41'6","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946463+0000","last_change":"2024-04-20T10:29:24.903312+0000","last_active":"2024-04-20T10:29:26.946463+0000","last_peered":"2024-04-20T10:29:26.946463+0000","last_clean":"2024-04-20T10:29:26.946463+0000","last_became_active":"2024-04-20T10:29:24.901601+0000","last_became_peered":"2024-04-20T10:29:24.901601+0000","last_unstale":"2024-04-20T10:29:26.946463+0000","last_undegraded":"2024-04-20T10:29:26.946463+0000","last_fullsized":"2024-04-20T10:29:26.946463+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:41.293656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945464+0000","last_change":"2024-04-20T10:29:25.937578+0000","last_active":"2024-04-20T10:29:26.945464+0000","last_peered":"2024-04-20T10:29:26.945464+0000","last_clean":"2024-04-20T10:29:26.945464+0000","last_became_active":"2024-04-20T10:29:25.936909+0000","last_became_peered":"2024-04-20T10:29:25.936909+0000","last_unstale":"2024-04-20T10:29:26.945464+0000","last_undegraded":"2024-04-20T10:29:26.945464+0000","last_fullsized":"2024-04-20T10:29:26.945464+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:04:15.256897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945492+0000","last_change":"2024-04-20T10:29:24.907284+0000","last_active":"2024-04-20T10:29:26.945492+0000","last_peered":"2024-04-20T10:29:26.945492+0000","last_clean":"2024-04-20T10:29:26.945492+0000","last_became_active":"2024-04-20T10:29:24.906902+0000","last_became_peered":"2024-04-20T10:29:24.906902+0000","last_unstale":"2024-04-20T10:29:26.945492+0000","last_undegraded":"2024-04-20T10:29:26.945492+0000","last_fullsized":"2024-04-20T10:29:26.945492+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:56.695798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944848+0000","last_change":"2024-04-20T10:29:25.921480+0000","last_active":"2024-04-20T10:29:26.944848+0000","last_peered":"2024-04-20T10:29:26.944848+0000","last_clean":"2024-04-20T10:29:26.944848+0000","last_became_active":"2024-04-20T10:29:25.921116+0000","last_became_peered":"2024-04-20T10:29:25.921116+0000","last_unstale":"2024-04-20T10:29:26.944848+0000","last_undegraded":"2024-04-20T10:29:26.944848+0000","last_fullsized":"2024-04-20T10:29:26.944848+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:25.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944288+0000","last_change":"2024-04-20T10:29:24.893579+0000","last_active":"2024-04-20T10:29:26.944288+0000","last_peered":"2024-04-20T10:29:26.944288+0000","last_clean":"2024-04-20T10:29:26.944288+0000","last_became_active":"2024-04-20T10:29:24.893233+0000","last_became_peered":"2024-04-20T10:29:24.893233+0000","last_unstale":"2024-04-20T10:29:26.944288+0000","last_undegraded":"2024-04-20T10:29:26.944288+0000","last_fullsized":"2024-04-20T10:29:26.944288+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942052+0000","last_change":"2024-04-20T10:29:25.932249+0000","last_active":"2024-04-20T10:29:26.942052+0000","last_peered":"2024-04-20T10:29:26.942052+0000","last_clean":"2024-04-20T10:29:26.942052+0000","last_became_active":"2024-04-20T10:29:25.931894+0000","last_became_peered":"2024-04-20T10:29:25.931894+0000","last_unstale":"2024-04-20T10:29:26.942052+0000","last_undegraded":"2024-04-20T10:29:26.942052+0000","last_fullsized":"2024-04-20T10:29:26.942052+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:36.910313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912319+0000","last_change":"2024-04-20T10:29:24.891939+0000","last_active":"2024-04-20T10:29:26.912319+0000","last_peered":"2024-04-20T10:29:26.912319+0000","last_clean":"2024-04-20T10:29:26.912319+0000","last_became_active":"2024-04-20T10:29:24.891666+0000","last_became_peered":"2024-04-20T10:29:24.891666+0000","last_unstale":"2024-04-20T10:29:26.912319+0000","last_undegraded":"2024-04-20T10:29:26.912319+0000","last_fullsized":"2024-04-20T10:29:26.912319+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:57.652260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918271+0000","last_change":"2024-04-20T10:29:25.932363+0000","last_active":"2024-04-20T10:29:26.918271+0000","last_peered":"2024-04-20T10:29:26.918271+0000","last_clean":"2024-04-20T10:29:26.918271+0000","last_became_active":"2024-04-20T10:29:25.931947+0000","last_became_peered":"2024-04-20T10:29:25.931947+0000","last_unstale":"2024-04-20T10:29:26.918271+0000","last_undegraded":"2024-04-20T10:29:26.918271+0000","last_fullsized":"2024-04-20T10:29:26.918271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:13.678333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"41'8","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940756+0000","last_change":"2024-04-20T10:29:24.890425+0000","last_active":"2024-04-20T10:29:26.940756+0000","last_peered":"2024-04-20T10:29:26.940756+0000","last_clean":"2024-04-20T10:29:26.940756+0000","last_became_active":"2024-04-20T10:29:24.890115+0000","last_became_peered":"2024-04-20T10:29:24.890115+0000","last_unstale":"2024-04-20T10:29:26.940756+0000","last_undegraded":"2024-04-20T10:29:26.940756+0000","last_fullsized":"2024-04-20T10:29:26.940756+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:50.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943993+0000","last_change":"2024-04-20T10:29:25.926540+0000","last_active":"2024-04-20T10:29:26.943993+0000","last_peered":"2024-04-20T10:29:26.943993+0000","last_clean":"2024-04-20T10:29:26.943993+0000","last_became_active":"2024-04-20T10:29:25.926226+0000","last_became_peered":"2024-04-20T10:29:25.926226+0000","last_unstale":"2024-04-20T10:29:26.943993+0000","last_undegraded":"2024-04-20T10:29:26.943993+0000","last_fullsized":"2024-04-20T10:29:26.943993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:45.751412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915736+0000","last_change":"2024-04-20T10:29:24.886285+0000","last_active":"2024-04-20T10:29:26.915736+0000","last_peered":"2024-04-20T10:29:26.915736+0000","last_clean":"2024-04-20T10:29:26.915736+0000","last_became_active":"2024-04-20T10:29:24.886016+0000","last_became_peered":"2024-04-20T10:29:24.886016+0000","last_unstale":"2024-04-20T10:29:26.915736+0000","last_undegraded":"2024-04-20T10:29:26.915736+0000","last_fullsized":"2024-04-20T10:29:26.915736+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:01.416574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915407+0000","last_change":"2024-04-20T10:29:25.929424+0000","last_active":"2024-04-20T10:29:26.915407+0000","last_peered":"2024-04-20T10:29:26.915407+0000","last_clean":"2024-04-20T10:29:26.915407+0000","last_became_active":"2024-04-20T10:29:25.929120+0000","last_became_peered":"2024-04-20T10:29:25.929120+0000","last_unstale":"2024-04-20T10:29:26.915407+0000","last_undegraded":"2024-04-20T10:29:26.915407+0000","last_fullsized":"2024-04-20T10:29:26.915407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:13.678333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913623+0000","last_change":"2024-04-20T10:29:24.907760+0000","last_active":"2024-04-20T10:29:26.913623+0000","last_peered":"2024-04-20T10:29:26.913623+0000","last_clean":"2024-04-20T10:29:26.913623+0000","last_became_active":"2024-04-20T10:29:24.907189+0000","last_became_peered":"2024-04-20T10:29:24.907189+0000","last_unstale":"2024-04-20T10:29:26.913623+0000","last_undegraded":"2024-04-20T10:29:26.913623+0000","last_fullsized":"2024-04-20T10:29:26.913623+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:50:55.174213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915724+0000","last_change":"2024-04-20T10:29:25.928928+0000","last_active":"2024-04-20T10:29:26.915724+0000","last_peered":"2024-04-20T10:29:26.915724+0000","last_clean":"2024-04-20T10:29:26.915724+0000","last_became_active":"2024-04-20T10:29:25.928646+0000","last_became_peered":"2024-04-20T10:29:25.928646+0000","last_unstale":"2024-04-20T10:29:26.915724+0000","last_undegraded":"2024-04-20T10:29:26.915724+0000","last_fullsized":"2024-04-20T10:29:26.915724+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:51.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917029+0000","last_change":"2024-04-20T10:29:24.902160+0000","last_active":"2024-04-20T10:29:26.917029+0000","last_peered":"2024-04-20T10:29:26.917029+0000","last_clean":"2024-04-20T10:29:26.917029+0000","last_became_active":"2024-04-20T10:29:24.900595+0000","last_became_peered":"2024-04-20T10:29:24.900595+0000","last_unstale":"2024-04-20T10:29:26.917029+0000","last_undegraded":"2024-04-20T10:29:26.917029+0000","last_fullsized":"2024-04-20T10:29:26.917029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:56.719517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943197+0000","last_change":"2024-04-20T10:29:25.930028+0000","last_active":"2024-04-20T10:29:26.943197+0000","last_peered":"2024-04-20T10:29:26.943197+0000","last_clean":"2024-04-20T10:29:26.943197+0000","last_became_active":"2024-04-20T10:29:25.929615+0000","last_became_peered":"2024-04-20T10:29:25.929615+0000","last_unstale":"2024-04-20T10:29:26.943197+0000","last_undegraded":"2024-04-20T10:29:26.943197+0000","last_fullsized":"2024-04-20T10:29:26.943197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:18.438200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945558+0000","last_change":"2024-04-20T10:29:24.896677+0000","last_active":"2024-04-20T10:29:26.945558+0000","last_peered":"2024-04-20T10:29:26.945558+0000","last_clean":"2024-04-20T10:29:26.945558+0000","last_became_active":"2024-04-20T10:29:24.896023+0000","last_became_peered":"2024-04-20T10:29:24.896023+0000","last_unstale":"2024-04-20T10:29:26.945558+0000","last_undegraded":"2024-04-20T10:29:26.945558+0000","last_fullsized":"2024-04-20T10:29:26.945558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:24.848053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.911955+0000","last_change":"2024-04-20T10:29:25.910686+0000","last_active":"2024-04-20T10:29:26.911955+0000","last_peered":"2024-04-20T10:29:26.911955+0000","last_clean":"2024-04-20T10:29:26.911955+0000","last_became_active":"2024-04-20T10:29:25.910346+0000","last_became_peered":"2024-04-20T10:29:25.910346+0000","last_unstale":"2024-04-20T10:29:26.911955+0000","last_undegraded":"2024-04-20T10:29:26.911955+0000","last_fullsized":"2024-04-20T10:29:26.911955+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:03.878269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942552+0000","last_change":"2024-04-20T10:29:24.903532+0000","last_active":"2024-04-20T10:29:26.942552+0000","last_peered":"2024-04-20T10:29:26.942552+0000","last_clean":"2024-04-20T10:29:26.942552+0000","last_became_active":"2024-04-20T10:29:24.902068+0000","last_became_peered":"2024-04-20T10:29:24.902068+0000","last_unstale":"2024-04-20T10:29:26.942552+0000","last_undegraded":"2024-04-20T10:29:26.942552+0000","last_fullsized":"2024-04-20T10:29:26.942552+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:10.606850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944230+0000","last_change":"2024-04-20T10:29:25.923660+0000","last_active":"2024-04-20T10:29:26.944230+0000","last_peered":"2024-04-20T10:29:26.944230+0000","last_clean":"2024-04-20T10:29:26.944230+0000","last_became_active":"2024-04-20T10:29:25.922736+0000","last_became_peered":"2024-04-20T10:29:25.922736+0000","last_unstale":"2024-04-20T10:29:26.944230+0000","last_undegraded":"2024-04-20T10:29:26.944230+0000","last_fullsized":"2024-04-20T10:29:26.944230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:18.551480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944380+0000","last_change":"2024-04-20T10:29:24.892163+0000","last_active":"2024-04-20T10:29:26.944380+0000","last_peered":"2024-04-20T10:29:26.944380+0000","last_clean":"2024-04-20T10:29:26.944380+0000","last_became_active":"2024-04-20T10:29:24.891419+0000","last_became_peered":"2024-04-20T10:29:24.891419+0000","last_unstale":"2024-04-20T10:29:26.944380+0000","last_undegraded":"2024-04-20T10:29:26.944380+0000","last_fullsized":"2024-04-20T10:29:26.944380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:35.313612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941617+0000","last_change":"2024-04-20T10:29:25.923502+0000","last_active":"2024-04-20T10:29:26.941617+0000","last_peered":"2024-04-20T10:29:26.941617+0000","last_clean":"2024-04-20T10:29:26.941617+0000","last_became_active":"2024-04-20T10:29:25.923213+0000","last_became_peered":"2024-04-20T10:29:25.923213+0000","last_unstale":"2024-04-20T10:29:26.941617+0000","last_undegraded":"2024-04-20T10:29:26.941617+0000","last_fullsized":"2024-04-20T10:29:26.941617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:12.260654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915324+0000","last_change":"2024-04-20T10:29:24.883849+0000","last_active":"2024-04-20T10:29:26.915324+0000","last_peered":"2024-04-20T10:29:26.915324+0000","last_clean":"2024-04-20T10:29:26.915324+0000","last_became_active":"2024-04-20T10:29:24.883436+0000","last_became_peered":"2024-04-20T10:29:24.883436+0000","last_unstale":"2024-04-20T10:29:26.915324+0000","last_undegraded":"2024-04-20T10:29:26.915324+0000","last_fullsized":"2024-04-20T10:29:26.915324+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:33.581245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945390+0000","last_change":"2024-04-20T10:29:25.925933+0000","last_active":"2024-04-20T10:29:26.945390+0000","last_peered":"2024-04-20T10:29:26.945390+0000","last_clean":"2024-04-20T10:29:26.945390+0000","last_became_active":"2024-04-20T10:29:25.925357+0000","last_became_peered":"2024-04-20T10:29:25.925357+0000","last_unstale":"2024-04-20T10:29:26.945390+0000","last_undegraded":"2024-04-20T10:29:26.945390+0000","last_fullsized":"2024-04-20T10:29:26.945390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:29.921464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943075+0000","last_change":"2024-04-20T10:29:24.902461+0000","last_active":"2024-04-20T10:29:26.943075+0000","last_peered":"2024-04-20T10:29:26.943075+0000","last_clean":"2024-04-20T10:29:26.943075+0000","last_became_active":"2024-04-20T10:29:24.901505+0000","last_became_peered":"2024-04-20T10:29:24.901505+0000","last_unstale":"2024-04-20T10:29:26.943075+0000","last_undegraded":"2024-04-20T10:29:26.943075+0000","last_fullsized":"2024-04-20T10:29:26.943075+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917697+0000","last_change":"2024-04-20T10:29:25.938482+0000","last_active":"2024-04-20T10:29:26.917697+0000","last_peered":"2024-04-20T10:29:26.917697+0000","last_clean":"2024-04-20T10:29:26.917697+0000","last_became_active":"2024-04-20T10:29:25.938109+0000","last_became_peered":"2024-04-20T10:29:25.938109+0000","last_unstale":"2024-04-20T10:29:26.917697+0000","last_undegraded":"2024-04-20T10:29:26.917697+0000","last_fullsized":"2024-04-20T10:29:26.917697+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:28.165829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913505+0000","last_change":"2024-04-20T10:29:24.899608+0000","last_active":"2024-04-20T10:29:26.913505+0000","last_peered":"2024-04-20T10:29:26.913505+0000","last_clean":"2024-04-20T10:29:26.913505+0000","last_became_active":"2024-04-20T10:29:24.899329+0000","last_became_peered":"2024-04-20T10:29:24.899329+0000","last_unstale":"2024-04-20T10:29:26.913505+0000","last_undegraded":"2024-04-20T10:29:26.913505+0000","last_fullsized":"2024-04-20T10:29:26.913505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:52.861600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942716+0000","last_change":"2024-04-20T10:29:25.923433+0000","last_active":"2024-04-20T10:29:26.942716+0000","last_peered":"2024-04-20T10:29:26.942716+0000","last_clean":"2024-04-20T10:29:26.942716+0000","last_became_active":"2024-04-20T10:29:25.923090+0000","last_became_peered":"2024-04-20T10:29:25.923090+0000","last_unstale":"2024-04-20T10:29:26.942716+0000","last_undegraded":"2024-04-20T10:29:26.942716+0000","last_fullsized":"2024-04-20T10:29:26.942716+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:52.880434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"42'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918090+0000","last_change":"2024-04-20T10:29:24.901189+0000","last_active":"2024-04-20T10:29:26.918090+0000","last_peered":"2024-04-20T10:29:26.918090+0000","last_clean":"2024-04-20T10:29:26.918090+0000","last_became_active":"2024-04-20T10:29:24.900748+0000","last_became_peered":"2024-04-20T10:29:24.900748+0000","last_unstale":"2024-04-20T10:29:26.918090+0000","last_undegraded":"2024-04-20T10:29:26.918090+0000","last_fullsized":"2024-04-20T10:29:26.918090+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:26:34.351089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942867+0000","last_change":"2024-04-20T10:29:25.941420+0000","last_active":"2024-04-20T10:29:26.942867+0000","last_peered":"2024-04-20T10:29:26.942867+0000","last_clean":"2024-04-20T10:29:26.942867+0000","last_became_active":"2024-04-20T10:29:25.941118+0000","last_became_peered":"2024-04-20T10:29:25.941118+0000","last_unstale":"2024-04-20T10:29:26.942867+0000","last_undegraded":"2024-04-20T10:29:26.942867+0000","last_fullsized":"2024-04-20T10:29:26.942867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:45.751412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917334+0000","last_change":"2024-04-20T10:29:24.907398+0000","last_active":"2024-04-20T10:29:26.917334+0000","last_peered":"2024-04-20T10:29:26.917334+0000","last_clean":"2024-04-20T10:29:26.917334+0000","last_became_active":"2024-04-20T10:29:24.907118+0000","last_became_peered":"2024-04-20T10:29:24.907118+0000","last_unstale":"2024-04-20T10:29:26.917334+0000","last_undegraded":"2024-04-20T10:29:26.917334+0000","last_fullsized":"2024-04-20T10:29:26.917334+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:08.982478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917476+0000","last_change":"2024-04-20T10:29:25.928091+0000","last_active":"2024-04-20T10:29:26.917476+0000","last_peered":"2024-04-20T10:29:26.917476+0000","last_clean":"2024-04-20T10:29:26.917476+0000","last_became_active":"2024-04-20T10:29:25.927758+0000","last_became_peered":"2024-04-20T10:29:25.927758+0000","last_unstale":"2024-04-20T10:29:26.917476+0000","last_undegraded":"2024-04-20T10:29:26.917476+0000","last_fullsized":"2024-04-20T10:29:26.917476+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:23.794826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945688+0000","last_change":"2024-04-20T10:29:24.906505+0000","last_active":"2024-04-20T10:29:26.945688+0000","last_peered":"2024-04-20T10:29:26.945688+0000","last_clean":"2024-04-20T10:29:26.945688+0000","last_became_active":"2024-04-20T10:29:24.906039+0000","last_became_peered":"2024-04-20T10:29:24.906039+0000","last_unstale":"2024-04-20T10:29:26.945688+0000","last_undegraded":"2024-04-20T10:29:26.945688+0000","last_fullsized":"2024-04-20T10:29:26.945688+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:41.357671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945214+0000","last_change":"2024-04-20T10:29:25.918637+0000","last_active":"2024-04-20T10:29:26.945214+0000","last_peered":"2024-04-20T10:29:26.945214+0000","last_clean":"2024-04-20T10:29:26.945214+0000","last_became_active":"2024-04-20T10:29:25.918152+0000","last_became_peered":"2024-04-20T10:29:25.918152+0000","last_unstale":"2024-04-20T10:29:26.945214+0000","last_undegraded":"2024-04-20T10:29:26.945214+0000","last_fullsized":"2024-04-20T10:29:26.945214+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:12.260654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941907+0000","last_change":"2024-04-20T10:29:24.900706+0000","last_active":"2024-04-20T10:29:26.941907+0000","last_peered":"2024-04-20T10:29:26.941907+0000","last_clean":"2024-04-20T10:29:26.941907+0000","last_became_active":"2024-04-20T10:29:24.900256+0000","last_became_peered":"2024-04-20T10:29:24.900256+0000","last_unstale":"2024-04-20T10:29:26.941907+0000","last_undegraded":"2024-04-20T10:29:26.941907+0000","last_fullsized":"2024-04-20T10:29:26.941907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:25.116742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912720+0000","last_change":"2024-04-20T10:29:25.926715+0000","last_active":"2024-04-20T10:29:26.912720+0000","last_peered":"2024-04-20T10:29:26.912720+0000","last_clean":"2024-04-20T10:29:26.912720+0000","last_became_active":"2024-04-20T10:29:25.926406+0000","last_became_peered":"2024-04-20T10:29:25.926406+0000","last_unstale":"2024-04-20T10:29:26.912720+0000","last_undegraded":"2024-04-20T10:29:26.912720+0000","last_fullsized":"2024-04-20T10:29:26.912720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:01.539035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912415+0000","last_change":"2024-04-20T10:29:24.901083+0000","last_active":"2024-04-20T10:29:26.912415+0000","last_peered":"2024-04-20T10:29:26.912415+0000","last_clean":"2024-04-20T10:29:26.912415+0000","last_became_active":"2024-04-20T10:29:24.899838+0000","last_became_peered":"2024-04-20T10:29:24.899838+0000","last_unstale":"2024-04-20T10:29:26.912415+0000","last_undegraded":"2024-04-20T10:29:26.912415+0000","last_fullsized":"2024-04-20T10:29:26.912415+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:28.810260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942078+0000","last_change":"2024-04-20T10:29:25.934874+0000","last_active":"2024-04-20T10:29:26.942078+0000","last_peered":"2024-04-20T10:29:26.942078+0000","last_clean":"2024-04-20T10:29:26.942078+0000","last_became_active":"2024-04-20T10:29:25.934573+0000","last_became_peered":"2024-04-20T10:29:25.934573+0000","last_unstale":"2024-04-20T10:29:26.942078+0000","last_undegraded":"2024-04-20T10:29:26.942078+0000","last_fullsized":"2024-04-20T10:29:26.942078+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:30.778546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945567+0000","last_change":"2024-04-20T10:29:24.903773+0000","last_active":"2024-04-20T10:29:26.945567+0000","last_peered":"2024-04-20T10:29:26.945567+0000","last_clean":"2024-04-20T10:29:26.945567+0000","last_became_active":"2024-04-20T10:29:24.903463+0000","last_became_peered":"2024-04-20T10:29:24.903463+0000","last_unstale":"2024-04-20T10:29:26.945567+0000","last_undegraded":"2024-04-20T10:29:26.945567+0000","last_fullsized":"2024-04-20T10:29:26.945567+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:02:26.875257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913400+0000","last_change":"2024-04-20T10:29:25.926885+0000","last_active":"2024-04-20T10:29:26.913400+0000","last_peered":"2024-04-20T10:29:26.913400+0000","last_clean":"2024-04-20T10:29:26.913400+0000","last_became_active":"2024-04-20T10:29:25.926601+0000","last_became_peered":"2024-04-20T10:29:25.926601+0000","last_unstale":"2024-04-20T10:29:26.913400+0000","last_undegraded":"2024-04-20T10:29:26.913400+0000","last_fullsized":"2024-04-20T10:29:26.913400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:43:49.676376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942300+0000","last_change":"2024-04-20T10:29:24.900850+0000","last_active":"2024-04-20T10:29:26.942300+0000","last_peered":"2024-04-20T10:29:26.942300+0000","last_clean":"2024-04-20T10:29:26.942300+0000","last_became_active":"2024-04-20T10:29:24.898940+0000","last_became_peered":"2024-04-20T10:29:24.898940+0000","last_unstale":"2024-04-20T10:29:26.942300+0000","last_undegraded":"2024-04-20T10:29:26.942300+0000","last_fullsized":"2024-04-20T10:29:26.942300+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:35.313612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":208,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915002+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:29:26.915002+0000","last_peered":"2024-04-20T10:29:26.915002+0000","last_clean":"2024-04-20T10:29:26.915002+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:29:26.915002+0000","last_undegraded":"2024-04-20T10:29:26.915002+0000","last_fullsized":"2024-04-20T10:29:26.915002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":205348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":205312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1077248,"data_stored":796228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":205368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":207472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1077248,"data_stored":795668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":487424,"data_stored":205370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":207014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:29:31.370 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-20T10:29:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.626+0000 7f9869800640 1 Processor -- start 2024-04-20T10:29:31.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.626+0000 7f9869800640 1 -- start start 2024-04-20T10:29:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.627+0000 7f9869800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9864162c00 0x7f98640d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.627+0000 7f9869800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f986406f090 0x7f98640d08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.627+0000 7f9869800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f986406fc30 0x7f986406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.627+0000 7f9869800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9864162fe0 con 0x7f986406fc30 2024-04-20T10:29:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.627+0000 7f9869800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f986406ec00 con 0x7f986406f090 2024-04-20T10:29:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.627+0000 7f9869800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f986406df70 con 0x7f9864162c00 2024-04-20T10:29:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.627+0000 7f9863400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f986406fc30 0x7f986406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9863400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f986406fc30 0x7f986406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34740/0 (socket says 172.21.15.47:34740) 2024-04-20T10:29:31.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9863400640 1 -- 172.21.15.47:0/390095358 learned_addr learned my addr 172.21.15.47:0/390095358 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9862a00640 1 --2- 172.21.15.47:0/390095358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9864162c00 0x7f98640d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9862000640 1 --2- 172.21.15.47:0/390095358 >> v2:172.21.15.175:3300/0 conn(0x7f986406f090 0x7f98640d08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9863400640 1 -- 172.21.15.47:0/390095358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9864162c00 msgr2=0x7f98640d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9863400640 1 --2- 172.21.15.47:0/390095358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9864162c00 0x7f98640d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9863400640 1 -- 172.21.15.47:0/390095358 >> v2:172.21.15.175:3300/0 conn(0x7f986406f090 msgr2=0x7f98640d08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9863400640 1 --2- 172.21.15.47:0/390095358 >> v2:172.21.15.175:3300/0 conn(0x7f986406f090 0x7f98640d08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.628+0000 7f9863400640 1 -- 172.21.15.47:0/390095358 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986406e310 con 0x7f986406fc30 2024-04-20T10:29:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.629+0000 7f9863400640 1 --2- 172.21.15.47:0/390095358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f986406b900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f98640d0f40 tx=0x7f985c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8095a89a408bd9f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9861600640 1 -- 172.21.15.47:0/390095358 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c02e070 con 0x7f986406fc30 2024-04-20T10:29:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9861600640 1 -- 172.21.15.47:0/390095358 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f985c037070 con 0x7f986406fc30 2024-04-20T10:29:31.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9861600640 1 -- 172.21.15.47:0/390095358 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985c031a00 con 0x7f986406fc30 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 -- 172.21.15.47:0/390095358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 msgr2=0x7f986406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 --2- 172.21.15.47:0/390095358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f986406b900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f98640d0f40 tx=0x7f985c02b880 comp rx=0 tx=0).stop 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 -- 172.21.15.47:0/390095358 shutdown_connections 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 --2- 172.21.15.47:0/390095358 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f986406b900 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 --2- 172.21.15.47:0/390095358 >> v2:172.21.15.175:3300/0 conn(0x7f986406f090 0x7f98640d08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 --2- 172.21.15.47:0/390095358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9864162c00 0x7f98640d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 -- 172.21.15.47:0/390095358 >> 172.21.15.47:0/390095358 conn(0x7f98640b3350 msgr2=0x7f98640b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.630+0000 7f9869800640 1 -- 172.21.15.47:0/390095358 shutdown_connections 2024-04-20T10:29:31.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.631+0000 7f9869800640 1 -- 172.21.15.47:0/390095358 wait complete. 2024-04-20T10:29:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.632+0000 7f9869800640 1 Processor -- start 2024-04-20T10:29:31.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.632+0000 7f9869800640 1 -- start start 2024-04-20T10:29:31.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.633+0000 7f9869800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986406f090 0x7f9864155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.633+0000 7f9869800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f9864156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.633+0000 7f9869800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864162c00 0x7f9864156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.633+0000 7f9869800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f986406dbb0 con 0x7f986406fc30 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.633+0000 7f9869800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f986406e890 con 0x7f9864162c00 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.633+0000 7f9869800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f986406dd30 con 0x7f986406f090 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f9864156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f9864156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34742/0 (socket says 172.21.15.47:34742) 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 -- 172.21.15.47:0/2484995481 learned_addr learned my addr 172.21.15.47:0/2484995481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9863400640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864162c00 0x7f9864156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862a00640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986406f090 0x7f9864155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 -- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986406f090 msgr2=0x7f9864155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986406f090 0x7f9864155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 -- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864162c00 msgr2=0x7f9864156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864162c00 0x7f9864156a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.634+0000 7f9862000640 1 -- 172.21.15.47:0/2484995481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9864163210 con 0x7f986406fc30 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.635+0000 7f9862000640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f9864156530 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f984c007e40 tx=0x7f984c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.635+0000 7f985aa00640 1 -- 172.21.15.47:0/2484995481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c014620 con 0x7f986406fc30 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.635+0000 7f985aa00640 1 -- 172.21.15.47:0/2484995481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f984c019070 con 0x7f986406fc30 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.635+0000 7f985aa00640 1 -- 172.21.15.47:0/2484995481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f984c014a00 con 0x7f986406fc30 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.635+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98641587f0 con 0x7f986406fc30 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.635+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9864156010 con 0x7f986406fc30 2024-04-20T10:29:31.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.636+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9864068980 con 0x7f986406fc30 2024-04-20T10:29:31.637 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.637+0000 7f985aa00640 1 -- 172.21.15.47:0/2484995481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f984c014ba0 con 0x7f986406fc30 2024-04-20T10:29:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.638+0000 7f985aa00640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c920 0x7f984006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.638+0000 7f985aa00640 1 -- 172.21.15.47:0/2484995481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f984c08c5a0 con 0x7f986406fc30 2024-04-20T10:29:31.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.639+0000 7f9862a00640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c920 0x7f984006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:31.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.640+0000 7f9862a00640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c920 0x7f984006ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9854006fd0 tx=0x7f9854008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:31.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.642+0000 7f985aa00640 1 -- 172.21.15.47:0/2484995481 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f984c011260 con 0x7f986406fc30 2024-04-20T10:29:31.764 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-04-20T10:29:31.764+0000 7f3677200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:29:31.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.932+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f9864160f30 con 0x7f986406fc30 2024-04-20T10:29:31.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.934+0000 7f985aa00640 1 -- 172.21.15.47:0/2484995481 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1109 (secure 0 0 0) 0x7f984c05a830 con 0x7f986406fc30 2024-04-20T10:29:31.934 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:31.934 INFO:teuthology.orchestra.run.smithi047.stdout:{"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":144,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":42,"num_osds":8,"num_up_osds":8,"osd_up_since":1713608839,"num_in_osds":8,"osd_in_since":1713608831,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":58},{"state_name":"active+clean","count":26},{"state_name":"unknown","count":13}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":243113984,"bytes_avail":772850999296,"bytes_total":773094113280,"unknown_pgs_ratio":0.1340206116437912,"inactive_pgs_ratio":0.59793812036514282},"fsmap":{"epoch":19,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5481}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-04-20T10:28:39.089931+0000","services":{}},"progress_events":{}} 2024-04-20T10:29:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.936+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c920 msgr2=0x7f984006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.936+0000 7f9869800640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c920 0x7f984006ede0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9854006fd0 tx=0x7f9854008040 comp rx=0 tx=0).stop 2024-04-20T10:29:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.936+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 msgr2=0x7f9864156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.936+0000 7f9869800640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f9864156530 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f984c007e40 tx=0x7f984c00e4b0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 shutdown_connections 2024-04-20T10:29:31.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9864162c00 0x7f9864156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f984006c920 0x7f984006ede0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f986406fc30 0x7f9864156530 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 --2- 172.21.15.47:0/2484995481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f986406f090 0x7f9864155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 >> 172.21.15.47:0/2484995481 conn(0x7f98640b3350 msgr2=0x7f9864160310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 shutdown_connections 2024-04-20T10:29:31.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:31.937+0000 7f9869800640 1 -- 172.21.15.47:0/2484995481 wait complete. 2024-04-20T10:29:31.957 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-20T10:29:31.957 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:29:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.172+0000 7f0039a00640 1 Processor -- start 2024-04-20T10:29:32.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.173+0000 7f0039a00640 1 -- start start 2024-04-20T10:29:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.174+0000 7f0039a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 0x7f00340d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.174+0000 7f0039a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003406c190 con 0x7f0034161d50 2024-04-20T10:29:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.174+0000 7f0039a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f003406c470 con 0x7f00340b3330 2024-04-20T10:29:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.174+0000 7f0039a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0034162130 con 0x7f003406cde0 2024-04-20T10:29:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0033e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0034161d50 0x7f003406c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:36634/0 (socket says 172.21.15.47:36634) 2024-04-20T10:29:32.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0033e00640 1 -- 172.21.15.47:0/111084155 learned_addr learned my addr 172.21.15.47:0/111084155 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4215175138 0 0) 0x7f003406c190 con 0x7f0034161d50 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0014003520 con 0x7f0034161d50 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0033400640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 0x7f00340d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1297388851 0 0) 0x7f0014003520 con 0x7f0034161d50 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 msgr2=0x7f00340d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0032a00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 0x7f00340d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.175+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 >> v1:172.21.15.175:6789/0 conn(0x7f00340b3330 legacy=0x7f00340b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00340d2e00 con 0x7f0034161d50 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f0024001670 con 0x7f0034161d50 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 >> v1:172.21.15.47:6789/0 conn(0x7f0034161d50 legacy=0x7f003406c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00140039e0 0x7f0014003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 0x7f0014004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0014004d40 0x7f00140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f001400bb50 con 0x7f00140039e0 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f001400bcd0 con 0x7f0014004d40 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f001400bea0 con 0x7f003406cde0 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0033e00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00140039e0 0x7f0014003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0033400640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 0x7f0014004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0033e00640 1 -- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 msgr2=0x7f0014004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0033e00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f003406cde0 0x7f0014004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0039000640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0014004d40 0x7f00140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.176+0000 7f0033e00640 1 -- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0014004d40 msgr2=0x7f00140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.177+0000 7f0033e00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0014004d40 0x7f00140051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.177+0000 7f0033e00640 1 -- 172.21.15.47:0/111084155 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0014003520 con 0x7f00140039e0 2024-04-20T10:29:32.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.178+0000 7f0033e00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00140039e0 0x7f0014003e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f002400c270 tx=0x7f0014005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef5e8a0e9205502 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:32.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.178+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024007cf0 con 0x7f00140039e0 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.178+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0024030070 con 0x7f00140039e0 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.178+0000 7f0032a00640 1 -- 172.21.15.47:0/111084155 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0024005d30 con 0x7f00140039e0 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.178+0000 7f0039a00640 1 -- 172.21.15.47:0/111084155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00140039e0 msgr2=0x7f0014003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.178+0000 7f0039a00640 1 --2- 172.21.15.47:0/111084155 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00140039e0 0x7f0014003e00 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f002400c270 tx=0x7f0014005830 comp rx=0 tx=0).stop 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.179+0000 7f0039000640 1 -- 172.21.15.47:0/111084155 reap_dead start 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.179+0000 7f0039a00640 1 -- 172.21.15.47:0/111084155 shutdown_connections 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.179+0000 7f0039a00640 1 -- 172.21.15.47:0/111084155 >> 172.21.15.47:0/111084155 conn(0x7f00340d4450 msgr2=0x7f00340d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.179+0000 7f0039a00640 1 -- 172.21.15.47:0/111084155 shutdown_connections 2024-04-20T10:29:32.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.179+0000 7f0039a00640 1 -- 172.21.15.47:0/111084155 wait complete. 2024-04-20T10:29:32.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.180+0000 7f0039a00640 1 Processor -- start 2024-04-20T10:29:32.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.181+0000 7f0039a00640 1 -- start start 2024-04-20T10:29:32.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0034154230 0x7f00340b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00341f8bb0 0x7f00341f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00341f8530 0x7f003415c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f00340d38e0 con 0x7f00341f8530 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00340d33c0 con 0x7f00341f8bb0 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00340d3a60 con 0x7f0034154230 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00341f8530 0x7f003415c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00341f8530 0x7f003415c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:34766/0 (socket says 172.21.15.47:34766) 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 -- 172.21.15.47:0/4238988182 learned_addr learned my addr 172.21.15.47:0/4238988182 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0033400640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00341f8bb0 0x7f00341f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0033e00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0034154230 0x7f00340b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 -- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0034154230 msgr2=0x7f00340b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0034154230 0x7f00340b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 -- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00341f8bb0 msgr2=0x7f00341f7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00341f8bb0 0x7f00341f7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0039000640 1 -- 172.21.15.47:0/4238988182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00340b3fd0 con 0x7f00341f8530 2024-04-20T10:29:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0033400640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00341f8bb0 0x7f00341f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.182+0000 7f0033e00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0034154230 0x7f00340b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.183+0000 7f0039000640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00341f8530 0x7f003415c430 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f00340ceea0 tx=0x7f002c00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.183+0000 7f002be00640 1 -- 172.21.15.47:0/4238988182 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c018070 con 0x7f00341f8530 2024-04-20T10:29:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.183+0000 7f002be00640 1 -- 172.21.15.47:0/4238988182 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f002c0035f0 con 0x7f00341f8530 2024-04-20T10:29:32.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.183+0000 7f002be00640 1 -- 172.21.15.47:0/4238988182 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0039c0 con 0x7f00341f8530 2024-04-20T10:29:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.183+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00340b65e0 con 0x7f00341f8530 2024-04-20T10:29:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.183+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00340b4f60 con 0x7f00341f8530 2024-04-20T10:29:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.185+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0034075b60 con 0x7f00341f8530 2024-04-20T10:29:32.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.185+0000 7f002be00640 1 -- 172.21.15.47:0/4238988182 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f002c01b070 con 0x7f00341f8530 2024-04-20T10:29:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.186+0000 7f002be00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f001006c8f0 0x7f001006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:32.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.187+0000 7f002be00640 1 -- 172.21.15.47:0/4238988182 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f002c08b350 con 0x7f00341f8530 2024-04-20T10:29:32.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.187+0000 7f0033e00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f001006c8f0 0x7f001006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:32.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.188+0000 7f0033e00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f001006c8f0 0x7f001006edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0024034000 tx=0x7f0024039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:32.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.191+0000 7f002be00640 1 -- 172.21.15.47:0/4238988182 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f002c059550 con 0x7f00341f8530 2024-04-20T10:29:32.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.388+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f00340b4200 con 0x7f001006c8f0 2024-04-20T10:29:32.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.408+0000 7f002be00640 1 -- 172.21.15.47:0/4238988182 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251966 (secure 0 0 0) 0x7f00340b4200 con 0x7f001006c8f0 2024-04-20T10:29:32.412 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:32.413 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:29:32.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f001006c8f0 msgr2=0x7f001006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f001006c8f0 0x7f001006edb0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f0024034000 tx=0x7f0024039040 comp rx=0 tx=0).stop 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00341f8530 msgr2=0x7f003415c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00341f8530 0x7f003415c430 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f00340ceea0 tx=0x7f002c00d890 comp rx=0 tx=0).stop 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 shutdown_connections 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f001006c8f0 0x7f001006edb0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f00341f8530 0x7f003415c430 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00341f8bb0 0x7f00341f7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 --2- 172.21.15.47:0/4238988182 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0034154230 0x7f00340b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.413+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 >> 172.21.15.47:0/4238988182 conn(0x7f00340d4450 msgr2=0x7f00340b8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.414+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 shutdown_connections 2024-04-20T10:29:32.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:32.414+0000 7f0039a00640 1 -- 172.21.15.47:0/4238988182 wait complete. 2024-04-20T10:29:32.433 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-04-20T10:29:31.108884+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237800,"kb_used_data":4968,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736920,"statfs":{"total":773094113280,"available":772850606080,"internally_reserved":0,"allocated":5087232,"data_stored":2827780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":61,"apply_latency_ms":61,"commit_latency_ns":61000000,"apply_latency_ns":61000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.241224"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946274+0000","last_change":"2024-04-20T10:29:25.932997+0000","last_active":"2024-04-20T10:29:26.946274+0000","last_peered":"2024-04-20T10:29:26.946274+0000","last_clean":"2024-04-20T10:29:26.946274+0000","last_became_active":"2024-04-20T10:29:25.932397+0000","last_became_peered":"2024-04-20T10:29:25.932397+0000","last_unstale":"2024-04-20T10:29:26.946274+0000","last_undegraded":"2024-04-20T10:29:26.946274+0000","last_fullsized":"2024-04-20T10:29:26.946274+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:06.846350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945063+0000","last_change":"2024-04-20T10:29:25.915562+0000","last_active":"2024-04-20T10:29:26.945063+0000","last_peered":"2024-04-20T10:29:26.945063+0000","last_clean":"2024-04-20T10:29:26.945063+0000","last_became_active":"2024-04-20T10:29:25.914716+0000","last_became_peered":"2024-04-20T10:29:25.914716+0000","last_unstale":"2024-04-20T10:29:26.945063+0000","last_undegraded":"2024-04-20T10:29:26.945063+0000","last_fullsized":"2024-04-20T10:29:26.945063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:28.944777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946847+0000","last_change":"2024-04-20T10:29:25.943381+0000","last_active":"2024-04-20T10:29:26.946847+0000","last_peered":"2024-04-20T10:29:26.946847+0000","last_clean":"2024-04-20T10:29:26.946847+0000","last_became_active":"2024-04-20T10:29:25.943065+0000","last_became_peered":"2024-04-20T10:29:25.943065+0000","last_unstale":"2024-04-20T10:29:26.946847+0000","last_undegraded":"2024-04-20T10:29:26.946847+0000","last_fullsized":"2024-04-20T10:29:26.946847+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:39.697615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946286+0000","last_change":"2024-04-20T10:29:25.927762+0000","last_active":"2024-04-20T10:29:26.946286+0000","last_peered":"2024-04-20T10:29:26.946286+0000","last_clean":"2024-04-20T10:29:26.946286+0000","last_became_active":"2024-04-20T10:29:25.927221+0000","last_became_peered":"2024-04-20T10:29:25.927221+0000","last_unstale":"2024-04-20T10:29:26.946286+0000","last_undegraded":"2024-04-20T10:29:26.946286+0000","last_fullsized":"2024-04-20T10:29:26.946286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:14:22.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917869+0000","last_change":"2024-04-20T10:29:25.942229+0000","last_active":"2024-04-20T10:29:26.917869+0000","last_peered":"2024-04-20T10:29:26.917869+0000","last_clean":"2024-04-20T10:29:26.917869+0000","last_became_active":"2024-04-20T10:29:25.941934+0000","last_became_peered":"2024-04-20T10:29:25.941934+0000","last_unstale":"2024-04-20T10:29:26.917869+0000","last_undegraded":"2024-04-20T10:29:26.917869+0000","last_fullsized":"2024-04-20T10:29:26.917869+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:48.909601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944030+0000","last_change":"2024-04-20T10:29:25.923700+0000","last_active":"2024-04-20T10:29:26.944030+0000","last_peered":"2024-04-20T10:29:26.944030+0000","last_clean":"2024-04-20T10:29:26.944030+0000","last_became_active":"2024-04-20T10:29:25.922823+0000","last_became_peered":"2024-04-20T10:29:25.922823+0000","last_unstale":"2024-04-20T10:29:26.944030+0000","last_undegraded":"2024-04-20T10:29:26.944030+0000","last_fullsized":"2024-04-20T10:29:26.944030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:18.438200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915101+0000","last_change":"2024-04-20T10:29:25.915804+0000","last_active":"2024-04-20T10:29:26.915101+0000","last_peered":"2024-04-20T10:29:26.915101+0000","last_clean":"2024-04-20T10:29:26.915101+0000","last_became_active":"2024-04-20T10:29:25.915493+0000","last_became_peered":"2024-04-20T10:29:25.915493+0000","last_unstale":"2024-04-20T10:29:26.915101+0000","last_undegraded":"2024-04-20T10:29:26.915101+0000","last_fullsized":"2024-04-20T10:29:26.915101+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:28.165829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944198+0000","last_change":"2024-04-20T10:29:25.930111+0000","last_active":"2024-04-20T10:29:26.944198+0000","last_peered":"2024-04-20T10:29:26.944198+0000","last_clean":"2024-04-20T10:29:26.944198+0000","last_became_active":"2024-04-20T10:29:25.929086+0000","last_became_peered":"2024-04-20T10:29:25.929086+0000","last_unstale":"2024-04-20T10:29:26.944198+0000","last_undegraded":"2024-04-20T10:29:26.944198+0000","last_fullsized":"2024-04-20T10:29:26.944198+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:05.706072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918351+0000","last_change":"2024-04-20T10:29:25.938030+0000","last_active":"2024-04-20T10:29:26.918351+0000","last_peered":"2024-04-20T10:29:26.918351+0000","last_clean":"2024-04-20T10:29:26.918351+0000","last_became_active":"2024-04-20T10:29:25.937705+0000","last_became_peered":"2024-04-20T10:29:25.937705+0000","last_unstale":"2024-04-20T10:29:26.918351+0000","last_undegraded":"2024-04-20T10:29:26.918351+0000","last_fullsized":"2024-04-20T10:29:26.918351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:51.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915834+0000","last_change":"2024-04-20T10:29:25.921186+0000","last_active":"2024-04-20T10:29:26.915834+0000","last_peered":"2024-04-20T10:29:26.915834+0000","last_clean":"2024-04-20T10:29:26.915834+0000","last_became_active":"2024-04-20T10:29:25.920862+0000","last_became_peered":"2024-04-20T10:29:25.920862+0000","last_unstale":"2024-04-20T10:29:26.915834+0000","last_undegraded":"2024-04-20T10:29:26.915834+0000","last_fullsized":"2024-04-20T10:29:26.915834+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:45.007288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942466+0000","last_change":"2024-04-20T10:29:25.928586+0000","last_active":"2024-04-20T10:29:26.942466+0000","last_peered":"2024-04-20T10:29:26.942466+0000","last_clean":"2024-04-20T10:29:26.942466+0000","last_became_active":"2024-04-20T10:29:25.925435+0000","last_became_peered":"2024-04-20T10:29:25.925435+0000","last_unstale":"2024-04-20T10:29:26.942466+0000","last_undegraded":"2024-04-20T10:29:26.942466+0000","last_fullsized":"2024-04-20T10:29:26.942466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:45:38.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917226+0000","last_change":"2024-04-20T10:29:25.931669+0000","last_active":"2024-04-20T10:29:26.917226+0000","last_peered":"2024-04-20T10:29:26.917226+0000","last_clean":"2024-04-20T10:29:26.917226+0000","last_became_active":"2024-04-20T10:29:25.931356+0000","last_became_peered":"2024-04-20T10:29:25.931356+0000","last_unstale":"2024-04-20T10:29:26.917226+0000","last_undegraded":"2024-04-20T10:29:26.917226+0000","last_fullsized":"2024-04-20T10:29:26.917226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:15.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914837+0000","last_change":"2024-04-20T10:29:25.931125+0000","last_active":"2024-04-20T10:29:26.914837+0000","last_peered":"2024-04-20T10:29:26.914837+0000","last_clean":"2024-04-20T10:29:26.914837+0000","last_became_active":"2024-04-20T10:29:25.930841+0000","last_became_peered":"2024-04-20T10:29:25.930841+0000","last_unstale":"2024-04-20T10:29:26.914837+0000","last_undegraded":"2024-04-20T10:29:26.914837+0000","last_fullsized":"2024-04-20T10:29:26.914837+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:23.794826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940863+0000","last_change":"2024-04-20T10:29:25.929516+0000","last_active":"2024-04-20T10:29:26.940863+0000","last_peered":"2024-04-20T10:29:26.940863+0000","last_clean":"2024-04-20T10:29:26.940863+0000","last_became_active":"2024-04-20T10:29:25.929218+0000","last_became_peered":"2024-04-20T10:29:25.929218+0000","last_unstale":"2024-04-20T10:29:26.940863+0000","last_undegraded":"2024-04-20T10:29:26.940863+0000","last_fullsized":"2024-04-20T10:29:26.940863+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:25.861823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.911735+0000","last_change":"2024-04-20T10:29:25.912156+0000","last_active":"2024-04-20T10:29:26.911735+0000","last_peered":"2024-04-20T10:29:26.911735+0000","last_clean":"2024-04-20T10:29:26.911735+0000","last_became_active":"2024-04-20T10:29:25.911810+0000","last_became_peered":"2024-04-20T10:29:25.911810+0000","last_unstale":"2024-04-20T10:29:26.911735+0000","last_undegraded":"2024-04-20T10:29:26.911735+0000","last_fullsized":"2024-04-20T10:29:26.911735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:03.872640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914609+0000","last_change":"2024-04-20T10:29:25.927858+0000","last_active":"2024-04-20T10:29:26.914609+0000","last_peered":"2024-04-20T10:29:26.914609+0000","last_clean":"2024-04-20T10:29:26.914609+0000","last_became_active":"2024-04-20T10:29:25.927563+0000","last_became_peered":"2024-04-20T10:29:25.927563+0000","last_unstale":"2024-04-20T10:29:26.914609+0000","last_undegraded":"2024-04-20T10:29:26.914609+0000","last_fullsized":"2024-04-20T10:29:26.914609+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:12.564631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913527+0000","last_change":"2024-04-20T10:29:25.929945+0000","last_active":"2024-04-20T10:29:26.913527+0000","last_peered":"2024-04-20T10:29:26.913527+0000","last_clean":"2024-04-20T10:29:26.913527+0000","last_became_active":"2024-04-20T10:29:25.929631+0000","last_became_peered":"2024-04-20T10:29:25.929631+0000","last_unstale":"2024-04-20T10:29:26.913527+0000","last_undegraded":"2024-04-20T10:29:26.913527+0000","last_fullsized":"2024-04-20T10:29:26.913527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:26:28.673759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913374+0000","last_change":"2024-04-20T10:29:25.944916+0000","last_active":"2024-04-20T10:29:26.913374+0000","last_peered":"2024-04-20T10:29:26.913374+0000","last_clean":"2024-04-20T10:29:26.913374+0000","last_became_active":"2024-04-20T10:29:25.944362+0000","last_became_peered":"2024-04-20T10:29:25.944362+0000","last_unstale":"2024-04-20T10:29:26.913374+0000","last_undegraded":"2024-04-20T10:29:26.913374+0000","last_fullsized":"2024-04-20T10:29:26.913374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:36:07.459320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942644+0000","last_change":"2024-04-20T10:29:25.936202+0000","last_active":"2024-04-20T10:29:26.942644+0000","last_peered":"2024-04-20T10:29:26.942644+0000","last_clean":"2024-04-20T10:29:26.942644+0000","last_became_active":"2024-04-20T10:29:25.935861+0000","last_became_peered":"2024-04-20T10:29:25.935861+0000","last_unstale":"2024-04-20T10:29:26.942644+0000","last_undegraded":"2024-04-20T10:29:26.942644+0000","last_fullsized":"2024-04-20T10:29:26.942644+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:31:01.813606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942442+0000","last_change":"2024-04-20T10:29:25.940311+0000","last_active":"2024-04-20T10:29:26.942442+0000","last_peered":"2024-04-20T10:29:26.942442+0000","last_clean":"2024-04-20T10:29:26.942442+0000","last_became_active":"2024-04-20T10:29:25.939941+0000","last_became_peered":"2024-04-20T10:29:25.939941+0000","last_unstale":"2024-04-20T10:29:26.942442+0000","last_undegraded":"2024-04-20T10:29:26.942442+0000","last_fullsized":"2024-04-20T10:29:26.942442+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:05.174105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942072+0000","last_change":"2024-04-20T10:29:25.937804+0000","last_active":"2024-04-20T10:29:26.942072+0000","last_peered":"2024-04-20T10:29:26.942072+0000","last_clean":"2024-04-20T10:29:26.942072+0000","last_became_active":"2024-04-20T10:29:25.937537+0000","last_became_peered":"2024-04-20T10:29:25.937537+0000","last_unstale":"2024-04-20T10:29:26.942072+0000","last_undegraded":"2024-04-20T10:29:26.942072+0000","last_fullsized":"2024-04-20T10:29:26.942072+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:10.326318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941472+0000","last_change":"2024-04-20T10:29:25.938440+0000","last_active":"2024-04-20T10:29:26.941472+0000","last_peered":"2024-04-20T10:29:26.941472+0000","last_clean":"2024-04-20T10:29:26.941472+0000","last_became_active":"2024-04-20T10:29:25.936330+0000","last_became_peered":"2024-04-20T10:29:25.936330+0000","last_unstale":"2024-04-20T10:29:26.941472+0000","last_undegraded":"2024-04-20T10:29:26.941472+0000","last_fullsized":"2024-04-20T10:29:26.941472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:28.944777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912104+0000","last_change":"2024-04-20T10:29:25.921848+0000","last_active":"2024-04-20T10:29:26.912104+0000","last_peered":"2024-04-20T10:29:26.912104+0000","last_clean":"2024-04-20T10:29:26.912104+0000","last_became_active":"2024-04-20T10:29:25.921530+0000","last_became_peered":"2024-04-20T10:29:25.921530+0000","last_unstale":"2024-04-20T10:29:26.912104+0000","last_undegraded":"2024-04-20T10:29:26.912104+0000","last_fullsized":"2024-04-20T10:29:26.912104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:14.691988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944601+0000","last_change":"2024-04-20T10:29:25.937828+0000","last_active":"2024-04-20T10:29:26.944601+0000","last_peered":"2024-04-20T10:29:26.944601+0000","last_clean":"2024-04-20T10:29:26.944601+0000","last_became_active":"2024-04-20T10:29:25.937289+0000","last_became_peered":"2024-04-20T10:29:25.937289+0000","last_unstale":"2024-04-20T10:29:26.944601+0000","last_undegraded":"2024-04-20T10:29:26.944601+0000","last_fullsized":"2024-04-20T10:29:26.944601+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:44.872739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941542+0000","last_change":"2024-04-20T10:29:25.931338+0000","last_active":"2024-04-20T10:29:26.941542+0000","last_peered":"2024-04-20T10:29:26.941542+0000","last_clean":"2024-04-20T10:29:26.941542+0000","last_became_active":"2024-04-20T10:29:25.931042+0000","last_became_peered":"2024-04-20T10:29:25.931042+0000","last_unstale":"2024-04-20T10:29:26.941542+0000","last_undegraded":"2024-04-20T10:29:26.941542+0000","last_fullsized":"2024-04-20T10:29:26.941542+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:11.620620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944692+0000","last_change":"2024-04-20T10:29:25.932131+0000","last_active":"2024-04-20T10:29:26.944692+0000","last_peered":"2024-04-20T10:29:26.944692+0000","last_clean":"2024-04-20T10:29:26.944692+0000","last_became_active":"2024-04-20T10:29:25.931787+0000","last_became_peered":"2024-04-20T10:29:25.931787+0000","last_unstale":"2024-04-20T10:29:26.944692+0000","last_undegraded":"2024-04-20T10:29:26.944692+0000","last_fullsized":"2024-04-20T10:29:26.944692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:34:32.114848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942556+0000","last_change":"2024-04-20T10:29:25.935814+0000","last_active":"2024-04-20T10:29:26.942556+0000","last_peered":"2024-04-20T10:29:26.942556+0000","last_clean":"2024-04-20T10:29:26.942556+0000","last_became_active":"2024-04-20T10:29:25.935537+0000","last_became_peered":"2024-04-20T10:29:25.935537+0000","last_unstale":"2024-04-20T10:29:26.942556+0000","last_undegraded":"2024-04-20T10:29:26.942556+0000","last_fullsized":"2024-04-20T10:29:26.942556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:58.666030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914952+0000","last_change":"2024-04-20T10:29:25.933134+0000","last_active":"2024-04-20T10:29:26.914952+0000","last_peered":"2024-04-20T10:29:26.914952+0000","last_clean":"2024-04-20T10:29:26.914952+0000","last_became_active":"2024-04-20T10:29:25.932739+0000","last_became_peered":"2024-04-20T10:29:25.932739+0000","last_unstale":"2024-04-20T10:29:26.914952+0000","last_undegraded":"2024-04-20T10:29:26.914952+0000","last_fullsized":"2024-04-20T10:29:26.914952+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:15.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912565+0000","last_change":"2024-04-20T10:29:25.931473+0000","last_active":"2024-04-20T10:29:26.912565+0000","last_peered":"2024-04-20T10:29:26.912565+0000","last_clean":"2024-04-20T10:29:26.912565+0000","last_became_active":"2024-04-20T10:29:25.927277+0000","last_became_peered":"2024-04-20T10:29:25.927277+0000","last_unstale":"2024-04-20T10:29:26.912565+0000","last_undegraded":"2024-04-20T10:29:26.912565+0000","last_fullsized":"2024-04-20T10:29:26.912565+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:57.452559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912631+0000","last_change":"2024-04-20T10:29:25.932232+0000","last_active":"2024-04-20T10:29:26.912631+0000","last_peered":"2024-04-20T10:29:26.912631+0000","last_clean":"2024-04-20T10:29:26.912631+0000","last_became_active":"2024-04-20T10:29:25.931639+0000","last_became_peered":"2024-04-20T10:29:25.931639+0000","last_unstale":"2024-04-20T10:29:26.912631+0000","last_undegraded":"2024-04-20T10:29:26.912631+0000","last_fullsized":"2024-04-20T10:29:26.912631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:04.208340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915170+0000","last_change":"2024-04-20T10:29:25.930526+0000","last_active":"2024-04-20T10:29:26.915170+0000","last_peered":"2024-04-20T10:29:26.915170+0000","last_clean":"2024-04-20T10:29:26.915170+0000","last_became_active":"2024-04-20T10:29:25.930226+0000","last_became_peered":"2024-04-20T10:29:25.930226+0000","last_unstale":"2024-04-20T10:29:26.915170+0000","last_undegraded":"2024-04-20T10:29:26.915170+0000","last_fullsized":"2024-04-20T10:29:26.915170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:48.909601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945043+0000","last_change":"2024-04-20T10:29:25.929491+0000","last_active":"2024-04-20T10:29:26.945043+0000","last_peered":"2024-04-20T10:29:26.945043+0000","last_clean":"2024-04-20T10:29:26.945043+0000","last_became_active":"2024-04-20T10:29:25.928653+0000","last_became_peered":"2024-04-20T10:29:25.928653+0000","last_unstale":"2024-04-20T10:29:26.945043+0000","last_undegraded":"2024-04-20T10:29:26.945043+0000","last_fullsized":"2024-04-20T10:29:26.945043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:24:26.609508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913097+0000","last_change":"2024-04-20T10:29:25.924820+0000","last_active":"2024-04-20T10:29:26.913097+0000","last_peered":"2024-04-20T10:29:26.913097+0000","last_clean":"2024-04-20T10:29:26.913097+0000","last_became_active":"2024-04-20T10:29:25.923609+0000","last_became_peered":"2024-04-20T10:29:25.923609+0000","last_unstale":"2024-04-20T10:29:26.913097+0000","last_undegraded":"2024-04-20T10:29:26.913097+0000","last_fullsized":"2024-04-20T10:29:26.913097+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:24:16.621728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"41'4","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941497+0000","last_change":"2024-04-20T10:29:24.902599+0000","last_active":"2024-04-20T10:29:26.941497+0000","last_peered":"2024-04-20T10:29:26.941497+0000","last_clean":"2024-04-20T10:29:26.941497+0000","last_became_active":"2024-04-20T10:29:24.902243+0000","last_became_peered":"2024-04-20T10:29:24.902243+0000","last_unstale":"2024-04-20T10:29:26.941497+0000","last_undegraded":"2024-04-20T10:29:26.941497+0000","last_fullsized":"2024-04-20T10:29:26.941497+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:12.664563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944052+0000","last_change":"2024-04-20T10:29:25.935762+0000","last_active":"2024-04-20T10:29:26.944052+0000","last_peered":"2024-04-20T10:29:26.944052+0000","last_clean":"2024-04-20T10:29:26.944052+0000","last_became_active":"2024-04-20T10:29:25.935453+0000","last_became_peered":"2024-04-20T10:29:25.935453+0000","last_unstale":"2024-04-20T10:29:26.944052+0000","last_undegraded":"2024-04-20T10:29:26.944052+0000","last_fullsized":"2024-04-20T10:29:26.944052+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:12:39.385668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946081+0000","last_change":"2024-04-20T10:29:24.895695+0000","last_active":"2024-04-20T10:29:26.946081+0000","last_peered":"2024-04-20T10:29:26.946081+0000","last_clean":"2024-04-20T10:29:26.946081+0000","last_became_active":"2024-04-20T10:29:24.895319+0000","last_became_peered":"2024-04-20T10:29:24.895319+0000","last_unstale":"2024-04-20T10:29:26.946081+0000","last_undegraded":"2024-04-20T10:29:26.946081+0000","last_fullsized":"2024-04-20T10:29:26.946081+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:25.116742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913617+0000","last_change":"2024-04-20T10:29:25.934817+0000","last_active":"2024-04-20T10:29:26.913617+0000","last_peered":"2024-04-20T10:29:26.913617+0000","last_clean":"2024-04-20T10:29:26.913617+0000","last_became_active":"2024-04-20T10:29:25.934543+0000","last_became_peered":"2024-04-20T10:29:25.934543+0000","last_unstale":"2024-04-20T10:29:26.913617+0000","last_undegraded":"2024-04-20T10:29:26.913617+0000","last_fullsized":"2024-04-20T10:29:26.913617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:29.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946937+0000","last_change":"2024-04-20T10:29:24.908735+0000","last_active":"2024-04-20T10:29:26.946937+0000","last_peered":"2024-04-20T10:29:26.946937+0000","last_clean":"2024-04-20T10:29:26.946937+0000","last_became_active":"2024-04-20T10:29:24.908390+0000","last_became_peered":"2024-04-20T10:29:24.908390+0000","last_unstale":"2024-04-20T10:29:26.946937+0000","last_undegraded":"2024-04-20T10:29:26.946937+0000","last_fullsized":"2024-04-20T10:29:26.946937+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:12.598913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945696+0000","last_change":"2024-04-20T10:29:25.937482+0000","last_active":"2024-04-20T10:29:26.945696+0000","last_peered":"2024-04-20T10:29:26.945696+0000","last_clean":"2024-04-20T10:29:26.945696+0000","last_became_active":"2024-04-20T10:29:25.937257+0000","last_became_peered":"2024-04-20T10:29:25.937257+0000","last_unstale":"2024-04-20T10:29:26.945696+0000","last_undegraded":"2024-04-20T10:29:26.945696+0000","last_fullsized":"2024-04-20T10:29:26.945696+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:30.778546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941060+0000","last_change":"2024-04-20T10:29:24.908508+0000","last_active":"2024-04-20T10:29:26.941060+0000","last_peered":"2024-04-20T10:29:26.941060+0000","last_clean":"2024-04-20T10:29:26.941060+0000","last_became_active":"2024-04-20T10:29:24.908247+0000","last_became_peered":"2024-04-20T10:29:24.908247+0000","last_unstale":"2024-04-20T10:29:26.941060+0000","last_undegraded":"2024-04-20T10:29:26.941060+0000","last_fullsized":"2024-04-20T10:29:26.941060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945286+0000","last_change":"2024-04-20T10:29:25.930392+0000","last_active":"2024-04-20T10:29:26.945286+0000","last_peered":"2024-04-20T10:29:26.945286+0000","last_clean":"2024-04-20T10:29:26.945286+0000","last_became_active":"2024-04-20T10:29:25.929589+0000","last_became_peered":"2024-04-20T10:29:25.929589+0000","last_unstale":"2024-04-20T10:29:26.945286+0000","last_undegraded":"2024-04-20T10:29:26.945286+0000","last_fullsized":"2024-04-20T10:29:26.945286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:51.314342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941152+0000","last_change":"2024-04-20T10:29:24.902051+0000","last_active":"2024-04-20T10:29:26.941152+0000","last_peered":"2024-04-20T10:29:26.941152+0000","last_clean":"2024-04-20T10:29:26.941152+0000","last_became_active":"2024-04-20T10:29:24.899974+0000","last_became_peered":"2024-04-20T10:29:24.899974+0000","last_unstale":"2024-04-20T10:29:26.941152+0000","last_undegraded":"2024-04-20T10:29:26.941152+0000","last_fullsized":"2024-04-20T10:29:26.941152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941292+0000","last_change":"2024-04-20T10:29:25.938506+0000","last_active":"2024-04-20T10:29:26.941292+0000","last_peered":"2024-04-20T10:29:26.941292+0000","last_clean":"2024-04-20T10:29:26.941292+0000","last_became_active":"2024-04-20T10:29:25.930725+0000","last_became_peered":"2024-04-20T10:29:25.930725+0000","last_unstale":"2024-04-20T10:29:26.941292+0000","last_undegraded":"2024-04-20T10:29:26.941292+0000","last_fullsized":"2024-04-20T10:29:26.941292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:25.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914740+0000","last_change":"2024-04-20T10:29:24.906546+0000","last_active":"2024-04-20T10:29:26.914740+0000","last_peered":"2024-04-20T10:29:26.914740+0000","last_clean":"2024-04-20T10:29:26.914740+0000","last_became_active":"2024-04-20T10:29:24.906232+0000","last_became_peered":"2024-04-20T10:29:24.906232+0000","last_unstale":"2024-04-20T10:29:26.914740+0000","last_undegraded":"2024-04-20T10:29:26.914740+0000","last_fullsized":"2024-04-20T10:29:26.914740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:55:18.848874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912573+0000","last_change":"2024-04-20T10:29:25.931931+0000","last_active":"2024-04-20T10:29:26.912573+0000","last_peered":"2024-04-20T10:29:26.912573+0000","last_clean":"2024-04-20T10:29:26.912573+0000","last_became_active":"2024-04-20T10:29:25.930641+0000","last_became_peered":"2024-04-20T10:29:25.930641+0000","last_unstale":"2024-04-20T10:29:26.912573+0000","last_undegraded":"2024-04-20T10:29:26.912573+0000","last_fullsized":"2024-04-20T10:29:26.912573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:29.219376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944374+0000","last_change":"2024-04-20T10:29:24.904088+0000","last_active":"2024-04-20T10:29:26.944374+0000","last_peered":"2024-04-20T10:29:26.944374+0000","last_clean":"2024-04-20T10:29:26.944374+0000","last_became_active":"2024-04-20T10:29:24.903794+0000","last_became_peered":"2024-04-20T10:29:24.903794+0000","last_unstale":"2024-04-20T10:29:26.944374+0000","last_undegraded":"2024-04-20T10:29:26.944374+0000","last_fullsized":"2024-04-20T10:29:26.944374+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:22:48.209250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945188+0000","last_change":"2024-04-20T10:29:25.926414+0000","last_active":"2024-04-20T10:29:26.945188+0000","last_peered":"2024-04-20T10:29:26.945188+0000","last_clean":"2024-04-20T10:29:26.945188+0000","last_became_active":"2024-04-20T10:29:25.925823+0000","last_became_peered":"2024-04-20T10:29:25.925823+0000","last_unstale":"2024-04-20T10:29:26.945188+0000","last_undegraded":"2024-04-20T10:29:26.945188+0000","last_fullsized":"2024-04-20T10:29:26.945188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:10.326318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914612+0000","last_change":"2024-04-20T10:29:24.885719+0000","last_active":"2024-04-20T10:29:26.914612+0000","last_peered":"2024-04-20T10:29:26.914612+0000","last_clean":"2024-04-20T10:29:26.914612+0000","last_became_active":"2024-04-20T10:29:24.885350+0000","last_became_peered":"2024-04-20T10:29:24.885350+0000","last_unstale":"2024-04-20T10:29:26.914612+0000","last_undegraded":"2024-04-20T10:29:26.914612+0000","last_fullsized":"2024-04-20T10:29:26.914612+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:23.752255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946847+0000","last_change":"2024-04-20T10:29:25.930977+0000","last_active":"2024-04-20T10:29:26.946847+0000","last_peered":"2024-04-20T10:29:26.946847+0000","last_clean":"2024-04-20T10:29:26.946847+0000","last_became_active":"2024-04-20T10:29:25.930670+0000","last_became_peered":"2024-04-20T10:29:25.930670+0000","last_unstale":"2024-04-20T10:29:26.946847+0000","last_undegraded":"2024-04-20T10:29:26.946847+0000","last_fullsized":"2024-04-20T10:29:26.946847+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:52:04.011278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946370+0000","last_change":"2024-04-20T10:29:24.908349+0000","last_active":"2024-04-20T10:29:26.946370+0000","last_peered":"2024-04-20T10:29:26.946370+0000","last_clean":"2024-04-20T10:29:26.946370+0000","last_became_active":"2024-04-20T10:29:24.908024+0000","last_became_peered":"2024-04-20T10:29:24.908024+0000","last_unstale":"2024-04-20T10:29:26.946370+0000","last_undegraded":"2024-04-20T10:29:26.946370+0000","last_fullsized":"2024-04-20T10:29:26.946370+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:32.664127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940678+0000","last_change":"2024-04-20T10:29:25.948527+0000","last_active":"2024-04-20T10:29:26.940678+0000","last_peered":"2024-04-20T10:29:26.940678+0000","last_clean":"2024-04-20T10:29:26.940678+0000","last_became_active":"2024-04-20T10:29:25.947562+0000","last_became_peered":"2024-04-20T10:29:25.947562+0000","last_unstale":"2024-04-20T10:29:26.940678+0000","last_undegraded":"2024-04-20T10:29:26.940678+0000","last_fullsized":"2024-04-20T10:29:26.940678+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:46.065918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913390+0000","last_change":"2024-04-20T10:29:24.891333+0000","last_active":"2024-04-20T10:29:26.913390+0000","last_peered":"2024-04-20T10:29:26.913390+0000","last_clean":"2024-04-20T10:29:26.913390+0000","last_became_active":"2024-04-20T10:29:24.890971+0000","last_became_peered":"2024-04-20T10:29:24.890971+0000","last_unstale":"2024-04-20T10:29:26.913390+0000","last_undegraded":"2024-04-20T10:29:26.913390+0000","last_fullsized":"2024-04-20T10:29:26.913390+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:56.588805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945083+0000","last_change":"2024-04-20T10:29:25.924543+0000","last_active":"2024-04-20T10:29:26.945083+0000","last_peered":"2024-04-20T10:29:26.945083+0000","last_clean":"2024-04-20T10:29:26.945083+0000","last_became_active":"2024-04-20T10:29:25.924057+0000","last_became_peered":"2024-04-20T10:29:25.924057+0000","last_unstale":"2024-04-20T10:29:26.945083+0000","last_undegraded":"2024-04-20T10:29:26.945083+0000","last_fullsized":"2024-04-20T10:29:26.945083+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:16:48.171394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"41'3","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912856+0000","last_change":"2024-04-20T10:29:24.906234+0000","last_active":"2024-04-20T10:29:26.912856+0000","last_peered":"2024-04-20T10:29:26.912856+0000","last_clean":"2024-04-20T10:29:26.912856+0000","last_became_active":"2024-04-20T10:29:24.905929+0000","last_became_peered":"2024-04-20T10:29:24.905929+0000","last_unstale":"2024-04-20T10:29:26.912856+0000","last_undegraded":"2024-04-20T10:29:26.912856+0000","last_fullsized":"2024-04-20T10:29:26.912856+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:51.866665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915813+0000","last_change":"2024-04-20T10:29:25.924240+0000","last_active":"2024-04-20T10:29:26.915813+0000","last_peered":"2024-04-20T10:29:26.915813+0000","last_clean":"2024-04-20T10:29:26.915813+0000","last_became_active":"2024-04-20T10:29:25.923953+0000","last_became_peered":"2024-04-20T10:29:25.923953+0000","last_unstale":"2024-04-20T10:29:26.915813+0000","last_undegraded":"2024-04-20T10:29:26.915813+0000","last_fullsized":"2024-04-20T10:29:26.915813+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:50.706951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"41'6","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946463+0000","last_change":"2024-04-20T10:29:24.903312+0000","last_active":"2024-04-20T10:29:26.946463+0000","last_peered":"2024-04-20T10:29:26.946463+0000","last_clean":"2024-04-20T10:29:26.946463+0000","last_became_active":"2024-04-20T10:29:24.901601+0000","last_became_peered":"2024-04-20T10:29:24.901601+0000","last_unstale":"2024-04-20T10:29:26.946463+0000","last_undegraded":"2024-04-20T10:29:26.946463+0000","last_fullsized":"2024-04-20T10:29:26.946463+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:41.293656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945464+0000","last_change":"2024-04-20T10:29:25.937578+0000","last_active":"2024-04-20T10:29:26.945464+0000","last_peered":"2024-04-20T10:29:26.945464+0000","last_clean":"2024-04-20T10:29:26.945464+0000","last_became_active":"2024-04-20T10:29:25.936909+0000","last_became_peered":"2024-04-20T10:29:25.936909+0000","last_unstale":"2024-04-20T10:29:26.945464+0000","last_undegraded":"2024-04-20T10:29:26.945464+0000","last_fullsized":"2024-04-20T10:29:26.945464+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:04:15.256897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945492+0000","last_change":"2024-04-20T10:29:24.907284+0000","last_active":"2024-04-20T10:29:26.945492+0000","last_peered":"2024-04-20T10:29:26.945492+0000","last_clean":"2024-04-20T10:29:26.945492+0000","last_became_active":"2024-04-20T10:29:24.906902+0000","last_became_peered":"2024-04-20T10:29:24.906902+0000","last_unstale":"2024-04-20T10:29:26.945492+0000","last_undegraded":"2024-04-20T10:29:26.945492+0000","last_fullsized":"2024-04-20T10:29:26.945492+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:56.695798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944848+0000","last_change":"2024-04-20T10:29:25.921480+0000","last_active":"2024-04-20T10:29:26.944848+0000","last_peered":"2024-04-20T10:29:26.944848+0000","last_clean":"2024-04-20T10:29:26.944848+0000","last_became_active":"2024-04-20T10:29:25.921116+0000","last_became_peered":"2024-04-20T10:29:25.921116+0000","last_unstale":"2024-04-20T10:29:26.944848+0000","last_undegraded":"2024-04-20T10:29:26.944848+0000","last_fullsized":"2024-04-20T10:29:26.944848+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:25.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944288+0000","last_change":"2024-04-20T10:29:24.893579+0000","last_active":"2024-04-20T10:29:26.944288+0000","last_peered":"2024-04-20T10:29:26.944288+0000","last_clean":"2024-04-20T10:29:26.944288+0000","last_became_active":"2024-04-20T10:29:24.893233+0000","last_became_peered":"2024-04-20T10:29:24.893233+0000","last_unstale":"2024-04-20T10:29:26.944288+0000","last_undegraded":"2024-04-20T10:29:26.944288+0000","last_fullsized":"2024-04-20T10:29:26.944288+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942052+0000","last_change":"2024-04-20T10:29:25.932249+0000","last_active":"2024-04-20T10:29:26.942052+0000","last_peered":"2024-04-20T10:29:26.942052+0000","last_clean":"2024-04-20T10:29:26.942052+0000","last_became_active":"2024-04-20T10:29:25.931894+0000","last_became_peered":"2024-04-20T10:29:25.931894+0000","last_unstale":"2024-04-20T10:29:26.942052+0000","last_undegraded":"2024-04-20T10:29:26.942052+0000","last_fullsized":"2024-04-20T10:29:26.942052+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:36.910313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912319+0000","last_change":"2024-04-20T10:29:24.891939+0000","last_active":"2024-04-20T10:29:26.912319+0000","last_peered":"2024-04-20T10:29:26.912319+0000","last_clean":"2024-04-20T10:29:26.912319+0000","last_became_active":"2024-04-20T10:29:24.891666+0000","last_became_peered":"2024-04-20T10:29:24.891666+0000","last_unstale":"2024-04-20T10:29:26.912319+0000","last_undegraded":"2024-04-20T10:29:26.912319+0000","last_fullsized":"2024-04-20T10:29:26.912319+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:57.652260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918271+0000","last_change":"2024-04-20T10:29:25.932363+0000","last_active":"2024-04-20T10:29:26.918271+0000","last_peered":"2024-04-20T10:29:26.918271+0000","last_clean":"2024-04-20T10:29:26.918271+0000","last_became_active":"2024-04-20T10:29:25.931947+0000","last_became_peered":"2024-04-20T10:29:25.931947+0000","last_unstale":"2024-04-20T10:29:26.918271+0000","last_undegraded":"2024-04-20T10:29:26.918271+0000","last_fullsized":"2024-04-20T10:29:26.918271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:13.678333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"41'8","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940756+0000","last_change":"2024-04-20T10:29:24.890425+0000","last_active":"2024-04-20T10:29:26.940756+0000","last_peered":"2024-04-20T10:29:26.940756+0000","last_clean":"2024-04-20T10:29:26.940756+0000","last_became_active":"2024-04-20T10:29:24.890115+0000","last_became_peered":"2024-04-20T10:29:24.890115+0000","last_unstale":"2024-04-20T10:29:26.940756+0000","last_undegraded":"2024-04-20T10:29:26.940756+0000","last_fullsized":"2024-04-20T10:29:26.940756+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:50.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943993+0000","last_change":"2024-04-20T10:29:25.926540+0000","last_active":"2024-04-20T10:29:26.943993+0000","last_peered":"2024-04-20T10:29:26.943993+0000","last_clean":"2024-04-20T10:29:26.943993+0000","last_became_active":"2024-04-20T10:29:25.926226+0000","last_became_peered":"2024-04-20T10:29:25.926226+0000","last_unstale":"2024-04-20T10:29:26.943993+0000","last_undegraded":"2024-04-20T10:29:26.943993+0000","last_fullsized":"2024-04-20T10:29:26.943993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:45.751412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915736+0000","last_change":"2024-04-20T10:29:24.886285+0000","last_active":"2024-04-20T10:29:26.915736+0000","last_peered":"2024-04-20T10:29:26.915736+0000","last_clean":"2024-04-20T10:29:26.915736+0000","last_became_active":"2024-04-20T10:29:24.886016+0000","last_became_peered":"2024-04-20T10:29:24.886016+0000","last_unstale":"2024-04-20T10:29:26.915736+0000","last_undegraded":"2024-04-20T10:29:26.915736+0000","last_fullsized":"2024-04-20T10:29:26.915736+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:01.416574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915407+0000","last_change":"2024-04-20T10:29:25.929424+0000","last_active":"2024-04-20T10:29:26.915407+0000","last_peered":"2024-04-20T10:29:26.915407+0000","last_clean":"2024-04-20T10:29:26.915407+0000","last_became_active":"2024-04-20T10:29:25.929120+0000","last_became_peered":"2024-04-20T10:29:25.929120+0000","last_unstale":"2024-04-20T10:29:26.915407+0000","last_undegraded":"2024-04-20T10:29:26.915407+0000","last_fullsized":"2024-04-20T10:29:26.915407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:13.678333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913623+0000","last_change":"2024-04-20T10:29:24.907760+0000","last_active":"2024-04-20T10:29:26.913623+0000","last_peered":"2024-04-20T10:29:26.913623+0000","last_clean":"2024-04-20T10:29:26.913623+0000","last_became_active":"2024-04-20T10:29:24.907189+0000","last_became_peered":"2024-04-20T10:29:24.907189+0000","last_unstale":"2024-04-20T10:29:26.913623+0000","last_undegraded":"2024-04-20T10:29:26.913623+0000","last_fullsized":"2024-04-20T10:29:26.913623+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:50:55.174213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915724+0000","last_change":"2024-04-20T10:29:25.928928+0000","last_active":"2024-04-20T10:29:26.915724+0000","last_peered":"2024-04-20T10:29:26.915724+0000","last_clean":"2024-04-20T10:29:26.915724+0000","last_became_active":"2024-04-20T10:29:25.928646+0000","last_became_peered":"2024-04-20T10:29:25.928646+0000","last_unstale":"2024-04-20T10:29:26.915724+0000","last_undegraded":"2024-04-20T10:29:26.915724+0000","last_fullsized":"2024-04-20T10:29:26.915724+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:51.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917029+0000","last_change":"2024-04-20T10:29:24.902160+0000","last_active":"2024-04-20T10:29:26.917029+0000","last_peered":"2024-04-20T10:29:26.917029+0000","last_clean":"2024-04-20T10:29:26.917029+0000","last_became_active":"2024-04-20T10:29:24.900595+0000","last_became_peered":"2024-04-20T10:29:24.900595+0000","last_unstale":"2024-04-20T10:29:26.917029+0000","last_undegraded":"2024-04-20T10:29:26.917029+0000","last_fullsized":"2024-04-20T10:29:26.917029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:56.719517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943197+0000","last_change":"2024-04-20T10:29:25.930028+0000","last_active":"2024-04-20T10:29:26.943197+0000","last_peered":"2024-04-20T10:29:26.943197+0000","last_clean":"2024-04-20T10:29:26.943197+0000","last_became_active":"2024-04-20T10:29:25.929615+0000","last_became_peered":"2024-04-20T10:29:25.929615+0000","last_unstale":"2024-04-20T10:29:26.943197+0000","last_undegraded":"2024-04-20T10:29:26.943197+0000","last_fullsized":"2024-04-20T10:29:26.943197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:18.438200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945558+0000","last_change":"2024-04-20T10:29:24.896677+0000","last_active":"2024-04-20T10:29:26.945558+0000","last_peered":"2024-04-20T10:29:26.945558+0000","last_clean":"2024-04-20T10:29:26.945558+0000","last_became_active":"2024-04-20T10:29:24.896023+0000","last_became_peered":"2024-04-20T10:29:24.896023+0000","last_unstale":"2024-04-20T10:29:26.945558+0000","last_undegraded":"2024-04-20T10:29:26.945558+0000","last_fullsized":"2024-04-20T10:29:26.945558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:24.848053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.911955+0000","last_change":"2024-04-20T10:29:25.910686+0000","last_active":"2024-04-20T10:29:26.911955+0000","last_peered":"2024-04-20T10:29:26.911955+0000","last_clean":"2024-04-20T10:29:26.911955+0000","last_became_active":"2024-04-20T10:29:25.910346+0000","last_became_peered":"2024-04-20T10:29:25.910346+0000","last_unstale":"2024-04-20T10:29:26.911955+0000","last_undegraded":"2024-04-20T10:29:26.911955+0000","last_fullsized":"2024-04-20T10:29:26.911955+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:03.878269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942552+0000","last_change":"2024-04-20T10:29:24.903532+0000","last_active":"2024-04-20T10:29:26.942552+0000","last_peered":"2024-04-20T10:29:26.942552+0000","last_clean":"2024-04-20T10:29:26.942552+0000","last_became_active":"2024-04-20T10:29:24.902068+0000","last_became_peered":"2024-04-20T10:29:24.902068+0000","last_unstale":"2024-04-20T10:29:26.942552+0000","last_undegraded":"2024-04-20T10:29:26.942552+0000","last_fullsized":"2024-04-20T10:29:26.942552+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:10.606850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944230+0000","last_change":"2024-04-20T10:29:25.923660+0000","last_active":"2024-04-20T10:29:26.944230+0000","last_peered":"2024-04-20T10:29:26.944230+0000","last_clean":"2024-04-20T10:29:26.944230+0000","last_became_active":"2024-04-20T10:29:25.922736+0000","last_became_peered":"2024-04-20T10:29:25.922736+0000","last_unstale":"2024-04-20T10:29:26.944230+0000","last_undegraded":"2024-04-20T10:29:26.944230+0000","last_fullsized":"2024-04-20T10:29:26.944230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:18.551480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944380+0000","last_change":"2024-04-20T10:29:24.892163+0000","last_active":"2024-04-20T10:29:26.944380+0000","last_peered":"2024-04-20T10:29:26.944380+0000","last_clean":"2024-04-20T10:29:26.944380+0000","last_became_active":"2024-04-20T10:29:24.891419+0000","last_became_peered":"2024-04-20T10:29:24.891419+0000","last_unstale":"2024-04-20T10:29:26.944380+0000","last_undegraded":"2024-04-20T10:29:26.944380+0000","last_fullsized":"2024-04-20T10:29:26.944380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:35.313612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941617+0000","last_change":"2024-04-20T10:29:25.923502+0000","last_active":"2024-04-20T10:29:26.941617+0000","last_peered":"2024-04-20T10:29:26.941617+0000","last_clean":"2024-04-20T10:29:26.941617+0000","last_became_active":"2024-04-20T10:29:25.923213+0000","last_became_peered":"2024-04-20T10:29:25.923213+0000","last_unstale":"2024-04-20T10:29:26.941617+0000","last_undegraded":"2024-04-20T10:29:26.941617+0000","last_fullsized":"2024-04-20T10:29:26.941617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:12.260654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915324+0000","last_change":"2024-04-20T10:29:24.883849+0000","last_active":"2024-04-20T10:29:26.915324+0000","last_peered":"2024-04-20T10:29:26.915324+0000","last_clean":"2024-04-20T10:29:26.915324+0000","last_became_active":"2024-04-20T10:29:24.883436+0000","last_became_peered":"2024-04-20T10:29:24.883436+0000","last_unstale":"2024-04-20T10:29:26.915324+0000","last_undegraded":"2024-04-20T10:29:26.915324+0000","last_fullsized":"2024-04-20T10:29:26.915324+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:33.581245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945390+0000","last_change":"2024-04-20T10:29:25.925933+0000","last_active":"2024-04-20T10:29:26.945390+0000","last_peered":"2024-04-20T10:29:26.945390+0000","last_clean":"2024-04-20T10:29:26.945390+0000","last_became_active":"2024-04-20T10:29:25.925357+0000","last_became_peered":"2024-04-20T10:29:25.925357+0000","last_unstale":"2024-04-20T10:29:26.945390+0000","last_undegraded":"2024-04-20T10:29:26.945390+0000","last_fullsized":"2024-04-20T10:29:26.945390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:29.921464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943075+0000","last_change":"2024-04-20T10:29:24.902461+0000","last_active":"2024-04-20T10:29:26.943075+0000","last_peered":"2024-04-20T10:29:26.943075+0000","last_clean":"2024-04-20T10:29:26.943075+0000","last_became_active":"2024-04-20T10:29:24.901505+0000","last_became_peered":"2024-04-20T10:29:24.901505+0000","last_unstale":"2024-04-20T10:29:26.943075+0000","last_undegraded":"2024-04-20T10:29:26.943075+0000","last_fullsized":"2024-04-20T10:29:26.943075+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917697+0000","last_change":"2024-04-20T10:29:25.938482+0000","last_active":"2024-04-20T10:29:26.917697+0000","last_peered":"2024-04-20T10:29:26.917697+0000","last_clean":"2024-04-20T10:29:26.917697+0000","last_became_active":"2024-04-20T10:29:25.938109+0000","last_became_peered":"2024-04-20T10:29:25.938109+0000","last_unstale":"2024-04-20T10:29:26.917697+0000","last_undegraded":"2024-04-20T10:29:26.917697+0000","last_fullsized":"2024-04-20T10:29:26.917697+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:28.165829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913505+0000","last_change":"2024-04-20T10:29:24.899608+0000","last_active":"2024-04-20T10:29:26.913505+0000","last_peered":"2024-04-20T10:29:26.913505+0000","last_clean":"2024-04-20T10:29:26.913505+0000","last_became_active":"2024-04-20T10:29:24.899329+0000","last_became_peered":"2024-04-20T10:29:24.899329+0000","last_unstale":"2024-04-20T10:29:26.913505+0000","last_undegraded":"2024-04-20T10:29:26.913505+0000","last_fullsized":"2024-04-20T10:29:26.913505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:52.861600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942716+0000","last_change":"2024-04-20T10:29:25.923433+0000","last_active":"2024-04-20T10:29:26.942716+0000","last_peered":"2024-04-20T10:29:26.942716+0000","last_clean":"2024-04-20T10:29:26.942716+0000","last_became_active":"2024-04-20T10:29:25.923090+0000","last_became_peered":"2024-04-20T10:29:25.923090+0000","last_unstale":"2024-04-20T10:29:26.942716+0000","last_undegraded":"2024-04-20T10:29:26.942716+0000","last_fullsized":"2024-04-20T10:29:26.942716+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:52.880434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"42'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918090+0000","last_change":"2024-04-20T10:29:24.901189+0000","last_active":"2024-04-20T10:29:26.918090+0000","last_peered":"2024-04-20T10:29:26.918090+0000","last_clean":"2024-04-20T10:29:26.918090+0000","last_became_active":"2024-04-20T10:29:24.900748+0000","last_became_peered":"2024-04-20T10:29:24.900748+0000","last_unstale":"2024-04-20T10:29:26.918090+0000","last_undegraded":"2024-04-20T10:29:26.918090+0000","last_fullsized":"2024-04-20T10:29:26.918090+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:26:34.351089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942867+0000","last_change":"2024-04-20T10:29:25.941420+0000","last_active":"2024-04-20T10:29:26.942867+0000","last_peered":"2024-04-20T10:29:26.942867+0000","last_clean":"2024-04-20T10:29:26.942867+0000","last_became_active":"2024-04-20T10:29:25.941118+0000","last_became_peered":"2024-04-20T10:29:25.941118+0000","last_unstale":"2024-04-20T10:29:26.942867+0000","last_undegraded":"2024-04-20T10:29:26.942867+0000","last_fullsized":"2024-04-20T10:29:26.942867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:45.751412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917334+0000","last_change":"2024-04-20T10:29:24.907398+0000","last_active":"2024-04-20T10:29:26.917334+0000","last_peered":"2024-04-20T10:29:26.917334+0000","last_clean":"2024-04-20T10:29:26.917334+0000","last_became_active":"2024-04-20T10:29:24.907118+0000","last_became_peered":"2024-04-20T10:29:24.907118+0000","last_unstale":"2024-04-20T10:29:26.917334+0000","last_undegraded":"2024-04-20T10:29:26.917334+0000","last_fullsized":"2024-04-20T10:29:26.917334+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:08.982478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917476+0000","last_change":"2024-04-20T10:29:25.928091+0000","last_active":"2024-04-20T10:29:26.917476+0000","last_peered":"2024-04-20T10:29:26.917476+0000","last_clean":"2024-04-20T10:29:26.917476+0000","last_became_active":"2024-04-20T10:29:25.927758+0000","last_became_peered":"2024-04-20T10:29:25.927758+0000","last_unstale":"2024-04-20T10:29:26.917476+0000","last_undegraded":"2024-04-20T10:29:26.917476+0000","last_fullsized":"2024-04-20T10:29:26.917476+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:23.794826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945688+0000","last_change":"2024-04-20T10:29:24.906505+0000","last_active":"2024-04-20T10:29:26.945688+0000","last_peered":"2024-04-20T10:29:26.945688+0000","last_clean":"2024-04-20T10:29:26.945688+0000","last_became_active":"2024-04-20T10:29:24.906039+0000","last_became_peered":"2024-04-20T10:29:24.906039+0000","last_unstale":"2024-04-20T10:29:26.945688+0000","last_undegraded":"2024-04-20T10:29:26.945688+0000","last_fullsized":"2024-04-20T10:29:26.945688+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:41.357671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945214+0000","last_change":"2024-04-20T10:29:25.918637+0000","last_active":"2024-04-20T10:29:26.945214+0000","last_peered":"2024-04-20T10:29:26.945214+0000","last_clean":"2024-04-20T10:29:26.945214+0000","last_became_active":"2024-04-20T10:29:25.918152+0000","last_became_peered":"2024-04-20T10:29:25.918152+0000","last_unstale":"2024-04-20T10:29:26.945214+0000","last_undegraded":"2024-04-20T10:29:26.945214+0000","last_fullsized":"2024-04-20T10:29:26.945214+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:12.260654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941907+0000","last_change":"2024-04-20T10:29:24.900706+0000","last_active":"2024-04-20T10:29:26.941907+0000","last_peered":"2024-04-20T10:29:26.941907+0000","last_clean":"2024-04-20T10:29:26.941907+0000","last_became_active":"2024-04-20T10:29:24.900256+0000","last_became_peered":"2024-04-20T10:29:24.900256+0000","last_unstale":"2024-04-20T10:29:26.941907+0000","last_undegraded":"2024-04-20T10:29:26.941907+0000","last_fullsized":"2024-04-20T10:29:26.941907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:25.116742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912720+0000","last_change":"2024-04-20T10:29:25.926715+0000","last_active":"2024-04-20T10:29:26.912720+0000","last_peered":"2024-04-20T10:29:26.912720+0000","last_clean":"2024-04-20T10:29:26.912720+0000","last_became_active":"2024-04-20T10:29:25.926406+0000","last_became_peered":"2024-04-20T10:29:25.926406+0000","last_unstale":"2024-04-20T10:29:26.912720+0000","last_undegraded":"2024-04-20T10:29:26.912720+0000","last_fullsized":"2024-04-20T10:29:26.912720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:01.539035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912415+0000","last_change":"2024-04-20T10:29:24.901083+0000","last_active":"2024-04-20T10:29:26.912415+0000","last_peered":"2024-04-20T10:29:26.912415+0000","last_clean":"2024-04-20T10:29:26.912415+0000","last_became_active":"2024-04-20T10:29:24.899838+0000","last_became_peered":"2024-04-20T10:29:24.899838+0000","last_unstale":"2024-04-20T10:29:26.912415+0000","last_undegraded":"2024-04-20T10:29:26.912415+0000","last_fullsized":"2024-04-20T10:29:26.912415+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:28.810260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942078+0000","last_change":"2024-04-20T10:29:25.934874+0000","last_active":"2024-04-20T10:29:26.942078+0000","last_peered":"2024-04-20T10:29:26.942078+0000","last_clean":"2024-04-20T10:29:26.942078+0000","last_became_active":"2024-04-20T10:29:25.934573+0000","last_became_peered":"2024-04-20T10:29:25.934573+0000","last_unstale":"2024-04-20T10:29:26.942078+0000","last_undegraded":"2024-04-20T10:29:26.942078+0000","last_fullsized":"2024-04-20T10:29:26.942078+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:30.778546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945567+0000","last_change":"2024-04-20T10:29:24.903773+0000","last_active":"2024-04-20T10:29:26.945567+0000","last_peered":"2024-04-20T10:29:26.945567+0000","last_clean":"2024-04-20T10:29:26.945567+0000","last_became_active":"2024-04-20T10:29:24.903463+0000","last_became_peered":"2024-04-20T10:29:24.903463+0000","last_unstale":"2024-04-20T10:29:26.945567+0000","last_undegraded":"2024-04-20T10:29:26.945567+0000","last_fullsized":"2024-04-20T10:29:26.945567+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:02:26.875257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913400+0000","last_change":"2024-04-20T10:29:25.926885+0000","last_active":"2024-04-20T10:29:26.913400+0000","last_peered":"2024-04-20T10:29:26.913400+0000","last_clean":"2024-04-20T10:29:26.913400+0000","last_became_active":"2024-04-20T10:29:25.926601+0000","last_became_peered":"2024-04-20T10:29:25.926601+0000","last_unstale":"2024-04-20T10:29:26.913400+0000","last_undegraded":"2024-04-20T10:29:26.913400+0000","last_fullsized":"2024-04-20T10:29:26.913400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:43:49.676376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942300+0000","last_change":"2024-04-20T10:29:24.900850+0000","last_active":"2024-04-20T10:29:26.942300+0000","last_peered":"2024-04-20T10:29:26.942300+0000","last_clean":"2024-04-20T10:29:26.942300+0000","last_became_active":"2024-04-20T10:29:24.898940+0000","last_became_peered":"2024-04-20T10:29:24.898940+0000","last_unstale":"2024-04-20T10:29:26.942300+0000","last_undegraded":"2024-04-20T10:29:26.942300+0000","last_fullsized":"2024-04-20T10:29:26.942300+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:35.313612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":208,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915002+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:29:26.915002+0000","last_peered":"2024-04-20T10:29:26.915002+0000","last_clean":"2024-04-20T10:29:26.915002+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:29:26.915002+0000","last_undegraded":"2024-04-20T10:29:26.915002+0000","last_fullsized":"2024-04-20T10:29:26.915002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":205348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":205312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1077248,"data_stored":796228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":205368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":207472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1077248,"data_stored":795668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":487424,"data_stored":205370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542173,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":207014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:29:35.438 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:29:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.657+0000 7f6185c00640 1 Processor -- start 2024-04-20T10:29:35.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.657+0000 7f6185c00640 1 -- start start 2024-04-20T10:29:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.659+0000 7f6185c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f618015a1f0 0x7f6180072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.659+0000 7f6185c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f618015cc90 con 0x7f6180072190 2024-04-20T10:29:35.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.659+0000 7f6185c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f618015cf70 con 0x7f6180159e10 2024-04-20T10:29:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.659+0000 7f6185c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6180072900 con 0x7f618015a1f0 2024-04-20T10:29:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.660+0000 7f6185200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6180072190 0x7f6180073420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40180/0 (socket says 172.21.15.47:40180) 2024-04-20T10:29:35.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.660+0000 7f6185200640 1 -- 172.21.15.47:0/1808240633 learned_addr learned my addr 172.21.15.47:0/1808240633 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.660+0000 7f617f400640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f618015a1f0 0x7f6180072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:35.661 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.660+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 583974373 0 0) 0x7f618015cc90 con 0x7f6180072190 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.660+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6164003520 con 0x7f6180072190 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3810747986 0 0) 0x7f6164003520 con 0x7f6180072190 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f618015a1f0 msgr2=0x7f6180072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f618015a1f0 0x7f6180072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 >> v1:172.21.15.175:6789/0 conn(0x7f6180159e10 legacy=0x7f6180071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f618015d250 con 0x7f6180072190 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f61780019f0 con 0x7f6180072190 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 >> v1:172.21.15.47:6789/0 conn(0x7f6180072190 legacy=0x7f6180073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61640039e0 0x7f6164003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6164004800 0x7f6164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f618015a1f0 0x7f61640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f616400baa0 con 0x7f61640039e0 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f616400bc50 con 0x7f6164004800 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f616400be50 con 0x7f618015a1f0 2024-04-20T10:29:35.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.661+0000 7f617fe00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61640039e0 0x7f6164003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.662+0000 7f617f400640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6164004800 0x7f6164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.662+0000 7f617fe00640 1 -- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f618015a1f0 msgr2=0x7f61640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.662+0000 7f617fe00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f618015a1f0 0x7f61640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.662+0000 7f617fe00640 1 -- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6164004800 msgr2=0x7f6164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.662+0000 7f617fe00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6164004800 0x7f6164004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.662+0000 7f617fe00640 1 -- 172.21.15.47:0/1808240633 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6164003520 con 0x7f61640039e0 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.663+0000 7f617fe00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61640039e0 0x7f6164003e00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6170001720 tx=0x7f6164005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4567fa02c4feda03 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.663+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170001370 con 0x7f61640039e0 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.663+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61700052d0 con 0x7f61640039e0 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.663+0000 7f617ea00640 1 -- 172.21.15.47:0/1808240633 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6170003830 con 0x7f61640039e0 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.664+0000 7f6185c00640 1 -- 172.21.15.47:0/1808240633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61640039e0 msgr2=0x7f6164003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.664+0000 7f6185c00640 1 --2- 172.21.15.47:0/1808240633 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61640039e0 0x7f6164003e00 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6170001720 tx=0x7f6164005830 comp rx=0 tx=0).stop 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.664+0000 7f6185200640 1 -- 172.21.15.47:0/1808240633 reap_dead start 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.664+0000 7f6185c00640 1 -- 172.21.15.47:0/1808240633 shutdown_connections 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.664+0000 7f6185c00640 1 -- 172.21.15.47:0/1808240633 >> 172.21.15.47:0/1808240633 conn(0x7f61800d0040 msgr2=0x7f61800d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.664+0000 7f6185c00640 1 -- 172.21.15.47:0/1808240633 shutdown_connections 2024-04-20T10:29:35.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.664+0000 7f6185c00640 1 -- 172.21.15.47:0/1808240633 wait complete. 2024-04-20T10:29:35.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.666+0000 7f6185c00640 1 Processor -- start 2024-04-20T10:29:35.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.666+0000 7f6185c00640 1 -- start start 2024-04-20T10:29:35.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f6185c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61640039e0 0x7f6180147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f6185c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61800cc9f0 0x7f61800ce960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f6185c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61800ceea0 0x7f618006b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f6185c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f618015d810 con 0x7f61800cc9f0 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f6185c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f618015ca40 con 0x7f61800ceea0 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f6185c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6180073b30 con 0x7f61640039e0 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f617f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61800cc9f0 0x7f61800ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f617f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61800cc9f0 0x7f61800ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53192/0 (socket says 172.21.15.47:53192) 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.667+0000 7f617f400640 1 -- 172.21.15.47:0/1653639719 learned_addr learned my addr 172.21.15.47:0/1653639719 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f617fe00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61640039e0 0x7f6180147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f6185200640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61800ceea0 0x7f618006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f617f400640 1 -- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61640039e0 msgr2=0x7f6180147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f617f400640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61640039e0 0x7f6180147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f617f400640 1 -- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61800ceea0 msgr2=0x7f618006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f617f400640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61800ceea0 0x7f618006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f617f400640 1 -- 172.21.15.47:0/1653639719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61800cce80 con 0x7f61800cc9f0 2024-04-20T10:29:35.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f617f400640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61800cc9f0 0x7f61800ce960 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f616c010860 tx=0x7f616c010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f6177e00640 1 -- 172.21.15.47:0/1653639719 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c00f040 con 0x7f61800cc9f0 2024-04-20T10:29:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f6177e00640 1 -- 172.21.15.47:0/1653639719 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f616c00eb60 con 0x7f61800cc9f0 2024-04-20T10:29:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61800cd160 con 0x7f61800cc9f0 2024-04-20T10:29:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.668+0000 7f6177e00640 1 -- 172.21.15.47:0/1653639719 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616c01c400 con 0x7f61800cc9f0 2024-04-20T10:29:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.669+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61800cd720 con 0x7f61800cc9f0 2024-04-20T10:29:35.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.670+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6180075b60 con 0x7f61800cc9f0 2024-04-20T10:29:35.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.671+0000 7f6177e00640 1 -- 172.21.15.47:0/1653639719 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f616c01c5a0 con 0x7f61800cc9f0 2024-04-20T10:29:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.672+0000 7f6177e00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f616006c8f0 0x7f616006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.672+0000 7f6177e00640 1 -- 172.21.15.47:0/1653639719 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f616c08c300 con 0x7f61800cc9f0 2024-04-20T10:29:35.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.673+0000 7f617fe00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f616006c8f0 0x7f616006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:35.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.674+0000 7f617fe00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f616006c8f0 0x7f616006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6170033000 tx=0x7f6170009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:35.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.676+0000 7f6177e00640 1 -- 172.21.15.47:0/1653639719 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f616c018030 con 0x7f61800cc9f0 2024-04-20T10:29:35.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.880+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f61800b0ff0 con 0x7f616006c8f0 2024-04-20T10:29:35.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.899+0000 7f6177e00640 1 -- 172.21.15.47:0/1653639719 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251954 (secure 0 0 0) 0x7f61800b0ff0 con 0x7f616006c8f0 2024-04-20T10:29:35.903 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:35.904 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f616006c8f0 msgr2=0x7f616006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f616006c8f0 0x7f616006edb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6170033000 tx=0x7f6170009030 comp rx=0 tx=0).stop 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61800cc9f0 msgr2=0x7f61800ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61800cc9f0 0x7f61800ce960 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f616c010860 tx=0x7f616c010d20 comp rx=0 tx=0).stop 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 shutdown_connections 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61800ceea0 0x7f618006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f616006c8f0 0x7f616006edb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f61800cc9f0 0x7f61800ce960 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 --2- 172.21.15.47:0/1653639719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61640039e0 0x7f6180147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 >> 172.21.15.47:0/1653639719 conn(0x7f61800d0040 msgr2=0x7f6180071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 shutdown_connections 2024-04-20T10:29:35.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:35.906+0000 7f6185c00640 1 -- 172.21.15.47:0/1653639719 wait complete. 2024-04-20T10:29:35.925 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-04-20T10:29:35.110459+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237800,"kb_used_data":4968,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736920,"statfs":{"total":773094113280,"available":772850606080,"internally_reserved":0,"allocated":5087232,"data_stored":2827780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":55,"apply_latency_ms":55,"commit_latency_ns":55000000,"apply_latency_ns":55000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.220076"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946274+0000","last_change":"2024-04-20T10:29:25.932997+0000","last_active":"2024-04-20T10:29:26.946274+0000","last_peered":"2024-04-20T10:29:26.946274+0000","last_clean":"2024-04-20T10:29:26.946274+0000","last_became_active":"2024-04-20T10:29:25.932397+0000","last_became_peered":"2024-04-20T10:29:25.932397+0000","last_unstale":"2024-04-20T10:29:26.946274+0000","last_undegraded":"2024-04-20T10:29:26.946274+0000","last_fullsized":"2024-04-20T10:29:26.946274+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:36:06.846350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945063+0000","last_change":"2024-04-20T10:29:25.915562+0000","last_active":"2024-04-20T10:29:26.945063+0000","last_peered":"2024-04-20T10:29:26.945063+0000","last_clean":"2024-04-20T10:29:26.945063+0000","last_became_active":"2024-04-20T10:29:25.914716+0000","last_became_peered":"2024-04-20T10:29:25.914716+0000","last_unstale":"2024-04-20T10:29:26.945063+0000","last_undegraded":"2024-04-20T10:29:26.945063+0000","last_fullsized":"2024-04-20T10:29:26.945063+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:28.944777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946847+0000","last_change":"2024-04-20T10:29:25.943381+0000","last_active":"2024-04-20T10:29:26.946847+0000","last_peered":"2024-04-20T10:29:26.946847+0000","last_clean":"2024-04-20T10:29:26.946847+0000","last_became_active":"2024-04-20T10:29:25.943065+0000","last_became_peered":"2024-04-20T10:29:25.943065+0000","last_unstale":"2024-04-20T10:29:26.946847+0000","last_undegraded":"2024-04-20T10:29:26.946847+0000","last_fullsized":"2024-04-20T10:29:26.946847+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:39.697615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946286+0000","last_change":"2024-04-20T10:29:25.927762+0000","last_active":"2024-04-20T10:29:26.946286+0000","last_peered":"2024-04-20T10:29:26.946286+0000","last_clean":"2024-04-20T10:29:26.946286+0000","last_became_active":"2024-04-20T10:29:25.927221+0000","last_became_peered":"2024-04-20T10:29:25.927221+0000","last_unstale":"2024-04-20T10:29:26.946286+0000","last_undegraded":"2024-04-20T10:29:26.946286+0000","last_fullsized":"2024-04-20T10:29:26.946286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:14:22.951728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917869+0000","last_change":"2024-04-20T10:29:25.942229+0000","last_active":"2024-04-20T10:29:26.917869+0000","last_peered":"2024-04-20T10:29:26.917869+0000","last_clean":"2024-04-20T10:29:26.917869+0000","last_became_active":"2024-04-20T10:29:25.941934+0000","last_became_peered":"2024-04-20T10:29:25.941934+0000","last_unstale":"2024-04-20T10:29:26.917869+0000","last_undegraded":"2024-04-20T10:29:26.917869+0000","last_fullsized":"2024-04-20T10:29:26.917869+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:48.909601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944030+0000","last_change":"2024-04-20T10:29:25.923700+0000","last_active":"2024-04-20T10:29:26.944030+0000","last_peered":"2024-04-20T10:29:26.944030+0000","last_clean":"2024-04-20T10:29:26.944030+0000","last_became_active":"2024-04-20T10:29:25.922823+0000","last_became_peered":"2024-04-20T10:29:25.922823+0000","last_unstale":"2024-04-20T10:29:26.944030+0000","last_undegraded":"2024-04-20T10:29:26.944030+0000","last_fullsized":"2024-04-20T10:29:26.944030+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:18.438200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915101+0000","last_change":"2024-04-20T10:29:25.915804+0000","last_active":"2024-04-20T10:29:26.915101+0000","last_peered":"2024-04-20T10:29:26.915101+0000","last_clean":"2024-04-20T10:29:26.915101+0000","last_became_active":"2024-04-20T10:29:25.915493+0000","last_became_peered":"2024-04-20T10:29:25.915493+0000","last_unstale":"2024-04-20T10:29:26.915101+0000","last_undegraded":"2024-04-20T10:29:26.915101+0000","last_fullsized":"2024-04-20T10:29:26.915101+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:28.165829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944198+0000","last_change":"2024-04-20T10:29:25.930111+0000","last_active":"2024-04-20T10:29:26.944198+0000","last_peered":"2024-04-20T10:29:26.944198+0000","last_clean":"2024-04-20T10:29:26.944198+0000","last_became_active":"2024-04-20T10:29:25.929086+0000","last_became_peered":"2024-04-20T10:29:25.929086+0000","last_unstale":"2024-04-20T10:29:26.944198+0000","last_undegraded":"2024-04-20T10:29:26.944198+0000","last_fullsized":"2024-04-20T10:29:26.944198+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:01:05.706072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918351+0000","last_change":"2024-04-20T10:29:25.938030+0000","last_active":"2024-04-20T10:29:26.918351+0000","last_peered":"2024-04-20T10:29:26.918351+0000","last_clean":"2024-04-20T10:29:26.918351+0000","last_became_active":"2024-04-20T10:29:25.937705+0000","last_became_peered":"2024-04-20T10:29:25.937705+0000","last_unstale":"2024-04-20T10:29:26.918351+0000","last_undegraded":"2024-04-20T10:29:26.918351+0000","last_fullsized":"2024-04-20T10:29:26.918351+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:51.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915834+0000","last_change":"2024-04-20T10:29:25.921186+0000","last_active":"2024-04-20T10:29:26.915834+0000","last_peered":"2024-04-20T10:29:26.915834+0000","last_clean":"2024-04-20T10:29:26.915834+0000","last_became_active":"2024-04-20T10:29:25.920862+0000","last_became_peered":"2024-04-20T10:29:25.920862+0000","last_unstale":"2024-04-20T10:29:26.915834+0000","last_undegraded":"2024-04-20T10:29:26.915834+0000","last_fullsized":"2024-04-20T10:29:26.915834+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:45.007288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942466+0000","last_change":"2024-04-20T10:29:25.928586+0000","last_active":"2024-04-20T10:29:26.942466+0000","last_peered":"2024-04-20T10:29:26.942466+0000","last_clean":"2024-04-20T10:29:26.942466+0000","last_became_active":"2024-04-20T10:29:25.925435+0000","last_became_peered":"2024-04-20T10:29:25.925435+0000","last_unstale":"2024-04-20T10:29:26.942466+0000","last_undegraded":"2024-04-20T10:29:26.942466+0000","last_fullsized":"2024-04-20T10:29:26.942466+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:45:38.745700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917226+0000","last_change":"2024-04-20T10:29:25.931669+0000","last_active":"2024-04-20T10:29:26.917226+0000","last_peered":"2024-04-20T10:29:26.917226+0000","last_clean":"2024-04-20T10:29:26.917226+0000","last_became_active":"2024-04-20T10:29:25.931356+0000","last_became_peered":"2024-04-20T10:29:25.931356+0000","last_unstale":"2024-04-20T10:29:26.917226+0000","last_undegraded":"2024-04-20T10:29:26.917226+0000","last_fullsized":"2024-04-20T10:29:26.917226+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:15.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914837+0000","last_change":"2024-04-20T10:29:25.931125+0000","last_active":"2024-04-20T10:29:26.914837+0000","last_peered":"2024-04-20T10:29:26.914837+0000","last_clean":"2024-04-20T10:29:26.914837+0000","last_became_active":"2024-04-20T10:29:25.930841+0000","last_became_peered":"2024-04-20T10:29:25.930841+0000","last_unstale":"2024-04-20T10:29:26.914837+0000","last_undegraded":"2024-04-20T10:29:26.914837+0000","last_fullsized":"2024-04-20T10:29:26.914837+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:23.794826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940863+0000","last_change":"2024-04-20T10:29:25.929516+0000","last_active":"2024-04-20T10:29:26.940863+0000","last_peered":"2024-04-20T10:29:26.940863+0000","last_clean":"2024-04-20T10:29:26.940863+0000","last_became_active":"2024-04-20T10:29:25.929218+0000","last_became_peered":"2024-04-20T10:29:25.929218+0000","last_unstale":"2024-04-20T10:29:26.940863+0000","last_undegraded":"2024-04-20T10:29:26.940863+0000","last_fullsized":"2024-04-20T10:29:26.940863+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:25.861823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.911735+0000","last_change":"2024-04-20T10:29:25.912156+0000","last_active":"2024-04-20T10:29:26.911735+0000","last_peered":"2024-04-20T10:29:26.911735+0000","last_clean":"2024-04-20T10:29:26.911735+0000","last_became_active":"2024-04-20T10:29:25.911810+0000","last_became_peered":"2024-04-20T10:29:25.911810+0000","last_unstale":"2024-04-20T10:29:26.911735+0000","last_undegraded":"2024-04-20T10:29:26.911735+0000","last_fullsized":"2024-04-20T10:29:26.911735+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:03.872640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914609+0000","last_change":"2024-04-20T10:29:25.927858+0000","last_active":"2024-04-20T10:29:26.914609+0000","last_peered":"2024-04-20T10:29:26.914609+0000","last_clean":"2024-04-20T10:29:26.914609+0000","last_became_active":"2024-04-20T10:29:25.927563+0000","last_became_peered":"2024-04-20T10:29:25.927563+0000","last_unstale":"2024-04-20T10:29:26.914609+0000","last_undegraded":"2024-04-20T10:29:26.914609+0000","last_fullsized":"2024-04-20T10:29:26.914609+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:06:12.564631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913527+0000","last_change":"2024-04-20T10:29:25.929945+0000","last_active":"2024-04-20T10:29:26.913527+0000","last_peered":"2024-04-20T10:29:26.913527+0000","last_clean":"2024-04-20T10:29:26.913527+0000","last_became_active":"2024-04-20T10:29:25.929631+0000","last_became_peered":"2024-04-20T10:29:25.929631+0000","last_unstale":"2024-04-20T10:29:26.913527+0000","last_undegraded":"2024-04-20T10:29:26.913527+0000","last_fullsized":"2024-04-20T10:29:26.913527+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:26:28.673759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913374+0000","last_change":"2024-04-20T10:29:25.944916+0000","last_active":"2024-04-20T10:29:26.913374+0000","last_peered":"2024-04-20T10:29:26.913374+0000","last_clean":"2024-04-20T10:29:26.913374+0000","last_became_active":"2024-04-20T10:29:25.944362+0000","last_became_peered":"2024-04-20T10:29:25.944362+0000","last_unstale":"2024-04-20T10:29:26.913374+0000","last_undegraded":"2024-04-20T10:29:26.913374+0000","last_fullsized":"2024-04-20T10:29:26.913374+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:36:07.459320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942644+0000","last_change":"2024-04-20T10:29:25.936202+0000","last_active":"2024-04-20T10:29:26.942644+0000","last_peered":"2024-04-20T10:29:26.942644+0000","last_clean":"2024-04-20T10:29:26.942644+0000","last_became_active":"2024-04-20T10:29:25.935861+0000","last_became_peered":"2024-04-20T10:29:25.935861+0000","last_unstale":"2024-04-20T10:29:26.942644+0000","last_undegraded":"2024-04-20T10:29:26.942644+0000","last_fullsized":"2024-04-20T10:29:26.942644+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:31:01.813606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942442+0000","last_change":"2024-04-20T10:29:25.940311+0000","last_active":"2024-04-20T10:29:26.942442+0000","last_peered":"2024-04-20T10:29:26.942442+0000","last_clean":"2024-04-20T10:29:26.942442+0000","last_became_active":"2024-04-20T10:29:25.939941+0000","last_became_peered":"2024-04-20T10:29:25.939941+0000","last_unstale":"2024-04-20T10:29:26.942442+0000","last_undegraded":"2024-04-20T10:29:26.942442+0000","last_fullsized":"2024-04-20T10:29:26.942442+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:05.174105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942072+0000","last_change":"2024-04-20T10:29:25.937804+0000","last_active":"2024-04-20T10:29:26.942072+0000","last_peered":"2024-04-20T10:29:26.942072+0000","last_clean":"2024-04-20T10:29:26.942072+0000","last_became_active":"2024-04-20T10:29:25.937537+0000","last_became_peered":"2024-04-20T10:29:25.937537+0000","last_unstale":"2024-04-20T10:29:26.942072+0000","last_undegraded":"2024-04-20T10:29:26.942072+0000","last_fullsized":"2024-04-20T10:29:26.942072+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:10.326318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941472+0000","last_change":"2024-04-20T10:29:25.938440+0000","last_active":"2024-04-20T10:29:26.941472+0000","last_peered":"2024-04-20T10:29:26.941472+0000","last_clean":"2024-04-20T10:29:26.941472+0000","last_became_active":"2024-04-20T10:29:25.936330+0000","last_became_peered":"2024-04-20T10:29:25.936330+0000","last_unstale":"2024-04-20T10:29:26.941472+0000","last_undegraded":"2024-04-20T10:29:26.941472+0000","last_fullsized":"2024-04-20T10:29:26.941472+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:13:28.944777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912104+0000","last_change":"2024-04-20T10:29:25.921848+0000","last_active":"2024-04-20T10:29:26.912104+0000","last_peered":"2024-04-20T10:29:26.912104+0000","last_clean":"2024-04-20T10:29:26.912104+0000","last_became_active":"2024-04-20T10:29:25.921530+0000","last_became_peered":"2024-04-20T10:29:25.921530+0000","last_unstale":"2024-04-20T10:29:26.912104+0000","last_undegraded":"2024-04-20T10:29:26.912104+0000","last_fullsized":"2024-04-20T10:29:26.912104+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:11:14.691988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944601+0000","last_change":"2024-04-20T10:29:25.937828+0000","last_active":"2024-04-20T10:29:26.944601+0000","last_peered":"2024-04-20T10:29:26.944601+0000","last_clean":"2024-04-20T10:29:26.944601+0000","last_became_active":"2024-04-20T10:29:25.937289+0000","last_became_peered":"2024-04-20T10:29:25.937289+0000","last_unstale":"2024-04-20T10:29:26.944601+0000","last_undegraded":"2024-04-20T10:29:26.944601+0000","last_fullsized":"2024-04-20T10:29:26.944601+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:44.872739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941542+0000","last_change":"2024-04-20T10:29:25.931338+0000","last_active":"2024-04-20T10:29:26.941542+0000","last_peered":"2024-04-20T10:29:26.941542+0000","last_clean":"2024-04-20T10:29:26.941542+0000","last_became_active":"2024-04-20T10:29:25.931042+0000","last_became_peered":"2024-04-20T10:29:25.931042+0000","last_unstale":"2024-04-20T10:29:26.941542+0000","last_undegraded":"2024-04-20T10:29:26.941542+0000","last_fullsized":"2024-04-20T10:29:26.941542+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:11.620620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944692+0000","last_change":"2024-04-20T10:29:25.932131+0000","last_active":"2024-04-20T10:29:26.944692+0000","last_peered":"2024-04-20T10:29:26.944692+0000","last_clean":"2024-04-20T10:29:26.944692+0000","last_became_active":"2024-04-20T10:29:25.931787+0000","last_became_peered":"2024-04-20T10:29:25.931787+0000","last_unstale":"2024-04-20T10:29:26.944692+0000","last_undegraded":"2024-04-20T10:29:26.944692+0000","last_fullsized":"2024-04-20T10:29:26.944692+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:34:32.114848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942556+0000","last_change":"2024-04-20T10:29:25.935814+0000","last_active":"2024-04-20T10:29:26.942556+0000","last_peered":"2024-04-20T10:29:26.942556+0000","last_clean":"2024-04-20T10:29:26.942556+0000","last_became_active":"2024-04-20T10:29:25.935537+0000","last_became_peered":"2024-04-20T10:29:25.935537+0000","last_unstale":"2024-04-20T10:29:26.942556+0000","last_undegraded":"2024-04-20T10:29:26.942556+0000","last_fullsized":"2024-04-20T10:29:26.942556+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:58.666030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914952+0000","last_change":"2024-04-20T10:29:25.933134+0000","last_active":"2024-04-20T10:29:26.914952+0000","last_peered":"2024-04-20T10:29:26.914952+0000","last_clean":"2024-04-20T10:29:26.914952+0000","last_became_active":"2024-04-20T10:29:25.932739+0000","last_became_peered":"2024-04-20T10:29:25.932739+0000","last_unstale":"2024-04-20T10:29:26.914952+0000","last_undegraded":"2024-04-20T10:29:26.914952+0000","last_fullsized":"2024-04-20T10:29:26.914952+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:38:15.936630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912565+0000","last_change":"2024-04-20T10:29:25.931473+0000","last_active":"2024-04-20T10:29:26.912565+0000","last_peered":"2024-04-20T10:29:26.912565+0000","last_clean":"2024-04-20T10:29:26.912565+0000","last_became_active":"2024-04-20T10:29:25.927277+0000","last_became_peered":"2024-04-20T10:29:25.927277+0000","last_unstale":"2024-04-20T10:29:26.912565+0000","last_undegraded":"2024-04-20T10:29:26.912565+0000","last_fullsized":"2024-04-20T10:29:26.912565+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:55:57.452559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912631+0000","last_change":"2024-04-20T10:29:25.932232+0000","last_active":"2024-04-20T10:29:26.912631+0000","last_peered":"2024-04-20T10:29:26.912631+0000","last_clean":"2024-04-20T10:29:26.912631+0000","last_became_active":"2024-04-20T10:29:25.931639+0000","last_became_peered":"2024-04-20T10:29:25.931639+0000","last_unstale":"2024-04-20T10:29:26.912631+0000","last_undegraded":"2024-04-20T10:29:26.912631+0000","last_fullsized":"2024-04-20T10:29:26.912631+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:29:04.208340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915170+0000","last_change":"2024-04-20T10:29:25.930526+0000","last_active":"2024-04-20T10:29:26.915170+0000","last_peered":"2024-04-20T10:29:26.915170+0000","last_clean":"2024-04-20T10:29:26.915170+0000","last_became_active":"2024-04-20T10:29:25.930226+0000","last_became_peered":"2024-04-20T10:29:25.930226+0000","last_unstale":"2024-04-20T10:29:26.915170+0000","last_undegraded":"2024-04-20T10:29:26.915170+0000","last_fullsized":"2024-04-20T10:29:26.915170+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:11:48.909601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945043+0000","last_change":"2024-04-20T10:29:25.929491+0000","last_active":"2024-04-20T10:29:26.945043+0000","last_peered":"2024-04-20T10:29:26.945043+0000","last_clean":"2024-04-20T10:29:26.945043+0000","last_became_active":"2024-04-20T10:29:25.928653+0000","last_became_peered":"2024-04-20T10:29:25.928653+0000","last_unstale":"2024-04-20T10:29:26.945043+0000","last_undegraded":"2024-04-20T10:29:26.945043+0000","last_fullsized":"2024-04-20T10:29:26.945043+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:24:26.609508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913097+0000","last_change":"2024-04-20T10:29:25.924820+0000","last_active":"2024-04-20T10:29:26.913097+0000","last_peered":"2024-04-20T10:29:26.913097+0000","last_clean":"2024-04-20T10:29:26.913097+0000","last_became_active":"2024-04-20T10:29:25.923609+0000","last_became_peered":"2024-04-20T10:29:25.923609+0000","last_unstale":"2024-04-20T10:29:26.913097+0000","last_undegraded":"2024-04-20T10:29:26.913097+0000","last_fullsized":"2024-04-20T10:29:26.913097+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:24:16.621728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"41'4","reported_seq":18,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941497+0000","last_change":"2024-04-20T10:29:24.902599+0000","last_active":"2024-04-20T10:29:26.941497+0000","last_peered":"2024-04-20T10:29:26.941497+0000","last_clean":"2024-04-20T10:29:26.941497+0000","last_became_active":"2024-04-20T10:29:24.902243+0000","last_became_peered":"2024-04-20T10:29:24.902243+0000","last_unstale":"2024-04-20T10:29:26.941497+0000","last_undegraded":"2024-04-20T10:29:26.941497+0000","last_fullsized":"2024-04-20T10:29:26.941497+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:12.664563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944052+0000","last_change":"2024-04-20T10:29:25.935762+0000","last_active":"2024-04-20T10:29:26.944052+0000","last_peered":"2024-04-20T10:29:26.944052+0000","last_clean":"2024-04-20T10:29:26.944052+0000","last_became_active":"2024-04-20T10:29:25.935453+0000","last_became_peered":"2024-04-20T10:29:25.935453+0000","last_unstale":"2024-04-20T10:29:26.944052+0000","last_undegraded":"2024-04-20T10:29:26.944052+0000","last_fullsized":"2024-04-20T10:29:26.944052+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:12:39.385668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946081+0000","last_change":"2024-04-20T10:29:24.895695+0000","last_active":"2024-04-20T10:29:26.946081+0000","last_peered":"2024-04-20T10:29:26.946081+0000","last_clean":"2024-04-20T10:29:26.946081+0000","last_became_active":"2024-04-20T10:29:24.895319+0000","last_became_peered":"2024-04-20T10:29:24.895319+0000","last_unstale":"2024-04-20T10:29:26.946081+0000","last_undegraded":"2024-04-20T10:29:26.946081+0000","last_fullsized":"2024-04-20T10:29:26.946081+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:25.116742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913617+0000","last_change":"2024-04-20T10:29:25.934817+0000","last_active":"2024-04-20T10:29:26.913617+0000","last_peered":"2024-04-20T10:29:26.913617+0000","last_clean":"2024-04-20T10:29:26.913617+0000","last_became_active":"2024-04-20T10:29:25.934543+0000","last_became_peered":"2024-04-20T10:29:25.934543+0000","last_unstale":"2024-04-20T10:29:26.913617+0000","last_undegraded":"2024-04-20T10:29:26.913617+0000","last_fullsized":"2024-04-20T10:29:26.913617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:13:29.992180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946937+0000","last_change":"2024-04-20T10:29:24.908735+0000","last_active":"2024-04-20T10:29:26.946937+0000","last_peered":"2024-04-20T10:29:26.946937+0000","last_clean":"2024-04-20T10:29:26.946937+0000","last_became_active":"2024-04-20T10:29:24.908390+0000","last_became_peered":"2024-04-20T10:29:24.908390+0000","last_unstale":"2024-04-20T10:29:26.946937+0000","last_undegraded":"2024-04-20T10:29:26.946937+0000","last_fullsized":"2024-04-20T10:29:26.946937+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:12.598913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945696+0000","last_change":"2024-04-20T10:29:25.937482+0000","last_active":"2024-04-20T10:29:26.945696+0000","last_peered":"2024-04-20T10:29:26.945696+0000","last_clean":"2024-04-20T10:29:26.945696+0000","last_became_active":"2024-04-20T10:29:25.937257+0000","last_became_peered":"2024-04-20T10:29:25.937257+0000","last_unstale":"2024-04-20T10:29:26.945696+0000","last_undegraded":"2024-04-20T10:29:26.945696+0000","last_fullsized":"2024-04-20T10:29:26.945696+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:30.778546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941060+0000","last_change":"2024-04-20T10:29:24.908508+0000","last_active":"2024-04-20T10:29:26.941060+0000","last_peered":"2024-04-20T10:29:26.941060+0000","last_clean":"2024-04-20T10:29:26.941060+0000","last_became_active":"2024-04-20T10:29:24.908247+0000","last_became_peered":"2024-04-20T10:29:24.908247+0000","last_unstale":"2024-04-20T10:29:26.941060+0000","last_undegraded":"2024-04-20T10:29:26.941060+0000","last_fullsized":"2024-04-20T10:29:26.941060+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945286+0000","last_change":"2024-04-20T10:29:25.930392+0000","last_active":"2024-04-20T10:29:26.945286+0000","last_peered":"2024-04-20T10:29:26.945286+0000","last_clean":"2024-04-20T10:29:26.945286+0000","last_became_active":"2024-04-20T10:29:25.929589+0000","last_became_peered":"2024-04-20T10:29:25.929589+0000","last_unstale":"2024-04-20T10:29:26.945286+0000","last_undegraded":"2024-04-20T10:29:26.945286+0000","last_fullsized":"2024-04-20T10:29:26.945286+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:51.314342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941152+0000","last_change":"2024-04-20T10:29:24.902051+0000","last_active":"2024-04-20T10:29:26.941152+0000","last_peered":"2024-04-20T10:29:26.941152+0000","last_clean":"2024-04-20T10:29:26.941152+0000","last_became_active":"2024-04-20T10:29:24.899974+0000","last_became_peered":"2024-04-20T10:29:24.899974+0000","last_unstale":"2024-04-20T10:29:26.941152+0000","last_undegraded":"2024-04-20T10:29:26.941152+0000","last_fullsized":"2024-04-20T10:29:26.941152+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941292+0000","last_change":"2024-04-20T10:29:25.938506+0000","last_active":"2024-04-20T10:29:26.941292+0000","last_peered":"2024-04-20T10:29:26.941292+0000","last_clean":"2024-04-20T10:29:26.941292+0000","last_became_active":"2024-04-20T10:29:25.930725+0000","last_became_peered":"2024-04-20T10:29:25.930725+0000","last_unstale":"2024-04-20T10:29:26.941292+0000","last_undegraded":"2024-04-20T10:29:26.941292+0000","last_fullsized":"2024-04-20T10:29:26.941292+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:25.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914740+0000","last_change":"2024-04-20T10:29:24.906546+0000","last_active":"2024-04-20T10:29:26.914740+0000","last_peered":"2024-04-20T10:29:26.914740+0000","last_clean":"2024-04-20T10:29:26.914740+0000","last_became_active":"2024-04-20T10:29:24.906232+0000","last_became_peered":"2024-04-20T10:29:24.906232+0000","last_unstale":"2024-04-20T10:29:26.914740+0000","last_undegraded":"2024-04-20T10:29:26.914740+0000","last_fullsized":"2024-04-20T10:29:26.914740+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:55:18.848874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912573+0000","last_change":"2024-04-20T10:29:25.931931+0000","last_active":"2024-04-20T10:29:26.912573+0000","last_peered":"2024-04-20T10:29:26.912573+0000","last_clean":"2024-04-20T10:29:26.912573+0000","last_became_active":"2024-04-20T10:29:25.930641+0000","last_became_peered":"2024-04-20T10:29:25.930641+0000","last_unstale":"2024-04-20T10:29:26.912573+0000","last_undegraded":"2024-04-20T10:29:26.912573+0000","last_fullsized":"2024-04-20T10:29:26.912573+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:54:29.219376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944374+0000","last_change":"2024-04-20T10:29:24.904088+0000","last_active":"2024-04-20T10:29:26.944374+0000","last_peered":"2024-04-20T10:29:26.944374+0000","last_clean":"2024-04-20T10:29:26.944374+0000","last_became_active":"2024-04-20T10:29:24.903794+0000","last_became_peered":"2024-04-20T10:29:24.903794+0000","last_unstale":"2024-04-20T10:29:26.944374+0000","last_undegraded":"2024-04-20T10:29:26.944374+0000","last_fullsized":"2024-04-20T10:29:26.944374+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:22:48.209250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945188+0000","last_change":"2024-04-20T10:29:25.926414+0000","last_active":"2024-04-20T10:29:26.945188+0000","last_peered":"2024-04-20T10:29:26.945188+0000","last_clean":"2024-04-20T10:29:26.945188+0000","last_became_active":"2024-04-20T10:29:25.925823+0000","last_became_peered":"2024-04-20T10:29:25.925823+0000","last_unstale":"2024-04-20T10:29:26.945188+0000","last_undegraded":"2024-04-20T10:29:26.945188+0000","last_fullsized":"2024-04-20T10:29:26.945188+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:44:10.326318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.914612+0000","last_change":"2024-04-20T10:29:24.885719+0000","last_active":"2024-04-20T10:29:26.914612+0000","last_peered":"2024-04-20T10:29:26.914612+0000","last_clean":"2024-04-20T10:29:26.914612+0000","last_became_active":"2024-04-20T10:29:24.885350+0000","last_became_peered":"2024-04-20T10:29:24.885350+0000","last_unstale":"2024-04-20T10:29:26.914612+0000","last_undegraded":"2024-04-20T10:29:26.914612+0000","last_fullsized":"2024-04-20T10:29:26.914612+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:23:23.752255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946847+0000","last_change":"2024-04-20T10:29:25.930977+0000","last_active":"2024-04-20T10:29:26.946847+0000","last_peered":"2024-04-20T10:29:26.946847+0000","last_clean":"2024-04-20T10:29:26.946847+0000","last_became_active":"2024-04-20T10:29:25.930670+0000","last_became_peered":"2024-04-20T10:29:25.930670+0000","last_unstale":"2024-04-20T10:29:26.946847+0000","last_undegraded":"2024-04-20T10:29:26.946847+0000","last_fullsized":"2024-04-20T10:29:26.946847+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:52:04.011278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946370+0000","last_change":"2024-04-20T10:29:24.908349+0000","last_active":"2024-04-20T10:29:26.946370+0000","last_peered":"2024-04-20T10:29:26.946370+0000","last_clean":"2024-04-20T10:29:26.946370+0000","last_became_active":"2024-04-20T10:29:24.908024+0000","last_became_peered":"2024-04-20T10:29:24.908024+0000","last_unstale":"2024-04-20T10:29:26.946370+0000","last_undegraded":"2024-04-20T10:29:26.946370+0000","last_fullsized":"2024-04-20T10:29:26.946370+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:53:32.664127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940678+0000","last_change":"2024-04-20T10:29:25.948527+0000","last_active":"2024-04-20T10:29:26.940678+0000","last_peered":"2024-04-20T10:29:26.940678+0000","last_clean":"2024-04-20T10:29:26.940678+0000","last_became_active":"2024-04-20T10:29:25.947562+0000","last_became_peered":"2024-04-20T10:29:25.947562+0000","last_unstale":"2024-04-20T10:29:26.940678+0000","last_undegraded":"2024-04-20T10:29:26.940678+0000","last_fullsized":"2024-04-20T10:29:26.940678+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:03:46.065918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913390+0000","last_change":"2024-04-20T10:29:24.891333+0000","last_active":"2024-04-20T10:29:26.913390+0000","last_peered":"2024-04-20T10:29:26.913390+0000","last_clean":"2024-04-20T10:29:26.913390+0000","last_became_active":"2024-04-20T10:29:24.890971+0000","last_became_peered":"2024-04-20T10:29:24.890971+0000","last_unstale":"2024-04-20T10:29:26.913390+0000","last_undegraded":"2024-04-20T10:29:26.913390+0000","last_fullsized":"2024-04-20T10:29:26.913390+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:41:56.588805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945083+0000","last_change":"2024-04-20T10:29:25.924543+0000","last_active":"2024-04-20T10:29:26.945083+0000","last_peered":"2024-04-20T10:29:26.945083+0000","last_clean":"2024-04-20T10:29:26.945083+0000","last_became_active":"2024-04-20T10:29:25.924057+0000","last_became_peered":"2024-04-20T10:29:25.924057+0000","last_unstale":"2024-04-20T10:29:26.945083+0000","last_undegraded":"2024-04-20T10:29:26.945083+0000","last_fullsized":"2024-04-20T10:29:26.945083+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:16:48.171394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"41'3","reported_seq":17,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912856+0000","last_change":"2024-04-20T10:29:24.906234+0000","last_active":"2024-04-20T10:29:26.912856+0000","last_peered":"2024-04-20T10:29:26.912856+0000","last_clean":"2024-04-20T10:29:26.912856+0000","last_became_active":"2024-04-20T10:29:24.905929+0000","last_became_peered":"2024-04-20T10:29:24.905929+0000","last_unstale":"2024-04-20T10:29:26.912856+0000","last_undegraded":"2024-04-20T10:29:26.912856+0000","last_fullsized":"2024-04-20T10:29:26.912856+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:51.866665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915813+0000","last_change":"2024-04-20T10:29:25.924240+0000","last_active":"2024-04-20T10:29:26.915813+0000","last_peered":"2024-04-20T10:29:26.915813+0000","last_clean":"2024-04-20T10:29:26.915813+0000","last_became_active":"2024-04-20T10:29:25.923953+0000","last_became_peered":"2024-04-20T10:29:25.923953+0000","last_unstale":"2024-04-20T10:29:26.915813+0000","last_undegraded":"2024-04-20T10:29:26.915813+0000","last_fullsized":"2024-04-20T10:29:26.915813+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:50.706951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.14","version":"41'6","reported_seq":20,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.946463+0000","last_change":"2024-04-20T10:29:24.903312+0000","last_active":"2024-04-20T10:29:26.946463+0000","last_peered":"2024-04-20T10:29:26.946463+0000","last_clean":"2024-04-20T10:29:26.946463+0000","last_became_active":"2024-04-20T10:29:24.901601+0000","last_became_peered":"2024-04-20T10:29:24.901601+0000","last_unstale":"2024-04-20T10:29:26.946463+0000","last_undegraded":"2024-04-20T10:29:26.946463+0000","last_fullsized":"2024-04-20T10:29:26.946463+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:02:41.293656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945464+0000","last_change":"2024-04-20T10:29:25.937578+0000","last_active":"2024-04-20T10:29:26.945464+0000","last_peered":"2024-04-20T10:29:26.945464+0000","last_clean":"2024-04-20T10:29:26.945464+0000","last_became_active":"2024-04-20T10:29:25.936909+0000","last_became_peered":"2024-04-20T10:29:25.936909+0000","last_unstale":"2024-04-20T10:29:26.945464+0000","last_undegraded":"2024-04-20T10:29:26.945464+0000","last_fullsized":"2024-04-20T10:29:26.945464+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:04:15.256897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945492+0000","last_change":"2024-04-20T10:29:24.907284+0000","last_active":"2024-04-20T10:29:26.945492+0000","last_peered":"2024-04-20T10:29:26.945492+0000","last_clean":"2024-04-20T10:29:26.945492+0000","last_became_active":"2024-04-20T10:29:24.906902+0000","last_became_peered":"2024-04-20T10:29:24.906902+0000","last_unstale":"2024-04-20T10:29:26.945492+0000","last_undegraded":"2024-04-20T10:29:26.945492+0000","last_fullsized":"2024-04-20T10:29:26.945492+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:56.695798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944848+0000","last_change":"2024-04-20T10:29:25.921480+0000","last_active":"2024-04-20T10:29:26.944848+0000","last_peered":"2024-04-20T10:29:26.944848+0000","last_clean":"2024-04-20T10:29:26.944848+0000","last_became_active":"2024-04-20T10:29:25.921116+0000","last_became_peered":"2024-04-20T10:29:25.921116+0000","last_unstale":"2024-04-20T10:29:26.944848+0000","last_undegraded":"2024-04-20T10:29:26.944848+0000","last_fullsized":"2024-04-20T10:29:26.944848+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:25.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944288+0000","last_change":"2024-04-20T10:29:24.893579+0000","last_active":"2024-04-20T10:29:26.944288+0000","last_peered":"2024-04-20T10:29:26.944288+0000","last_clean":"2024-04-20T10:29:26.944288+0000","last_became_active":"2024-04-20T10:29:24.893233+0000","last_became_peered":"2024-04-20T10:29:24.893233+0000","last_unstale":"2024-04-20T10:29:26.944288+0000","last_undegraded":"2024-04-20T10:29:26.944288+0000","last_fullsized":"2024-04-20T10:29:26.944288+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:28:49.693181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942052+0000","last_change":"2024-04-20T10:29:25.932249+0000","last_active":"2024-04-20T10:29:26.942052+0000","last_peered":"2024-04-20T10:29:26.942052+0000","last_clean":"2024-04-20T10:29:26.942052+0000","last_became_active":"2024-04-20T10:29:25.931894+0000","last_became_peered":"2024-04-20T10:29:25.931894+0000","last_unstale":"2024-04-20T10:29:26.942052+0000","last_undegraded":"2024-04-20T10:29:26.942052+0000","last_fullsized":"2024-04-20T10:29:26.942052+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:31:36.910313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912319+0000","last_change":"2024-04-20T10:29:24.891939+0000","last_active":"2024-04-20T10:29:26.912319+0000","last_peered":"2024-04-20T10:29:26.912319+0000","last_clean":"2024-04-20T10:29:26.912319+0000","last_became_active":"2024-04-20T10:29:24.891666+0000","last_became_peered":"2024-04-20T10:29:24.891666+0000","last_unstale":"2024-04-20T10:29:26.912319+0000","last_undegraded":"2024-04-20T10:29:26.912319+0000","last_fullsized":"2024-04-20T10:29:26.912319+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:24:57.652260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918271+0000","last_change":"2024-04-20T10:29:25.932363+0000","last_active":"2024-04-20T10:29:26.918271+0000","last_peered":"2024-04-20T10:29:26.918271+0000","last_clean":"2024-04-20T10:29:26.918271+0000","last_became_active":"2024-04-20T10:29:25.931947+0000","last_became_peered":"2024-04-20T10:29:25.931947+0000","last_unstale":"2024-04-20T10:29:26.918271+0000","last_undegraded":"2024-04-20T10:29:26.918271+0000","last_fullsized":"2024-04-20T10:29:26.918271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:13.678333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.10","version":"41'8","reported_seq":22,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.940756+0000","last_change":"2024-04-20T10:29:24.890425+0000","last_active":"2024-04-20T10:29:26.940756+0000","last_peered":"2024-04-20T10:29:26.940756+0000","last_clean":"2024-04-20T10:29:26.940756+0000","last_became_active":"2024-04-20T10:29:24.890115+0000","last_became_peered":"2024-04-20T10:29:24.890115+0000","last_unstale":"2024-04-20T10:29:26.940756+0000","last_undegraded":"2024-04-20T10:29:26.940756+0000","last_fullsized":"2024-04-20T10:29:26.940756+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:50.765554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943993+0000","last_change":"2024-04-20T10:29:25.926540+0000","last_active":"2024-04-20T10:29:26.943993+0000","last_peered":"2024-04-20T10:29:26.943993+0000","last_clean":"2024-04-20T10:29:26.943993+0000","last_became_active":"2024-04-20T10:29:25.926226+0000","last_became_peered":"2024-04-20T10:29:25.926226+0000","last_unstale":"2024-04-20T10:29:26.943993+0000","last_undegraded":"2024-04-20T10:29:26.943993+0000","last_fullsized":"2024-04-20T10:29:26.943993+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:45.751412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.f","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915736+0000","last_change":"2024-04-20T10:29:24.886285+0000","last_active":"2024-04-20T10:29:26.915736+0000","last_peered":"2024-04-20T10:29:26.915736+0000","last_clean":"2024-04-20T10:29:26.915736+0000","last_became_active":"2024-04-20T10:29:24.886016+0000","last_became_peered":"2024-04-20T10:29:24.886016+0000","last_unstale":"2024-04-20T10:29:26.915736+0000","last_undegraded":"2024-04-20T10:29:26.915736+0000","last_fullsized":"2024-04-20T10:29:26.915736+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:11:01.416574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915407+0000","last_change":"2024-04-20T10:29:25.929424+0000","last_active":"2024-04-20T10:29:26.915407+0000","last_peered":"2024-04-20T10:29:26.915407+0000","last_clean":"2024-04-20T10:29:26.915407+0000","last_became_active":"2024-04-20T10:29:25.929120+0000","last_became_peered":"2024-04-20T10:29:25.929120+0000","last_unstale":"2024-04-20T10:29:26.915407+0000","last_undegraded":"2024-04-20T10:29:26.915407+0000","last_fullsized":"2024-04-20T10:29:26.915407+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:02:13.678333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913623+0000","last_change":"2024-04-20T10:29:24.907760+0000","last_active":"2024-04-20T10:29:26.913623+0000","last_peered":"2024-04-20T10:29:26.913623+0000","last_clean":"2024-04-20T10:29:26.913623+0000","last_became_active":"2024-04-20T10:29:24.907189+0000","last_became_peered":"2024-04-20T10:29:24.907189+0000","last_unstale":"2024-04-20T10:29:26.913623+0000","last_undegraded":"2024-04-20T10:29:26.913623+0000","last_fullsized":"2024-04-20T10:29:26.913623+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:50:55.174213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915724+0000","last_change":"2024-04-20T10:29:25.928928+0000","last_active":"2024-04-20T10:29:26.915724+0000","last_peered":"2024-04-20T10:29:26.915724+0000","last_clean":"2024-04-20T10:29:26.915724+0000","last_became_active":"2024-04-20T10:29:25.928646+0000","last_became_peered":"2024-04-20T10:29:25.928646+0000","last_unstale":"2024-04-20T10:29:26.915724+0000","last_undegraded":"2024-04-20T10:29:26.915724+0000","last_fullsized":"2024-04-20T10:29:26.915724+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:59:51.779324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"41'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917029+0000","last_change":"2024-04-20T10:29:24.902160+0000","last_active":"2024-04-20T10:29:26.917029+0000","last_peered":"2024-04-20T10:29:26.917029+0000","last_clean":"2024-04-20T10:29:26.917029+0000","last_became_active":"2024-04-20T10:29:24.900595+0000","last_became_peered":"2024-04-20T10:29:24.900595+0000","last_unstale":"2024-04-20T10:29:26.917029+0000","last_undegraded":"2024-04-20T10:29:26.917029+0000","last_fullsized":"2024-04-20T10:29:26.917029+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:15:56.719517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943197+0000","last_change":"2024-04-20T10:29:25.930028+0000","last_active":"2024-04-20T10:29:26.943197+0000","last_peered":"2024-04-20T10:29:26.943197+0000","last_clean":"2024-04-20T10:29:26.943197+0000","last_became_active":"2024-04-20T10:29:25.929615+0000","last_became_peered":"2024-04-20T10:29:25.929615+0000","last_unstale":"2024-04-20T10:29:26.943197+0000","last_undegraded":"2024-04-20T10:29:26.943197+0000","last_fullsized":"2024-04-20T10:29:26.943197+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:16:18.438200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945558+0000","last_change":"2024-04-20T10:29:24.896677+0000","last_active":"2024-04-20T10:29:26.945558+0000","last_peered":"2024-04-20T10:29:26.945558+0000","last_clean":"2024-04-20T10:29:26.945558+0000","last_became_active":"2024-04-20T10:29:24.896023+0000","last_became_peered":"2024-04-20T10:29:24.896023+0000","last_unstale":"2024-04-20T10:29:26.945558+0000","last_undegraded":"2024-04-20T10:29:26.945558+0000","last_fullsized":"2024-04-20T10:29:26.945558+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:24.848053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.911955+0000","last_change":"2024-04-20T10:29:25.910686+0000","last_active":"2024-04-20T10:29:26.911955+0000","last_peered":"2024-04-20T10:29:26.911955+0000","last_clean":"2024-04-20T10:29:26.911955+0000","last_became_active":"2024-04-20T10:29:25.910346+0000","last_became_peered":"2024-04-20T10:29:25.910346+0000","last_unstale":"2024-04-20T10:29:26.911955+0000","last_undegraded":"2024-04-20T10:29:26.911955+0000","last_fullsized":"2024-04-20T10:29:26.911955+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:02:03.878269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942552+0000","last_change":"2024-04-20T10:29:24.903532+0000","last_active":"2024-04-20T10:29:26.942552+0000","last_peered":"2024-04-20T10:29:26.942552+0000","last_clean":"2024-04-20T10:29:26.942552+0000","last_became_active":"2024-04-20T10:29:24.902068+0000","last_became_peered":"2024-04-20T10:29:24.902068+0000","last_unstale":"2024-04-20T10:29:26.942552+0000","last_undegraded":"2024-04-20T10:29:26.942552+0000","last_fullsized":"2024-04-20T10:29:26.942552+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:10.606850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944230+0000","last_change":"2024-04-20T10:29:25.923660+0000","last_active":"2024-04-20T10:29:26.944230+0000","last_peered":"2024-04-20T10:29:26.944230+0000","last_clean":"2024-04-20T10:29:26.944230+0000","last_became_active":"2024-04-20T10:29:25.922736+0000","last_became_peered":"2024-04-20T10:29:25.922736+0000","last_unstale":"2024-04-20T10:29:26.944230+0000","last_undegraded":"2024-04-20T10:29:26.944230+0000","last_fullsized":"2024-04-20T10:29:26.944230+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:08:18.551480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.944380+0000","last_change":"2024-04-20T10:29:24.892163+0000","last_active":"2024-04-20T10:29:26.944380+0000","last_peered":"2024-04-20T10:29:26.944380+0000","last_clean":"2024-04-20T10:29:26.944380+0000","last_became_active":"2024-04-20T10:29:24.891419+0000","last_became_peered":"2024-04-20T10:29:24.891419+0000","last_unstale":"2024-04-20T10:29:26.944380+0000","last_undegraded":"2024-04-20T10:29:26.944380+0000","last_fullsized":"2024-04-20T10:29:26.944380+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:35.313612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941617+0000","last_change":"2024-04-20T10:29:25.923502+0000","last_active":"2024-04-20T10:29:26.941617+0000","last_peered":"2024-04-20T10:29:26.941617+0000","last_clean":"2024-04-20T10:29:26.941617+0000","last_became_active":"2024-04-20T10:29:25.923213+0000","last_became_peered":"2024-04-20T10:29:25.923213+0000","last_unstale":"2024-04-20T10:29:26.941617+0000","last_undegraded":"2024-04-20T10:29:26.941617+0000","last_fullsized":"2024-04-20T10:29:26.941617+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:12.260654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915324+0000","last_change":"2024-04-20T10:29:24.883849+0000","last_active":"2024-04-20T10:29:26.915324+0000","last_peered":"2024-04-20T10:29:26.915324+0000","last_clean":"2024-04-20T10:29:26.915324+0000","last_became_active":"2024-04-20T10:29:24.883436+0000","last_became_peered":"2024-04-20T10:29:24.883436+0000","last_unstale":"2024-04-20T10:29:26.915324+0000","last_undegraded":"2024-04-20T10:29:26.915324+0000","last_fullsized":"2024-04-20T10:29:26.915324+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:40:33.581245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945390+0000","last_change":"2024-04-20T10:29:25.925933+0000","last_active":"2024-04-20T10:29:26.945390+0000","last_peered":"2024-04-20T10:29:26.945390+0000","last_clean":"2024-04-20T10:29:26.945390+0000","last_became_active":"2024-04-20T10:29:25.925357+0000","last_became_peered":"2024-04-20T10:29:25.925357+0000","last_unstale":"2024-04-20T10:29:26.945390+0000","last_undegraded":"2024-04-20T10:29:26.945390+0000","last_fullsized":"2024-04-20T10:29:26.945390+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:52:29.921464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.943075+0000","last_change":"2024-04-20T10:29:24.902461+0000","last_active":"2024-04-20T10:29:26.943075+0000","last_peered":"2024-04-20T10:29:26.943075+0000","last_clean":"2024-04-20T10:29:26.943075+0000","last_became_active":"2024-04-20T10:29:24.901505+0000","last_became_peered":"2024-04-20T10:29:24.901505+0000","last_unstale":"2024-04-20T10:29:26.943075+0000","last_undegraded":"2024-04-20T10:29:26.943075+0000","last_fullsized":"2024-04-20T10:29:26.943075+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:39:43.993519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917697+0000","last_change":"2024-04-20T10:29:25.938482+0000","last_active":"2024-04-20T10:29:26.917697+0000","last_peered":"2024-04-20T10:29:26.917697+0000","last_clean":"2024-04-20T10:29:26.917697+0000","last_became_active":"2024-04-20T10:29:25.938109+0000","last_became_peered":"2024-04-20T10:29:25.938109+0000","last_unstale":"2024-04-20T10:29:26.917697+0000","last_undegraded":"2024-04-20T10:29:26.917697+0000","last_fullsized":"2024-04-20T10:29:26.917697+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:44:28.165829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"41'2","reported_seq":16,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913505+0000","last_change":"2024-04-20T10:29:24.899608+0000","last_active":"2024-04-20T10:29:26.913505+0000","last_peered":"2024-04-20T10:29:26.913505+0000","last_clean":"2024-04-20T10:29:26.913505+0000","last_became_active":"2024-04-20T10:29:24.899329+0000","last_became_peered":"2024-04-20T10:29:24.899329+0000","last_unstale":"2024-04-20T10:29:26.913505+0000","last_undegraded":"2024-04-20T10:29:26.913505+0000","last_fullsized":"2024-04-20T10:29:26.913505+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:36:52.861600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942716+0000","last_change":"2024-04-20T10:29:25.923433+0000","last_active":"2024-04-20T10:29:26.942716+0000","last_peered":"2024-04-20T10:29:26.942716+0000","last_clean":"2024-04-20T10:29:26.942716+0000","last_became_active":"2024-04-20T10:29:25.923090+0000","last_became_peered":"2024-04-20T10:29:25.923090+0000","last_unstale":"2024-04-20T10:29:26.942716+0000","last_undegraded":"2024-04-20T10:29:26.942716+0000","last_fullsized":"2024-04-20T10:29:26.942716+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:35:52.880434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"42'1","reported_seq":15,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.918090+0000","last_change":"2024-04-20T10:29:24.901189+0000","last_active":"2024-04-20T10:29:26.918090+0000","last_peered":"2024-04-20T10:29:26.918090+0000","last_clean":"2024-04-20T10:29:26.918090+0000","last_became_active":"2024-04-20T10:29:24.900748+0000","last_became_peered":"2024-04-20T10:29:24.900748+0000","last_unstale":"2024-04-20T10:29:26.918090+0000","last_undegraded":"2024-04-20T10:29:26.918090+0000","last_fullsized":"2024-04-20T10:29:26.918090+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:26:34.351089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942867+0000","last_change":"2024-04-20T10:29:25.941420+0000","last_active":"2024-04-20T10:29:26.942867+0000","last_peered":"2024-04-20T10:29:26.942867+0000","last_clean":"2024-04-20T10:29:26.942867+0000","last_became_active":"2024-04-20T10:29:25.941118+0000","last_became_peered":"2024-04-20T10:29:25.941118+0000","last_unstale":"2024-04-20T10:29:26.942867+0000","last_undegraded":"2024-04-20T10:29:26.942867+0000","last_fullsized":"2024-04-20T10:29:26.942867+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:13:45.751412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917334+0000","last_change":"2024-04-20T10:29:24.907398+0000","last_active":"2024-04-20T10:29:26.917334+0000","last_peered":"2024-04-20T10:29:26.917334+0000","last_clean":"2024-04-20T10:29:26.917334+0000","last_became_active":"2024-04-20T10:29:24.907118+0000","last_became_peered":"2024-04-20T10:29:24.907118+0000","last_unstale":"2024-04-20T10:29:26.917334+0000","last_undegraded":"2024-04-20T10:29:26.917334+0000","last_fullsized":"2024-04-20T10:29:26.917334+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:17:08.982478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.917476+0000","last_change":"2024-04-20T10:29:25.928091+0000","last_active":"2024-04-20T10:29:26.917476+0000","last_peered":"2024-04-20T10:29:26.917476+0000","last_clean":"2024-04-20T10:29:26.917476+0000","last_became_active":"2024-04-20T10:29:25.927758+0000","last_became_peered":"2024-04-20T10:29:25.927758+0000","last_unstale":"2024-04-20T10:29:26.917476+0000","last_undegraded":"2024-04-20T10:29:26.917476+0000","last_fullsized":"2024-04-20T10:29:26.917476+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:46:23.794826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945688+0000","last_change":"2024-04-20T10:29:24.906505+0000","last_active":"2024-04-20T10:29:26.945688+0000","last_peered":"2024-04-20T10:29:26.945688+0000","last_clean":"2024-04-20T10:29:26.945688+0000","last_became_active":"2024-04-20T10:29:24.906039+0000","last_became_peered":"2024-04-20T10:29:24.906039+0000","last_unstale":"2024-04-20T10:29:26.945688+0000","last_undegraded":"2024-04-20T10:29:26.945688+0000","last_fullsized":"2024-04-20T10:29:26.945688+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:01:41.357671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945214+0000","last_change":"2024-04-20T10:29:25.918637+0000","last_active":"2024-04-20T10:29:26.945214+0000","last_peered":"2024-04-20T10:29:26.945214+0000","last_clean":"2024-04-20T10:29:26.945214+0000","last_became_active":"2024-04-20T10:29:25.918152+0000","last_became_peered":"2024-04-20T10:29:25.918152+0000","last_unstale":"2024-04-20T10:29:26.945214+0000","last_undegraded":"2024-04-20T10:29:26.945214+0000","last_fullsized":"2024-04-20T10:29:26.945214+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:59:12.260654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.941907+0000","last_change":"2024-04-20T10:29:24.900706+0000","last_active":"2024-04-20T10:29:26.941907+0000","last_peered":"2024-04-20T10:29:26.941907+0000","last_clean":"2024-04-20T10:29:26.941907+0000","last_became_active":"2024-04-20T10:29:24.900256+0000","last_became_peered":"2024-04-20T10:29:24.900256+0000","last_unstale":"2024-04-20T10:29:26.941907+0000","last_undegraded":"2024-04-20T10:29:26.941907+0000","last_fullsized":"2024-04-20T10:29:26.941907+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:03:25.116742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912720+0000","last_change":"2024-04-20T10:29:25.926715+0000","last_active":"2024-04-20T10:29:26.912720+0000","last_peered":"2024-04-20T10:29:26.912720+0000","last_clean":"2024-04-20T10:29:26.912720+0000","last_became_active":"2024-04-20T10:29:25.926406+0000","last_became_peered":"2024-04-20T10:29:25.926406+0000","last_unstale":"2024-04-20T10:29:26.912720+0000","last_undegraded":"2024-04-20T10:29:26.912720+0000","last_fullsized":"2024-04-20T10:29:26.912720+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:01.539035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.912415+0000","last_change":"2024-04-20T10:29:24.901083+0000","last_active":"2024-04-20T10:29:26.912415+0000","last_peered":"2024-04-20T10:29:26.912415+0000","last_clean":"2024-04-20T10:29:26.912415+0000","last_became_active":"2024-04-20T10:29:24.899838+0000","last_became_peered":"2024-04-20T10:29:24.899838+0000","last_unstale":"2024-04-20T10:29:26.912415+0000","last_undegraded":"2024-04-20T10:29:26.912415+0000","last_fullsized":"2024-04-20T10:29:26.912415+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:54:28.810260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942078+0000","last_change":"2024-04-20T10:29:25.934874+0000","last_active":"2024-04-20T10:29:26.942078+0000","last_peered":"2024-04-20T10:29:26.942078+0000","last_clean":"2024-04-20T10:29:26.942078+0000","last_became_active":"2024-04-20T10:29:25.934573+0000","last_became_peered":"2024-04-20T10:29:25.934573+0000","last_unstale":"2024-04-20T10:29:26.942078+0000","last_undegraded":"2024-04-20T10:29:26.942078+0000","last_fullsized":"2024-04-20T10:29:26.942078+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:25:30.778546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.945567+0000","last_change":"2024-04-20T10:29:24.903773+0000","last_active":"2024-04-20T10:29:26.945567+0000","last_peered":"2024-04-20T10:29:26.945567+0000","last_clean":"2024-04-20T10:29:26.945567+0000","last_became_active":"2024-04-20T10:29:24.903463+0000","last_became_peered":"2024-04-20T10:29:24.903463+0000","last_unstale":"2024-04-20T10:29:26.945567+0000","last_undegraded":"2024-04-20T10:29:26.945567+0000","last_fullsized":"2024-04-20T10:29:26.945567+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:02:26.875257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.913400+0000","last_change":"2024-04-20T10:29:25.926885+0000","last_active":"2024-04-20T10:29:26.913400+0000","last_peered":"2024-04-20T10:29:26.913400+0000","last_clean":"2024-04-20T10:29:26.913400+0000","last_became_active":"2024-04-20T10:29:25.926601+0000","last_became_peered":"2024-04-20T10:29:25.926601+0000","last_unstale":"2024-04-20T10:29:26.913400+0000","last_undegraded":"2024-04-20T10:29:26.913400+0000","last_fullsized":"2024-04-20T10:29:26.913400+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:24.858281+0000","last_clean_scrub_stamp":"2024-04-20T10:29:24.858281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:43:49.676376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.942300+0000","last_change":"2024-04-20T10:29:24.900850+0000","last_active":"2024-04-20T10:29:26.942300+0000","last_peered":"2024-04-20T10:29:26.942300+0000","last_clean":"2024-04-20T10:29:26.942300+0000","last_became_active":"2024-04-20T10:29:24.898940+0000","last_became_peered":"2024-04-20T10:29:24.898940+0000","last_unstale":"2024-04-20T10:29:26.942300+0000","last_undegraded":"2024-04-20T10:29:26.942300+0000","last_fullsized":"2024-04-20T10:29:26.942300+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":39,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:29:23.844512+0000","last_clean_scrub_stamp":"2024-04-20T10:29:23.844512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:07:35.313612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":208,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-04-20T10:29:26.915002+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:29:26.915002+0000","last_peered":"2024-04-20T10:29:26.915002+0000","last_clean":"2024-04-20T10:29:26.915002+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:29:26.915002+0000","last_undegraded":"2024-04-20T10:29:26.915002+0000","last_fullsized":"2024-04-20T10:29:26.915002+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509469,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27492,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344348,"statfs":{"total":96636764160,"available":96608612352,"internally_reserved":0,"allocated":483328,"data_stored":205348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509469,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27500,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344340,"statfs":{"total":96636764160,"available":96608604160,"internally_reserved":0,"allocated":483328,"data_stored":205312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509469,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343764,"statfs":{"total":96636764160,"available":96608014336,"internally_reserved":0,"allocated":1077248,"data_stored":796228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31660,"kb_used_data":472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340180,"statfs":{"total":96636764160,"available":96604344320,"internally_reserved":0,"allocated":483328,"data_stored":205368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509469,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31672,"kb_used_data":484,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340168,"statfs":{"total":96636764160,"available":96604332032,"internally_reserved":0,"allocated":495616,"data_stored":207472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509469,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32236,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339604,"statfs":{"total":96636764160,"available":96603754496,"internally_reserved":0,"allocated":1077248,"data_stored":795668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509469,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31656,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340184,"statfs":{"total":96636764160,"available":96604348416,"internally_reserved":0,"allocated":487424,"data_stored":205370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542174,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27508,"kb_used_data":488,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344332,"statfs":{"total":96636764160,"available":96608595968,"internally_reserved":0,"allocated":499712,"data_stored":207014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:29:35.930 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T10:29:35.931 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T10:29:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.185+0000 7f0654200640 1 Processor -- start 2024-04-20T10:29:36.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.186+0000 7f0654200640 1 -- start start 2024-04-20T10:29:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.187+0000 7f0654200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f064c15b3b0 0x7f064c06eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.187+0000 7f0654200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f064c0d0360 0x7f064c0d0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.187+0000 7f0654200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f064c0ade40 con 0x7f064c06dcb0 2024-04-20T10:29:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.187+0000 7f0654200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f064c0ae0e0 con 0x7f064c0d0360 2024-04-20T10:29:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.187+0000 7f0654200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f064c15b790 con 0x7f064c15b3b0 2024-04-20T10:29:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0652200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f064c0d0360 0x7f064c0d0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0652200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f064c0d0360 0x7f064c0d0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53200/0 (socket says 172.21.15.47:53200) 2024-04-20T10:29:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0652200640 1 -- 172.21.15.47:0/190198499 learned_addr learned my addr 172.21.15.47:0/190198499 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:36.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0651800640 1 --2- 172.21.15.47:0/190198499 >> v2:172.21.15.175:3300/0 conn(0x7f064c15b3b0 0x7f064c06eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0652200640 1 -- 172.21.15.47:0/190198499 >> v2:172.21.15.175:3300/0 conn(0x7f064c15b3b0 msgr2=0x7f064c06eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:36.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0652200640 1 --2- 172.21.15.47:0/190198499 >> v2:172.21.15.175:3300/0 conn(0x7f064c15b3b0 0x7f064c06eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0652200640 1 -- 172.21.15.47:0/190198499 >> v1:172.21.15.175:6789/0 conn(0x7f064c06dcb0 legacy=0x7f064c071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:36.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.188+0000 7f0652200640 1 -- 172.21.15.47:0/190198499 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c072260 con 0x7f064c0d0360 2024-04-20T10:29:36.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.189+0000 7f0652200640 1 --2- 172.21.15.47:0/190198499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c0d0770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f064c06cfa0 tx=0x7f064402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861dab88f35d05b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.190+0000 7f064be00640 1 -- 172.21.15.47:0/190198499 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f064402ca80 con 0x7f064c0d0360 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.190+0000 7f064be00640 1 -- 172.21.15.47:0/190198499 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f064402cc20 con 0x7f064c0d0360 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.190+0000 7f064be00640 1 -- 172.21.15.47:0/190198499 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06440343f0 con 0x7f064c0d0360 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 -- 172.21.15.47:0/190198499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 msgr2=0x7f064c0d0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 --2- 172.21.15.47:0/190198499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c0d0770 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f064c06cfa0 tx=0x7f064402bee0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 -- 172.21.15.47:0/190198499 shutdown_connections 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 --2- 172.21.15.47:0/190198499 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c0d0770 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 --2- 172.21.15.47:0/190198499 >> v2:172.21.15.175:3300/0 conn(0x7f064c15b3b0 0x7f064c06eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 -- 172.21.15.47:0/190198499 >> 172.21.15.47:0/190198499 conn(0x7f064c0b3350 msgr2=0x7f064c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:36.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 -- 172.21.15.47:0/190198499 shutdown_connections 2024-04-20T10:29:36.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.191+0000 7f0654200640 1 -- 172.21.15.47:0/190198499 wait complete. 2024-04-20T10:29:36.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.193+0000 7f0654200640 1 Processor -- start 2024-04-20T10:29:36.193 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.193+0000 7f0654200640 1 -- start start 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0654200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f064c06dcb0 0x7f064c149430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0654200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0654200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f064c15b3b0 0x7f064c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0654200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f064c0ae940 con 0x7f064c0d0360 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0654200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f064c0ae660 con 0x7f064c15b3b0 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0654200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f064c0aec20 con 0x7f064c06dcb0 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0650e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0650e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53206/0 (socket says 172.21.15.47:53206) 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0650e00640 1 -- 172.21.15.47:0/3819184375 learned_addr learned my addr 172.21.15.47:0/3819184375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.194+0000 7f0651800640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f064c06dcb0 0x7f064c149430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0652200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f064c15b3b0 0x7f064c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0650e00640 1 -- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f064c06dcb0 msgr2=0x7f064c149430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0650e00640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f064c06dcb0 0x7f064c149430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0650e00640 1 -- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f064c15b3b0 msgr2=0x7f064c1526a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0650e00640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f064c15b3b0 0x7f064c1526a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0650e00640 1 -- 172.21.15.47:0/3819184375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064c0afc90 con 0x7f064c0d0360 2024-04-20T10:29:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0652200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f064c15b3b0 0x7f064c1526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:36.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0650e00640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c152160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f064000c410 tx=0x7f064000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0649600640 1 -- 172.21.15.47:0/3819184375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640017070 con 0x7f064c0d0360 2024-04-20T10:29:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0649600640 1 -- 172.21.15.47:0/3819184375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f06400035f0 con 0x7f064c0d0360 2024-04-20T10:29:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0649600640 1 -- 172.21.15.47:0/3819184375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0640003a20 con 0x7f064c0d0360 2024-04-20T10:29:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.195+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f064c0aff70 con 0x7f064c0d0360 2024-04-20T10:29:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.196+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f064c0b0530 con 0x7f064c0d0360 2024-04-20T10:29:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.197+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f064c068980 con 0x7f064c0d0360 2024-04-20T10:29:36.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.198+0000 7f0649600640 1 -- 172.21.15.47:0/3819184375 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f064001a070 con 0x7f064c0d0360 2024-04-20T10:29:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.199+0000 7f0649600640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f063006c940 0x7f063006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.199+0000 7f0649600640 1 -- 172.21.15.47:0/3819184375 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f064008b1f0 con 0x7f064c0d0360 2024-04-20T10:29:36.199 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.200+0000 7f0651800640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f063006c940 0x7f063006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.200 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.201+0000 7f0651800640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f063006c940 0x7f063006ee00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f063c006fd0 tx=0x7f063c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:36.203 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.203+0000 7f0649600640 1 -- 172.21.15.47:0/3819184375 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0640059500 con 0x7f064c0d0360 2024-04-20T10:29:36.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.477+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f064c1594b0 con 0x7f064c0d0360 2024-04-20T10:29:36.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.481+0000 7f0649600640 1 -- 172.21.15.47:0/3819184375 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7f0640058e00 con 0x7f064c0d0360 2024-04-20T10:29:36.481 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-20T10:29:36.481 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBXmCNmrHlNFRAAY90gnx6m90huEzUUDh9rtw== 2024-04-20T10:29:36.481 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-20T10:29:36.482 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-20T10:29:36.482 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T10:29:36.482 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.483+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f063006c940 msgr2=0x7f063006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.483+0000 7f0654200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f063006c940 0x7f063006ee00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f063c006fd0 tx=0x7f063c008040 comp rx=0 tx=0).stop 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.483+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 msgr2=0x7f064c152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.483+0000 7f0654200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c152160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f064000c410 tx=0x7f064000c8d0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 shutdown_connections 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f064c15b3b0 0x7f064c1526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f063006c940 0x7f063006ee00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f064c0d0360 0x7f064c152160 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 --2- 172.21.15.47:0/3819184375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f064c06dcb0 0x7f064c149430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 >> 172.21.15.47:0/3819184375 conn(0x7f064c0b3350 msgr2=0x7f064c15ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 shutdown_connections 2024-04-20T10:29:36.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.484+0000 7f0654200640 1 -- 172.21.15.47:0/3819184375 wait complete. 2024-04-20T10:29:36.502 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:36.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.710+0000 7fc883200640 1 Processor -- start 2024-04-20T10:29:36.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.710+0000 7fc883200640 1 -- start start 2024-04-20T10:29:36.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.711+0000 7fc883200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc884161d50 0x7fc8840d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.711+0000 7fc883200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 0x7fc8840b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.711+0000 7fc883200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc88406c190 con 0x7fc88406c6d0 2024-04-20T10:29:36.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.711+0000 7fc883200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc884162130 con 0x7fc884161d50 2024-04-20T10:29:36.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.711+0000 7fc883200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc88406c470 con 0x7fc8840b3330 2024-04-20T10:29:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc882800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc88406c6d0 0x7fc8840cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40186/0 (socket says 172.21.15.47:40186) 2024-04-20T10:29:36.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc882800640 1 -- 172.21.15.47:0/2236114617 learned_addr learned my addr 172.21.15.47:0/2236114617 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881e00640 1 --2- 172.21.15.47:0/2236114617 >> v2:172.21.15.175:3300/0 conn(0x7fc884161d50 0x7fc8840d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3546866714 0 0) 0x7fc88406c190 con 0x7fc88406c6d0 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc879400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 0x7fc8840b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc868003520 con 0x7fc88406c6d0 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1359808005 0 0) 0x7fc868003520 con 0x7fc88406c6d0 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 msgr2=0x7fc8840b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 0x7fc8840b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 >> v2:172.21.15.175:3300/0 conn(0x7fc884161d50 msgr2=0x7fc8840d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 --2- 172.21.15.47:0/2236114617 >> v2:172.21.15.175:3300/0 conn(0x7fc884161d50 0x7fc8840d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8840d2e00 con 0x7fc88406c6d0 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fc87c0019f0 con 0x7fc88406c6d0 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 >> v1:172.21.15.47:6789/0 conn(0x7fc88406c6d0 legacy=0x7fc8840cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.712+0000 7fc881400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 0x7fc8680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc881400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc868003fe0 0x7fc868004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc881400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868004d40 0x7fc8680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc88406c470 con 0x7fc868003fe0 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc86800ba70 con 0x7fc868004d40 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc86800bc40 con 0x7fc8840b3330 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc879400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc868003fe0 0x7fc868004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc881e00640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 0x7fc8680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc882800640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868004d40 0x7fc8680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc879400640 1 -- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 msgr2=0x7fc8680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc879400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8840b3330 0x7fc8680039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc879400640 1 -- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868004d40 msgr2=0x7fc8680051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc879400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868004d40 0x7fc8680051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.713+0000 7fc879400640 1 -- 172.21.15.47:0/2236114617 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc868003520 con 0x7fc868003fe0 2024-04-20T10:29:36.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.714+0000 7fc879400640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc868003fe0 0x7fc868004800 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ba30 tx=0x7fc868005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34daf9af5511b4bf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:36.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c036020 con 0x7fc868003fe0 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc86c032640 con 0x7fc868003fe0 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc881400640 1 -- 172.21.15.47:0/2236114617 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00b600 con 0x7fc868003fe0 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc883200640 1 -- 172.21.15.47:0/2236114617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc868003fe0 msgr2=0x7fc868004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc883200640 1 --2- 172.21.15.47:0/2236114617 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc868003fe0 0x7fc868004800 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc86c00ba30 tx=0x7fc868005830 comp rx=0 tx=0).stop 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc882800640 1 -- 172.21.15.47:0/2236114617 reap_dead start 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc883200640 1 -- 172.21.15.47:0/2236114617 shutdown_connections 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc883200640 1 -- 172.21.15.47:0/2236114617 >> 172.21.15.47:0/2236114617 conn(0x7fc8840d4450 msgr2=0x7fc8840d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.715+0000 7fc883200640 1 -- 172.21.15.47:0/2236114617 shutdown_connections 2024-04-20T10:29:36.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.716+0000 7fc883200640 1 -- 172.21.15.47:0/2236114617 wait complete. 2024-04-20T10:29:36.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.717+0000 7fc883200640 1 Processor -- start 2024-04-20T10:29:36.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.717+0000 7fc883200640 1 -- start start 2024-04-20T10:29:36.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.718+0000 7fc883200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868003fe0 0x7fc8840c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.718+0000 7fc883200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc88414a9e0 0x7fc8840ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.718+0000 7fc883200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8840cf2e0 0x7fc8840cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.718+0000 7fc883200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc8840b3880 con 0x7fc8840cf2e0 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.718+0000 7fc883200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8840d33c0 con 0x7fc868003fe0 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.718+0000 7fc883200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8840b3a00 con 0x7fc88414a9e0 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8840cf2e0 0x7fc8840cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8840cf2e0 0x7fc8840cf760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53218/0 (socket says 172.21.15.47:53218) 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 -- 172.21.15.47:0/808511190 learned_addr learned my addr 172.21.15.47:0/808511190 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc881e00640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868003fe0 0x7fc8840c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc879400640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc88414a9e0 0x7fc8840ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 -- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc88414a9e0 msgr2=0x7fc8840ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc88414a9e0 0x7fc8840ca6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 -- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868003fe0 msgr2=0x7fc8840c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868003fe0 0x7fc8840c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 -- 172.21.15.47:0/808511190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8841f7df0 con 0x7fc8840cf2e0 2024-04-20T10:29:36.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.719+0000 7fc882800640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8840cf2e0 0x7fc8840cf760 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc87c007c40 tx=0x7fc87c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.720+0000 7fc87aa00640 1 -- 172.21.15.47:0/808511190 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c004d60 con 0x7fc8840cf2e0 2024-04-20T10:29:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.720+0000 7fc87aa00640 1 -- 172.21.15.47:0/808511190 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc87c00bb80 con 0x7fc8840cf2e0 2024-04-20T10:29:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.720+0000 7fc87aa00640 1 -- 172.21.15.47:0/808511190 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc87c007070 con 0x7fc8840cf2e0 2024-04-20T10:29:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.720+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8841f80d0 con 0x7fc8840cf2e0 2024-04-20T10:29:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.720+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8841f8690 con 0x7fc8840cf2e0 2024-04-20T10:29:36.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.721+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8841f88d0 con 0x7fc8840cf2e0 2024-04-20T10:29:36.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.722+0000 7fc87aa00640 1 -- 172.21.15.47:0/808511190 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc87c01b060 con 0x7fc8840cf2e0 2024-04-20T10:29:36.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.723+0000 7fc87aa00640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc86006c8f0 0x7fc86006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:36.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.723+0000 7fc87aa00640 1 -- 172.21.15.47:0/808511190 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fc87c08bfc0 con 0x7fc8840cf2e0 2024-04-20T10:29:36.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.724+0000 7fc881e00640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc86006c8f0 0x7fc86006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:36.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.725+0000 7fc881e00640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc86006c8f0 0x7fc86006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc874006fd0 tx=0x7fc874008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:36.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.727+0000 7fc87aa00640 1 -- 172.21.15.47:0/808511190 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc8841f88d0 con 0x7fc8840cf2e0 2024-04-20T10:29:36.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.973+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8840cb2e0 con 0x7fc8840cf2e0 2024-04-20T10:29:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.974+0000 7fc87aa00640 1 -- 172.21.15.47:0/808511190 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5205 (secure 0 0 0) 0x7fc87c05a1c0 con 0x7fc8840cf2e0 2024-04-20T10:29:36.974 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:36.974 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:28.440368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/1013167846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1013167846},{"type":"v1","addr":"172.21.15.47:6835","nonce":1013167846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5481,"qdb_cluster":[5481]},"id":2}]} 2024-04-20T10:29:36.974 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-20T10:29:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.976+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc86006c8f0 msgr2=0x7fc86006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.976+0000 7fc883200640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc86006c8f0 0x7fc86006edb0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc874006fd0 tx=0x7fc874008040 comp rx=0 tx=0).stop 2024-04-20T10:29:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.976+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8840cf2e0 msgr2=0x7fc8840cf760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:36.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.976+0000 7fc883200640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8840cf2e0 0x7fc8840cf760 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc87c007c40 tx=0x7fc87c00b4a0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 shutdown_connections 2024-04-20T10:29:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc86006c8f0 0x7fc86006edb0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8840cf2e0 0x7fc8840cf760 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc88414a9e0 0x7fc8840ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 --2- 172.21.15.47:0/808511190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc868003fe0 0x7fc8840c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 >> 172.21.15.47:0/808511190 conn(0x7fc8840d4450 msgr2=0x7fc88415f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 shutdown_connections 2024-04-20T10:29:36.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:36.977+0000 7fc883200640 1 -- 172.21.15.47:0/808511190 wait complete. 2024-04-20T10:29:37.399 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:29:25.353181+0000', 'modified': '2024-04-20T10:29:28.440368+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5481}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5481': {'gid': 5481, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6835/1013167846', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 1013167846}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 1013167846}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5481, 'qdb_cluster': [5481]} 2024-04-20T10:29:37.399 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:29:37.399 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T10:29:37.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.622+0000 7f133f400640 1 Processor -- start 2024-04-20T10:29:37.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.623+0000 7f133f400640 1 -- start start 2024-04-20T10:29:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.624+0000 7f133f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13380d0a70 0x7f13380ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.624+0000 7f133f400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133806df70 con 0x7f133806f090 2024-04-20T10:29:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.624+0000 7f133f400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f133806e310 con 0x7f1338162c00 2024-04-20T10:29:37.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.624+0000 7f133f400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1338162fe0 con 0x7f13380d0a70 2024-04-20T10:29:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.624+0000 7f132fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13380d0a70 0x7f13380ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.624+0000 7f132fe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13380d0a70 0x7f13380ad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53226/0 (socket says 172.21.15.47:53226) 2024-04-20T10:29:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.624+0000 7f132fe00640 1 -- 172.21.15.47:0/946380205 learned_addr learned my addr 172.21.15.47:0/946380205 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.625+0000 7f132fe00640 1 -- 172.21.15.47:0/946380205 >> v1:172.21.15.175:6789/0 conn(0x7f1338162c00 legacy=0x7f13380d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.625+0000 7f132fe00640 1 -- 172.21.15.47:0/946380205 >> v1:172.21.15.47:6789/0 conn(0x7f133806f090 legacy=0x7f13380ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.625+0000 7f132fe00640 1 -- 172.21.15.47:0/946380205 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133806e5b0 con 0x7f13380d0a70 2024-04-20T10:29:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.626+0000 7f132fe00640 1 --2- 172.21.15.47:0/946380205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13380ad050 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f133806e4e0 tx=0x7f132002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3c6e002fc21b396 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.626+0000 7f1337e00640 1 -- 172.21.15.47:0/946380205 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320035020 con 0x7f13380d0a70 2024-04-20T10:29:37.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.626+0000 7f1337e00640 1 -- 172.21.15.47:0/946380205 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1320031640 con 0x7f13380d0a70 2024-04-20T10:29:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f1337e00640 1 -- 172.21.15.47:0/946380205 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132002d040 con 0x7f13380d0a70 2024-04-20T10:29:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f133f400640 1 -- 172.21.15.47:0/946380205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 msgr2=0x7f13380ad050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f133f400640 1 --2- 172.21.15.47:0/946380205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13380ad050 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f133806e4e0 tx=0x7f132002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f133f400640 1 -- 172.21.15.47:0/946380205 shutdown_connections 2024-04-20T10:29:37.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f133f400640 1 --2- 172.21.15.47:0/946380205 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13380ad050 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f133f400640 1 -- 172.21.15.47:0/946380205 >> 172.21.15.47:0/946380205 conn(0x7f13380b3350 msgr2=0x7f13380b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f133f400640 1 -- 172.21.15.47:0/946380205 shutdown_connections 2024-04-20T10:29:37.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.627+0000 7f133f400640 1 -- 172.21.15.47:0/946380205 wait complete. 2024-04-20T10:29:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.629+0000 7f133f400640 1 Processor -- start 2024-04-20T10:29:37.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.629+0000 7f133f400640 1 -- start start 2024-04-20T10:29:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.630+0000 7f133f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133806f090 0x7f13381556a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.630+0000 7f133f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13381646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.630+0000 7f133f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1338162c00 0x7f1338158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.630+0000 7f133f400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f133806dbb0 con 0x7f13380d0a70 2024-04-20T10:29:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.630+0000 7f133f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f133806ec00 con 0x7f133806f090 2024-04-20T10:29:37.631 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.630+0000 7f133f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f133806dd30 con 0x7f1338162c00 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13381646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13381646b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53238/0 (socket says 172.21.15.47:53238) 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 -- 172.21.15.47:0/1934176360 learned_addr learned my addr 172.21.15.47:0/1934176360 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f133ca00640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133806f090 0x7f13381556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 -- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1338162c00 msgr2=0x7f1338158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1338162c00 0x7f1338158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 -- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133806f090 msgr2=0x7f13381556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133806f090 0x7f13381556a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 -- 172.21.15.47:0/1934176360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1338156800 con 0x7f13380d0a70 2024-04-20T10:29:37.632 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.631+0000 7f132fe00640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13381646b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1320000c00 tx=0x7f13200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.632+0000 7f1335600640 1 -- 172.21.15.47:0/1934176360 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132003ac60 con 0x7f13380d0a70 2024-04-20T10:29:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.632+0000 7f1335600640 1 -- 172.21.15.47:0/1934176360 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132003ae20 con 0x7f13380d0a70 2024-04-20T10:29:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.632+0000 7f1335600640 1 -- 172.21.15.47:0/1934176360 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320044630 con 0x7f13380d0a70 2024-04-20T10:29:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.632+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1338165850 con 0x7f13380d0a70 2024-04-20T10:29:37.633 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.632+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1338165290 con 0x7f13380d0a70 2024-04-20T10:29:37.634 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.634+0000 7f1335600640 1 -- 172.21.15.47:0/1934176360 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f132004b060 con 0x7f13380d0a70 2024-04-20T10:29:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.635+0000 7f1335600640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f131406c8f0 0x7f131406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.635 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.635+0000 7f1335600640 1 -- 172.21.15.47:0/1934176360 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f13200adde0 con 0x7f13380d0a70 2024-04-20T10:29:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.636+0000 7f1342211740 1 Processor -- start 2024-04-20T10:29:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.636+0000 7f1342211740 1 -- start start 2024-04-20T10:29:37.636 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.636+0000 7f133ca00640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f131406c8f0 0x7f131406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f1342211740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cc34e0 0x561040cc4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.638 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cbf930 0x561040cef770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f1342211740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561040c2b9d0 con 0x561040cc34e0 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561040bd2270 con 0x561040cbf930 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561040c27990 con 0x561040d0d460 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cc34e0 0x561040cc4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.637+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cc34e0 0x561040cc4fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53252/0 (socket says 172.21.15.47:53252) 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132fe00640 1 -- 172.21.15.47:0/258705111 learned_addr learned my addr 172.21.15.47:0/258705111 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f133ca00640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f131406c8f0 0x7f131406edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f133815d000 tx=0x7f1328008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f133ca00640 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f133d400640 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cbf930 0x561040cef770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132fe00640 1 -- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 msgr2=0x561040cc4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132fe00640 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132fe00640 1 -- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cbf930 msgr2=0x561040cef770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132fe00640 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cbf930 0x561040cef770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.639 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132fe00640 1 -- 172.21.15.47:0/258705111 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56104093e2d0 con 0x561040cc34e0 2024-04-20T10:29:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f133d400640 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cbf930 0x561040cef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132fe00640 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cc34e0 0x561040cc4fd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f132007b870 tx=0x7f132007b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a2125cc43f3fc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132d200640 1 -- 172.21.15.47:0/258705111 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132007bfe0 con 0x561040cc34e0 2024-04-20T10:29:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.638+0000 7f132d200640 1 -- 172.21.15.47:0/258705111 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132007b8e0 con 0x561040cc34e0 2024-04-20T10:29:37.640 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f132d200640 1 -- 172.21.15.47:0/258705111 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132007fc90 con 0x561040cc34e0 2024-04-20T10:29:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 -- 172.21.15.47:0/258705111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cc34e0 msgr2=0x561040cc4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cc34e0 0x561040cc4fd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f132007b870 tx=0x7f132007b7e0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 -- 172.21.15.47:0/258705111 shutdown_connections 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cbf930 0x561040cef770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cc34e0 0x561040cc4fd0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 --2- 172.21.15.47:0/258705111 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 -- 172.21.15.47:0/258705111 >> 172.21.15.47:0/258705111 conn(0x561040cc1bf0 msgr2=0x561040d0a770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 -- 172.21.15.47:0/258705111 shutdown_connections 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.639+0000 7f1342211740 1 -- 172.21.15.47:0/258705111 wait complete. 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.640+0000 7f1342211740 1 -- start start 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.640+0000 7f1342211740 1 Processor -- start 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.641+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040cd74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.641+0000 7f1342211740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040ceebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.641+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cd6770 0x561040cd6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.641+0000 7f1342211740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561040cc39f0 con 0x561040d0d460 2024-04-20T10:29:37.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.641+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561040be55e0 con 0x561040cc34e0 2024-04-20T10:29:37.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.641+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561040cc3cd0 con 0x561040cd6770 2024-04-20T10:29:37.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.641+0000 7f1342211740 10 client.0 _ll_register_callbacks cb 0x56104093dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:29:37.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040ceebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040ceebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53264/0 (socket says 172.21.15.47:53264) 2024-04-20T10:29:37.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 -- 172.21.15.47:0/2136856853 learned_addr learned my addr 172.21.15.47:0/2136856853 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f133ca00640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040cd74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f133d400640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cd6770 0x561040cd6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 -- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cd6770 msgr2=0x561040cd6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cd6770 0x561040cd6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 -- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 msgr2=0x561040cd74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040cd74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 -- 172.21.15.47:0/2136856853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561040a086b0 con 0x561040d0d460 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f133d400640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cd6770 0x561040cd6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f133ca00640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040cd74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:37.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f132fe00640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040ceebf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f132007b870 tx=0x7f1320031f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=279fb00d2b4036b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.642+0000 7f1325600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1325600640 1 -- 172.21.15.47:0/2136856853 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13200844c0 con 0x561040d0d460 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1325600640 1 -- 172.21.15.47:0/2136856853 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f132007bb40 con 0x561040d0d460 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f132007b8e0 con 0x561040d0d460 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1325600640 1 -- 172.21.15.47:0/2136856853 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f132007fc90 con 0x561040d0d460 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 10 client.5676 fetch_fsmap learned FSMap version 19 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x561040a086b0 con 0x561040d0d460 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1325600640 1 -- 172.21.15.47:0/2136856853 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132007b8e0 con 0x561040d0d460 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1325600640 1 -- 172.21.15.47:0/2136856853 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 3495+0+0 (secure 0 0 0) 0x7f1320091030 con 0x561040d0d460 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 10 client.5676 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 10 client.5676 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 20 client.5676 populate_metadata read hostname 'smithi047' 2024-04-20T10:29:37.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040d0f310 0x561040d0f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 4 client.5676 mds_command: new command op to 5481 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:29:37.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.643+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 --> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561040c393f0 con 0x561040d0f310 2024-04-20T10:29:37.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.644+0000 7f133ca00640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040d0f310 0x561040d0f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.644+0000 7f133ca00640 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040d0f310 0x561040d0f6f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4447470eb5f84253 server_cookie=49d34671741fc3a2 in_seq=0 out_seq=0 2024-04-20T10:29:37.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.644+0000 7f1325600640 10 client.5676 ms_handle_connect on v2:172.21.15.47:6834/1013167846 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.645+0000 7f1325600640 1 -- 172.21.15.47:0/2136856853 <== mds.0 v2:172.21.15.47:6834/1013167846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561040c393f0 con 0x561040d0f310 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.645+0000 7f1325600640 10 client.5676 handle_command_reply: tid=0 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.645+0000 7f1342211740 1 client.5676 shutdown 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 msgr2=0x561040ceebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040ceebf0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f132007b870 tx=0x7f1320031f70 comp rx=0 tx=0).stop 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 shutdown_connections 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040d0f310 0x561040d0f6f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cd6770 0x561040cd6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040ceebf0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 --2- 172.21.15.47:0/2136856853 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040cd74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1325600640 0 client.5676 ms_handle_reset on v2:172.21.15.47:6834/1013167846 2024-04-20T10:29:37.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 >> 172.21.15.47:0/2136856853 conn(0x561040cbf930 msgr2=0x561040cbefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:37.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 shutdown_connections 2024-04-20T10:29:37.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 1 -- 172.21.15.47:0/2136856853 wait complete. 2024-04-20T10:29:37.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.646+0000 7f1342211740 20 client.5676 trim_cache size 0 max 16384 2024-04-20T10:29:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.666+0000 7f1342211740 1 Processor -- start 2024-04-20T10:29:37.666 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.666+0000 7f1342211740 1 -- start start 2024-04-20T10:29:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cc34e0 0x561040ceeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cd6770 0x561040cc4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f1342211740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040d0f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f1342211740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1338204840 con 0x561040d0d460 2024-04-20T10:29:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1338157a00 con 0x561040cd6770 2024-04-20T10:29:37.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1338204340 con 0x561040cc34e0 2024-04-20T10:29:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f133d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040d0f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f133d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040d0f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53274/0 (socket says 172.21.15.47:53274) 2024-04-20T10:29:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.667+0000 7f133d400640 1 -- 172.21.15.47:0/2689803401 learned_addr learned my addr 172.21.15.47:0/2689803401 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.668 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f133ca00640 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cc34e0 0x561040ceeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f132fe00640 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cd6770 0x561040cc4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f133d400640 1 -- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cc34e0 msgr2=0x561040ceeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f133d400640 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cc34e0 0x561040ceeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f133d400640 1 -- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cd6770 msgr2=0x561040cc4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f133d400640 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cd6770 0x561040cc4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f133d400640 1 -- 172.21.15.47:0/2689803401 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561040cc5250 con 0x561040d0d460 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f133d400640 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040d0f6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1330001c20 tx=0x7f13300020e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634927eb2295c15 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f1327e00640 1 -- 172.21.15.47:0/2689803401 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1330003d80 con 0x561040d0d460 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f1327e00640 1 -- 172.21.15.47:0/2689803401 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133000e8c0 con 0x561040d0d460 2024-04-20T10:29:37.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.668+0000 7f1327e00640 1 -- 172.21.15.47:0/2689803401 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f133000ec70 con 0x561040d0d460 2024-04-20T10:29:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 -- 172.21.15.47:0/2689803401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 msgr2=0x561040d0f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040d0f6f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1330001c20 tx=0x7f13300020e0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 -- 172.21.15.47:0/2689803401 shutdown_connections 2024-04-20T10:29:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040d0d460 0x561040d0f6f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cd6770 0x561040cc4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 --2- 172.21.15.47:0/2689803401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040cc34e0 0x561040ceeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 -- 172.21.15.47:0/2689803401 >> 172.21.15.47:0/2689803401 conn(0x561040cbf930 msgr2=0x561040cc5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:37.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 -- 172.21.15.47:0/2689803401 shutdown_connections 2024-04-20T10:29:37.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.669+0000 7f1342211740 1 -- 172.21.15.47:0/2689803401 wait complete. 2024-04-20T10:29:37.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.670+0000 7f1342211740 1 -- start start 2024-04-20T10:29:37.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.670+0000 7f1342211740 1 Processor -- start 2024-04-20T10:29:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040d85170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f1342211740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cd6770 0x561040cc55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f1342211740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f1342211740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561040cc39f0 con 0x561040cd6770 2024-04-20T10:29:37.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561040be55e0 con 0x561040cc34e0 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f1342211740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561040ce83f0 con 0x561040d0d460 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f1342211740 10 client.0 _ll_register_callbacks cb 0x7f132007b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cd6770 0x561040cc55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f132fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cd6770 0x561040cc55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53276/0 (socket says 172.21.15.47:53276) 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.671+0000 7f132fe00640 1 -- 172.21.15.47:0/2452947349 learned_addr learned my addr 172.21.15.47:0/2452947349 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f133d400640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f132fe00640 1 -- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 msgr2=0x561040cc5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f133ca00640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040d85170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f132fe00640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f132fe00640 1 -- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 msgr2=0x561040d85170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f132fe00640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040d85170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f132fe00640 1 -- 172.21.15.47:0/2452947349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561040d86590 con 0x561040cd6770 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f133ca00640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040d85170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:37.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f132fe00640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cd6770 0x561040cc55a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f132007b5e0 tx=0x7f1320046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ae84a80a54f669 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f1325600640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f1325600640 1 -- 172.21.15.47:0/2452947349 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13200a5070 con 0x561040cd6770 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f1325600640 1 -- 172.21.15.47:0/2452947349 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f1320084230 con 0x561040cd6770 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.672+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f133000e8c0 con 0x561040cd6770 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1325600640 1 -- 172.21.15.47:0/2452947349 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) ==== 24+0+0 (secure 0 0 0) 0x7f132008a050 con 0x561040cd6770 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 10 client.5682 fetch_fsmap learned FSMap version 19 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) -- 0x561040d86590 con 0x561040cd6770 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1325600640 1 -- 172.21.15.47:0/2452947349 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f133000e8c0 con 0x561040cd6770 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1325600640 1 -- 172.21.15.47:0/2452947349 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 19) ==== 3495+0+0 (secure 0 0 0) 0x7f132008d020 con 0x561040cd6770 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 10 client.5682 fetch_fsmap finished waiting for FSMap version 19 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 10 client.5682 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 20 client.5682 populate_metadata read hostname 'smithi047' 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040ceb570 0x561040d1d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 4 client.5682 mds_command: new command op to 5481 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:29:37.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 --> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] -- command(tid 0: {"prefix": "status"}) -- 0x561040c33260 con 0x561040ceb570 2024-04-20T10:29:37.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.673+0000 7f133ca00640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040ceb570 0x561040d1d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f133ca00640 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040ceb570 0x561040d1d6f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b48d425ef9ce92da server_cookie=2bc5367d11e46754 in_seq=0 out_seq=0 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1325600640 10 client.5682 ms_handle_connect on v2:172.21.15.47:6834/1013167846 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1325600640 1 -- 172.21.15.47:0/2452947349 <== mds.0 v2:172.21.15.47:6834/1013167846 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x561040c33260 con 0x561040ceb570 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1325600640 10 client.5682 handle_command_reply: tid=0 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 client.5682 shutdown 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cd6770 msgr2=0x561040cc55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cd6770 0x561040cc55a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f132007b5e0 tx=0x7f1320046000 comp rx=0 tx=0).stop 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 shutdown_connections 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:6834/1013167846,v1:172.21.15.47:6835/1013167846] conn(0x561040ceb570 0x561040d1d6f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561040d0d460 0x561040cc5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561040cd6770 0x561040cc55a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1325600640 0 client.5682 ms_handle_reset on v2:172.21.15.47:6834/1013167846 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 --2- 172.21.15.47:0/2452947349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561040cc34e0 0x561040d85170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.674+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 >> 172.21.15.47:0/2452947349 conn(0x561040cbf930 msgr2=0x561040d0f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 shutdown_connections 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f1342211740 1 -- 172.21.15.47:0/2452947349 wait complete. 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f1342211740 20 client.5682 trim_cache size 0 max 16384 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "775946e6-93c9-4843-aaf4-db758801eec1", 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5481, 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-20T10:29:37.678 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-20T10:29:37.679 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-20T10:29:37.679 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 11.768208346, 2024-04-20T10:29:37.679 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 19, 2024-04-20T10:29:37.679 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 42, 2024-04-20T10:29:37.679 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 42, 2024-04-20T10:29:37.679 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 21.759847443999998 2024-04-20T10:29:37.679 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f131406c8f0 msgr2=0x7f131406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f131406c8f0 0x7f131406edb0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f133815d000 tx=0x7f1328008040 comp rx=0 tx=0).stop 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 msgr2=0x7f13381646b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13381646b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1320000c00 tx=0x7f13200046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 shutdown_connections 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1338162c00 0x7f1338158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f131406c8f0 0x7f131406edb0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13380d0a70 0x7f13381646b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 --2- 172.21.15.47:0/1934176360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f133806f090 0x7f13381556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.675+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 >> 172.21.15.47:0/1934176360 conn(0x7f13380b3350 msgr2=0x7f13380b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.676+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 shutdown_connections 2024-04-20T10:29:37.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.676+0000 7f133f400640 1 -- 172.21.15.47:0/1934176360 wait complete. 2024-04-20T10:29:37.692 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:29:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.898+0000 7f6bd3e00640 1 Processor -- start 2024-04-20T10:29:37.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.898+0000 7f6bd3e00640 1 -- start start 2024-04-20T10:29:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.899+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd4161d50 0x7f6bd40d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.899+0000 7f6bd3e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6bd40b3330 0x7f6bd40b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.899+0000 7f6bd3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bd406c6d0 0x7f6bd406cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.899+0000 7f6bd3e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd4162130 con 0x7f6bd406c6d0 2024-04-20T10:29:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.899+0000 7f6bd3e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6bd406c190 con 0x7f6bd40b3330 2024-04-20T10:29:37.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.899+0000 7f6bd3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd406c470 con 0x7f6bd4161d50 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bd406c6d0 0x7f6bd406cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6bd406c6d0 0x7f6bd406cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53288/0 (socket says 172.21.15.47:53288) 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd3400640 1 -- 172.21.15.47:0/1713666998 learned_addr learned my addr 172.21.15.47:0/1713666998 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd2000640 1 --2- 172.21.15.47:0/1713666998 >> v2:172.21.15.175:3300/0 conn(0x7f6bd40b3330 0x7f6bd40b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd2a00640 1 --2- 172.21.15.47:0/1713666998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd4161d50 0x7f6bd40d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd3400640 1 -- 172.21.15.47:0/1713666998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd4161d50 msgr2=0x7f6bd40d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd3400640 1 --2- 172.21.15.47:0/1713666998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd4161d50 0x7f6bd40d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.900+0000 7f6bd3400640 1 -- 172.21.15.47:0/1713666998 >> v2:172.21.15.175:3300/0 conn(0x7f6bd40b3330 msgr2=0x7f6bd40b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.901+0000 7f6bd3400640 1 --2- 172.21.15.47:0/1713666998 >> v2:172.21.15.175:3300/0 conn(0x7f6bd40b3330 0x7f6bd40b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.901+0000 7f6bd3400640 1 -- 172.21.15.47:0/1713666998 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd40d2e00 con 0x7f6bd406c6d0 2024-04-20T10:29:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.902+0000 7f6bd3400640 1 --2- 172.21.15.47:0/1713666998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd406c6d0 0x7f6bd406cb60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40b7000 tx=0x7f6bcc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c403eeaead7b93 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.902+0000 7f6bd1600640 1 -- 172.21.15.47:0/1713666998 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc035020 con 0x7f6bd406c6d0 2024-04-20T10:29:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.902+0000 7f6bd1600640 1 -- 172.21.15.47:0/1713666998 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bcc031870 con 0x7f6bd406c6d0 2024-04-20T10:29:37.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.902+0000 7f6bd1600640 1 -- 172.21.15.47:0/1713666998 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bcc031c40 con 0x7f6bd406c6d0 2024-04-20T10:29:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1713666998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd406c6d0 msgr2=0x7f6bd406cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1713666998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd406c6d0 0x7f6bd406cb60 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6bd40b7000 tx=0x7f6bcc02b910 comp rx=0 tx=0).stop 2024-04-20T10:29:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1713666998 shutdown_connections 2024-04-20T10:29:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1713666998 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd406c6d0 0x7f6bd406cb60 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1713666998 >> v2:172.21.15.175:3300/0 conn(0x7f6bd40b3330 0x7f6bd40b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/1713666998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd4161d50 0x7f6bd40d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1713666998 >> 172.21.15.47:0/1713666998 conn(0x7f6bd40d4450 msgr2=0x7f6bd40d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:37.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1713666998 shutdown_connections 2024-04-20T10:29:37.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.903+0000 7f6bd3e00640 1 -- 172.21.15.47:0/1713666998 wait complete. 2024-04-20T10:29:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.905+0000 7f6bd3e00640 1 Processor -- start 2024-04-20T10:29:37.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.905+0000 7f6bd3e00640 1 -- start start 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd406c6d0 0x7f6bd414c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd40b3330 0x7f6bd415d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bd4161d50 0x7f6bd415d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd40cfa00 con 0x7f6bd40b3330 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6bd40d33c0 con 0x7f6bd4161d50 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bd40cfb80 con 0x7f6bd406c6d0 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd40b3330 0x7f6bd415d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd40b3330 0x7f6bd415d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53296/0 (socket says 172.21.15.47:53296) 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.906+0000 7f6bd2000640 1 -- 172.21.15.47:0/3520787964 learned_addr learned my addr 172.21.15.47:0/3520787964 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd2000640 1 -- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd406c6d0 msgr2=0x7f6bd414c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd2000640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd406c6d0 0x7f6bd414c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd2000640 1 -- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bd4161d50 msgr2=0x7f6bd415d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:37.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd2000640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bd4161d50 0x7f6bd415d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd2000640 1 -- 172.21.15.47:0/3520787964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bd4159730 con 0x7f6bd40b3330 2024-04-20T10:29:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd2000640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd40b3330 0x7f6bd415d470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00d800 tx=0x7f6bbc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bcb400640 1 -- 172.21.15.47:0/3520787964 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc018020 con 0x7f6bd40b3330 2024-04-20T10:29:37.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bcb400640 1 -- 172.21.15.47:0/3520787964 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6bbc0096a0 con 0x7f6bd40b3330 2024-04-20T10:29:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bd41599f0 con 0x7f6bd40b3330 2024-04-20T10:29:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bcb400640 1 -- 172.21.15.47:0/3520787964 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc009a70 con 0x7f6bd40b3330 2024-04-20T10:29:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.907+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bd4159fb0 con 0x7f6bd40b3330 2024-04-20T10:29:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.908+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bd4160f20 con 0x7f6bd40b3330 2024-04-20T10:29:37.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.910+0000 7f6bcb400640 1 -- 172.21.15.47:0/3520787964 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6bbc01b020 con 0x7f6bd40b3330 2024-04-20T10:29:37.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.910+0000 7f6bcb400640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bb006c8f0 0x7f6bb006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:37.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.911+0000 7f6bcb400640 1 -- 172.21.15.47:0/3520787964 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f6bbc08bee0 con 0x7f6bd40b3330 2024-04-20T10:29:37.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.911+0000 7f6bd2a00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bb006c8f0 0x7f6bb006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:37.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.912+0000 7f6bd2a00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bb006c8f0 0x7f6bb006edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40070b0 tx=0x7f6bc4007040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:37.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:37.915+0000 7f6bcb400640 1 -- 172.21.15.47:0/3520787964 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6bbc022030 con 0x7f6bd40b3330 2024-04-20T10:29:38.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.165+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6bd414cd20 con 0x7f6bd40b3330 2024-04-20T10:29:38.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.166+0000 7f6bcb400640 1 -- 172.21.15.47:0/3520787964 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) ==== 53+0+73 (secure 0 0 0) 0x7f6bbc05a160 con 0x7f6bd40b3330 2024-04-20T10:29:38.166 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.168+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bb006c8f0 msgr2=0x7f6bb006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.168+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bb006c8f0 0x7f6bb006edb0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6bc40070b0 tx=0x7f6bc4007040 comp rx=0 tx=0).stop 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.168+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd40b3330 msgr2=0x7f6bd415d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.168+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd40b3330 0x7f6bd415d470 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00d800 tx=0x7f6bbc00dcc0 comp rx=0 tx=0).stop 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 shutdown_connections 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bd4161d50 0x7f6bd415d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6bb006c8f0 0x7f6bb006edb0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6bd40b3330 0x7f6bd415d470 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 --2- 172.21.15.47:0/3520787964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bd406c6d0 0x7f6bd414c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:38.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 >> 172.21.15.47:0/3520787964 conn(0x7f6bd40d4450 msgr2=0x7f6bd4160350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 shutdown_connections 2024-04-20T10:29:38.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38.169+0000 7f6bd3e00640 1 -- 172.21.15.47:0/3520787964 wait complete. 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:29:38.188 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:29:38.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:38.189 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-20T10:29:38.239 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42704sec preferred_lft 42704sec 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:38.240 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::50dc:baff:fe1b:7c56/64 scope link 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::8014:63ff:fed5:b599/64 scope link 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether aa:c8:12:38:db:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::a8c8:12ff:fe38:db8e/64 scope link 2024-04-20T10:29:38.241 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:38.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:38.242 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:29:38.325 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-20T10:29:38.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:38.329 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-20T10:29:38.384 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-20T10:29:38.384 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:29:38.384 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:29:38.384 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:29:38.385 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:29:38.386 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:29:38.386 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:29:38.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:29:38.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:29:38.386 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:29:38.386 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:29:38.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:29:38.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:29:38.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:38.602 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:29:38.653 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:29:38.653 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T10:29:38.654 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:29:38.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:38.654 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-20T10:29:38.742 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T10:29:38.742 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:29:38.742 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:38.798 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:29:38.798 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:29:38.798 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:38.853 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:29:38.943 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:29:39.000 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:29:39.001 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:39.002 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:39.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:39.003 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:29:39.054 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:29:39.054 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T10:29:39.060 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:29:39.131 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:29:39.171 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:29:39.172 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:29:39.173 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:29:39.174 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:29:39.174 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:29:39.174 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:29:39.174 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:39.174 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:39.174 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:39.174 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:39.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:39.175 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:29:39.236 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-04-20T10:29:39.237+0000 7f994dc46480 -1 init, newargv = 0x55d7cf8a3f60 newargc=15 2024-04-20T10:29:39.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[37648]: starting ceph client 2024-04-20T10:29:39.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[37648]: starting fuse 2024-04-20T10:29:39.263 INFO:teuthology.orchestra.run.smithi047.stdout:42 2024-04-20T10:29:39.263 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T10:29:39.264 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:29:39.265 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:39.266 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-04-20T10:29:39.266 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:29:39.447 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.37648.asok 2024-04-20T10:29:39.451 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:29:39.463 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.37648.asok 2024-04-20T10:29:39.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:39.463 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.37648.asok status 2024-04-20T10:29:39.702 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:29:39.702 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "37648", 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5694, 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5694 2024-04-20T10:29:39.703 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2024423885 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 2024423885 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5694 192.168.0.1:0/2024423885", 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/2024423885", 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-04-20T10:29:39.704 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 19, 2024-04-20T10:29:39.705 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 42, 2024-04-20T10:29:39.705 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:29:39.705 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-04-20T10:29:39.705 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-04-20T10:29:39.705 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:29:39.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:39.717 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-04-20T10:29:39.718 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:39.769 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-04-20T10:29:39.769 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:39.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:39.769 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:39.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:39.855 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir1) 2024-04-20T10:29:39.924 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:39.924 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch testdir1/testfile) 2024-04-20T10:29:39.990 DEBUG:teuthology.orchestra.run.smithi047:> stdin-killer -- python3 -c ' 2024-04-20T10:29:39.990 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:29:39.990 DEBUG:teuthology.orchestra.run.smithi047:> import stat 2024-04-20T10:29:39.990 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:39.990 DEBUG:teuthology.orchestra.run.smithi047:> print(os.stat("/home/ubuntu/cephtest/mnt.0/testdir1").st_ino) 2024-04-20T10:29:39.990 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:29:40.139 INFO:teuthology.orchestra.run.smithi047.stdout:1099511627776 2024-04-20T10:29:40.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:29:40.154 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:29:40.154 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:29:40.253 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.468+0000 7fe9de600640 1 Processor -- start 2024-04-20T10:29:40.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.468+0000 7fe9de600640 1 -- start start 2024-04-20T10:29:40.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.469+0000 7fe9de600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9d80d1a00 0x7fe9d80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.469+0000 7fe9de600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe9d80b17d0 0x7fe9d80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.469+0000 7fe9de600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d80d1e90 con 0x7fe9d80d4270 2024-04-20T10:29:40.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.469+0000 7fe9de600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d80d40c0 con 0x7fe9d80d1a00 2024-04-20T10:29:40.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.469+0000 7fe9de600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d80d3820 con 0x7fe9d80b17d0 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9d80d1a00 0x7fe9d80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9d80d1a00 0x7fe9d80b1290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53300/0 (socket says 172.21.15.47:53300) 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d6a00640 1 -- 172.21.15.47:0/2689563727 learned_addr learned my addr 172.21.15.47:0/2689563727 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d7e00640 1 --2- 172.21.15.47:0/2689563727 >> v2:172.21.15.175:3300/0 conn(0x7fe9d80b17d0 0x7fe9d80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d6a00640 1 -- 172.21.15.47:0/2689563727 >> v2:172.21.15.175:3300/0 conn(0x7fe9d80b17d0 msgr2=0x7fe9d80d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d6a00640 1 --2- 172.21.15.47:0/2689563727 >> v2:172.21.15.175:3300/0 conn(0x7fe9d80b17d0 0x7fe9d80d0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d6a00640 1 -- 172.21.15.47:0/2689563727 >> v1:172.21.15.175:6789/0 conn(0x7fe9d80d4270 legacy=0x7fe9d80d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:40.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.470+0000 7fe9d6a00640 1 -- 172.21.15.47:0/2689563727 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d80d3b00 con 0x7fe9d80d1a00 2024-04-20T10:29:40.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.471+0000 7fe9d6a00640 1 --2- 172.21.15.47:0/2689563727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80b1290 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8068550 tx=0x7fe9c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb46a0d9a6bd8ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:40.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.472+0000 7fe9d6000640 1 -- 172.21.15.47:0/2689563727 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0035020 con 0x7fe9d80d1a00 2024-04-20T10:29:40.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.472+0000 7fe9d6000640 1 -- 172.21.15.47:0/2689563727 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c0031870 con 0x7fe9d80d1a00 2024-04-20T10:29:40.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.472+0000 7fe9d6000640 1 -- 172.21.15.47:0/2689563727 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0031c40 con 0x7fe9d80d1a00 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.472+0000 7fe9de600640 1 -- 172.21.15.47:0/2689563727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d1a00 msgr2=0x7fe9d80b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.472+0000 7fe9de600640 1 --2- 172.21.15.47:0/2689563727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80b1290 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fe9d8068550 tx=0x7fe9c002b910 comp rx=0 tx=0).stop 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.473+0000 7fe9de600640 1 -- 172.21.15.47:0/2689563727 shutdown_connections 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.473+0000 7fe9de600640 1 --2- 172.21.15.47:0/2689563727 >> v2:172.21.15.175:3300/0 conn(0x7fe9d80b17d0 0x7fe9d80d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.473+0000 7fe9de600640 1 --2- 172.21.15.47:0/2689563727 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80b1290 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.473+0000 7fe9de600640 1 -- 172.21.15.47:0/2689563727 >> 172.21.15.47:0/2689563727 conn(0x7fe9d80b99c0 msgr2=0x7fe9d80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.473+0000 7fe9de600640 1 -- 172.21.15.47:0/2689563727 shutdown_connections 2024-04-20T10:29:40.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.473+0000 7fe9de600640 1 -- 172.21.15.47:0/2689563727 wait complete. 2024-04-20T10:29:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.475+0000 7fe9de600640 1 Processor -- start 2024-04-20T10:29:40.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.475+0000 7fe9de600640 1 -- start start 2024-04-20T10:29:40.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.476+0000 7fe9de600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d80b17d0 0x7fe9d80ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.476+0000 7fe9de600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80d2830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.476+0000 7fe9de600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d4270 0x7fe9d80d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.476+0000 7fe9de600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d80d53b0 con 0x7fe9d80d4270 2024-04-20T10:29:40.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.476+0000 7fe9de600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d80cfe60 con 0x7fe9d80d1a00 2024-04-20T10:29:40.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.476+0000 7fe9de600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d8161f70 con 0x7fe9d80b17d0 2024-04-20T10:29:40.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d4270 0x7fe9d80d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d4270 0x7fe9d80d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53312/0 (socket says 172.21.15.47:53312) 2024-04-20T10:29:40.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 -- 172.21.15.47:0/2608488554 learned_addr learned my addr 172.21.15.47:0/2608488554 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:40.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7400640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d80b17d0 0x7fe9d80ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d6a00640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80d2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 -- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d80b17d0 msgr2=0x7fe9d80ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d80b17d0 0x7fe9d80ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 -- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d80d1a00 msgr2=0x7fe9d80d2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80d2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7e00640 1 -- 172.21.15.47:0/2608488554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d806e3c0 con 0x7fe9d80d4270 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d6a00640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.477+0000 7fe9d7400640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d80b17d0 0x7fe9d80ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.478+0000 7fe9d7e00640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d4270 0x7fe9d80d2d70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0007e30 tx=0x7fe9d000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.478+0000 7fe9cf400640 1 -- 172.21.15.47:0/2608488554 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0014620 con 0x7fe9d80d4270 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.478+0000 7fe9cf400640 1 -- 172.21.15.47:0/2608488554 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9d0019070 con 0x7fe9d80d4270 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.478+0000 7fe9cf400640 1 -- 172.21.15.47:0/2608488554 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9d0014a00 con 0x7fe9d80d4270 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.478+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d80cd090 con 0x7fe9d80d4270 2024-04-20T10:29:40.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.478+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d80cd650 con 0x7fe9d80d4270 2024-04-20T10:29:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.480+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d8068980 con 0x7fe9d80d4270 2024-04-20T10:29:40.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.480+0000 7fe9cf400640 1 -- 172.21.15.47:0/2608488554 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9d0014ba0 con 0x7fe9d80d4270 2024-04-20T10:29:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.481+0000 7fe9cf400640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.482+0000 7fe9cf400640 1 -- 172.21.15.47:0/2608488554 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fe9d008c690 con 0x7fe9d80d4270 2024-04-20T10:29:40.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.482+0000 7fe9d7400640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.483+0000 7fe9d7400640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800a830 tx=0x7fe9c8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:40.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.487+0000 7fe9cf400640 1 -- 172.21.15.47:0/2608488554 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9d806e3c0 con 0x7fe9d80d4270 2024-04-20T10:29:40.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.700+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9d81612e0 con 0x7fe9d80d4270 2024-04-20T10:29:40.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.702+0000 7fe9cf400640 1 -- 172.21.15.47:0/2608488554 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16954 (secure 0 0 0) 0x7fe9d005a890 con 0x7fe9d80d4270 2024-04-20T10:29:40.703 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:40.703 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:26.904749+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 msgr2=0x7fe9b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe9c800a830 tx=0x7fe9c8008040 comp rx=0 tx=0).stop 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d4270 msgr2=0x7fe9d80d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d4270 0x7fe9d80d2d70 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0007e30 tx=0x7fe9d000e4b0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 shutdown_connections 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d80d4270 0x7fe9d80d2d70 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d80d1a00 0x7fe9d80d2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 --2- 172.21.15.47:0/2608488554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d80b17d0 0x7fe9d80ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.705+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 >> 172.21.15.47:0/2608488554 conn(0x7fe9d80b99c0 msgr2=0x7fe9d80b3020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.706+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 shutdown_connections 2024-04-20T10:29:40.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.706+0000 7fe9de600640 1 -- 172.21.15.47:0/2608488554 wait complete. 2024-04-20T10:29:40.726 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:29:23.363565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:29:24.330114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:29:40.726 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:29:40.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.947+0000 7fc713a00640 1 Processor -- start 2024-04-20T10:29:40.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.947+0000 7fc713a00640 1 -- start start 2024-04-20T10:29:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc713a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc70c15ca40 0x7fc70c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc713a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c0ad8e0 0x7fc70c0d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc713a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc70c141c60 con 0x7fc70c156c20 2024-04-20T10:29:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc713a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc70c06c010 con 0x7fc70c15ca40 2024-04-20T10:29:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc713a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c15cf90 con 0x7fc70c0ad8e0 2024-04-20T10:29:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc711000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc70c156c20 0x7fc70c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40206/0 (socket says 172.21.15.47:40206) 2024-04-20T10:29:40.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc711000640 1 -- 172.21.15.47:0/2103380812 learned_addr learned my addr 172.21.15.47:0/2103380812 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.949+0000 7fc703e00640 1 --2- 172.21.15.47:0/2103380812 >> v2:172.21.15.175:3300/0 conn(0x7fc70c15ca40 0x7fc70c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2998616657 0 0) 0x7fc70c141c60 con 0x7fc70c156c20 2024-04-20T10:29:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc711a00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c0ad8e0 0x7fc70c0d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc6f0003520 con 0x7fc70c156c20 2024-04-20T10:29:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2942066378 0 0) 0x7fc6f0003520 con 0x7fc70c156c20 2024-04-20T10:29:40.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c0ad8e0 msgr2=0x7fc70c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c0ad8e0 0x7fc70c0d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 >> v2:172.21.15.175:3300/0 conn(0x7fc70c15ca40 msgr2=0x7fc70c0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 --2- 172.21.15.47:0/2103380812 >> v2:172.21.15.175:3300/0 conn(0x7fc70c15ca40 0x7fc70c0ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c15d270 con 0x7fc70c156c20 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fc6fc001670 con 0x7fc70c156c20 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 >> v1:172.21.15.47:6789/0 conn(0x7fc70c156c20 legacy=0x7fc70c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f00039b0 0x7fc6f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f0004800 0x7fc6f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c0ad8e0 0x7fc6f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c06c010 con 0x7fc6f0004800 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c15cf90 con 0x7fc6f00039b0 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6f000b9f0 con 0x7fc70c0ad8e0 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.950+0000 7fc703e00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f0004800 0x7fc6f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.951+0000 7fc711000640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f00039b0 0x7fc6f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.951+0000 7fc703e00640 1 -- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c0ad8e0 msgr2=0x7fc6f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:40.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.951+0000 7fc703e00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c0ad8e0 0x7fc6f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.951+0000 7fc703e00640 1 -- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f00039b0 msgr2=0x7fc6f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:40.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.951+0000 7fc703e00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f00039b0 0x7fc6f0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.951+0000 7fc703e00640 1 -- 172.21.15.47:0/2103380812 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f0003520 con 0x7fc6f0004800 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.952+0000 7fc703e00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f0004800 0x7fc6f0004c80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009ac0 tx=0x7fc6f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b566686ddae3ee server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.952+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4039070 con 0x7fc6f0004800 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.952+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc6f402c710 con 0x7fc6f0004800 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.952+0000 7fc70be00640 1 -- 172.21.15.47:0/2103380812 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f4009690 con 0x7fc6f0004800 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.953+0000 7fc713a00640 1 -- 172.21.15.47:0/2103380812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f0004800 msgr2=0x7fc6f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.953+0000 7fc713a00640 1 --2- 172.21.15.47:0/2103380812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc6f0004800 0x7fc6f0004c80 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009ac0 tx=0x7fc6f0005830 comp rx=0 tx=0).stop 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.953+0000 7fc711a00640 1 -- 172.21.15.47:0/2103380812 reap_dead start 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.953+0000 7fc713a00640 1 -- 172.21.15.47:0/2103380812 shutdown_connections 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.953+0000 7fc713a00640 1 -- 172.21.15.47:0/2103380812 >> 172.21.15.47:0/2103380812 conn(0x7fc70c0b3350 msgr2=0x7fc70c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.953+0000 7fc713a00640 1 -- 172.21.15.47:0/2103380812 shutdown_connections 2024-04-20T10:29:40.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.953+0000 7fc713a00640 1 -- 172.21.15.47:0/2103380812 wait complete. 2024-04-20T10:29:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.955+0000 7fc713a00640 1 Processor -- start 2024-04-20T10:29:40.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.955+0000 7fc713a00640 1 -- start start 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc713a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f0004800 0x7fc70c1f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc713a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c14c2a0 0x7fc70c149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc713a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c073af0 0x7fc70c072ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc713a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c0d0360 con 0x7fc70c073af0 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc713a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c15d830 con 0x7fc6f0004800 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc713a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc70c0d04e0 con 0x7fc70c14c2a0 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc711a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c073af0 0x7fc70c072ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc711a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c073af0 0x7fc70c072ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53332/0 (socket says 172.21.15.47:53332) 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.956+0000 7fc711a00640 1 -- 172.21.15.47:0/4044344461 learned_addr learned my addr 172.21.15.47:0/4044344461 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc711000640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f0004800 0x7fc70c1f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc711a00640 1 -- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c14c2a0 msgr2=0x7fc70c149710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc711a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c14c2a0 0x7fc70c149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc711a00640 1 -- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f0004800 msgr2=0x7fc70c1f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc711a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f0004800 0x7fc70c1f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc711a00640 1 -- 172.21.15.47:0/4044344461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c09c3b0 con 0x7fc70c073af0 2024-04-20T10:29:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc711a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c073af0 0x7fc70c072ba0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0d00d0 tx=0x7fc704007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc709600640 1 -- 172.21.15.47:0/4044344461 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704003990 con 0x7fc70c073af0 2024-04-20T10:29:40.958 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc709600640 1 -- 172.21.15.47:0/4044344461 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc704003b50 con 0x7fc70c073af0 2024-04-20T10:29:40.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc709600640 1 -- 172.21.15.47:0/4044344461 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70400e960 con 0x7fc70c073af0 2024-04-20T10:29:40.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.957+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c09c690 con 0x7fc70c073af0 2024-04-20T10:29:40.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.958+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c09cc50 con 0x7fc70c073af0 2024-04-20T10:29:40.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.959+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c068980 con 0x7fc70c073af0 2024-04-20T10:29:40.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.960+0000 7fc709600640 1 -- 172.21.15.47:0/4044344461 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc70401b020 con 0x7fc70c073af0 2024-04-20T10:29:40.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.961+0000 7fc709600640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc6e806c920 0x7fc6e806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:40.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.961+0000 7fc709600640 1 -- 172.21.15.47:0/4044344461 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fc70408b8c0 con 0x7fc70c073af0 2024-04-20T10:29:40.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.961+0000 7fc711000640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc6e806c920 0x7fc6e806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:40.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.963+0000 7fc711000640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc6e806c920 0x7fc6e806ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000f60 tx=0x7fc6fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:40.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:40.965+0000 7fc709600640 1 -- 172.21.15.47:0/4044344461 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc7040122e0 con 0x7fc70c073af0 2024-04-20T10:29:41.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.170+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc70c0b3950 con 0x7fc70c073af0 2024-04-20T10:29:41.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.171+0000 7fc709600640 1 -- 172.21.15.47:0/4044344461 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fc704059b50 con 0x7fc70c073af0 2024-04-20T10:29:41.171 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:29:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc6e806c920 msgr2=0x7fc6e806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc6e806c920 0x7fc6e806ede0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc000f60 tx=0x7fc6fc009040 comp rx=0 tx=0).stop 2024-04-20T10:29:41.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c073af0 msgr2=0x7fc70c072ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c073af0 0x7fc70c072ba0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc70c0d00d0 tx=0x7fc704007ec0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 shutdown_connections 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc6e806c920 0x7fc6e806ede0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc70c073af0 0x7fc70c072ba0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc70c14c2a0 0x7fc70c149710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 --2- 172.21.15.47:0/4044344461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6f0004800 0x7fc70c1f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 >> 172.21.15.47:0/4044344461 conn(0x7fc70c0b3350 msgr2=0x7fc70c154600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 shutdown_connections 2024-04-20T10:29:41.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.173+0000 7fc713a00640 1 -- 172.21.15.47:0/4044344461 wait complete. 2024-04-20T10:29:41.192 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:29:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.403+0000 7f4038400640 1 Processor -- start 2024-04-20T10:29:41.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.404+0000 7f4038400640 1 -- start start 2024-04-20T10:29:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.405+0000 7f4038400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4030071b50 0x7f40300adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.405+0000 7f4038400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403006e840 con 0x7f403015abb0 2024-04-20T10:29:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.405+0000 7f4038400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f403015b350 con 0x7f4030072cc0 2024-04-20T10:29:41.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.405+0000 7f4038400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f403006eb00 con 0x7f4030071b50 2024-04-20T10:29:41.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.405+0000 7f4035a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f403015abb0 0x7f40300725b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40216/0 (socket says 172.21.15.47:40216) 2024-04-20T10:29:41.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.405+0000 7f4035a00640 1 -- 172.21.15.47:0/586501185 learned_addr learned my addr 172.21.15.47:0/586501185 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:41.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f4036400640 1 --2- 172.21.15.47:0/586501185 >> v2:172.21.15.175:3300/0 conn(0x7f4030071b50 0x7f40300adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 390495284 0 0) 0x7f403006e840 con 0x7f403015abb0 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4018003520 con 0x7f403015abb0 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 708609406 0 0) 0x7f4018003520 con 0x7f403015abb0 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 >> v2:172.21.15.175:3300/0 conn(0x7f4030071b50 msgr2=0x7f40300adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 --2- 172.21.15.47:0/586501185 >> v2:172.21.15.175:3300/0 conn(0x7f4030071b50 0x7f40300adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 >> v1:172.21.15.175:6789/0 conn(0x7f4030072cc0 legacy=0x7f40300ad8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403015b630 con 0x7f403015abb0 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.406+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f4024001670 con 0x7f403015abb0 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 >> v1:172.21.15.47:6789/0 conn(0x7f403015abb0 legacy=0x7f40300725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f402fe00640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4018003a00 0x7f4018003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f402fe00640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4018004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f402fe00640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40180051c0 0x7f401800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f401800c090 con 0x7f4018004800 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f401800c240 con 0x7f40180051c0 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f401800c440 con 0x7f4018003a00 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f4035000640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4018004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f4036400640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40180051c0 0x7f401800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.407+0000 7f4035a00640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4018003a00 0x7f4018003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.408+0000 7f4035000640 1 -- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4018003a00 msgr2=0x7f4018003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.408+0000 7f4035000640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4018003a00 0x7f4018003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.408+0000 7f4035000640 1 -- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40180051c0 msgr2=0x7f401800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.408+0000 7f4035000640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40180051c0 0x7f401800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.408+0000 7f4035000640 1 -- 172.21.15.47:0/586501185 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4018003520 con 0x7f4018004800 2024-04-20T10:29:41.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.408+0000 7f4036400640 1 -- 172.21.15.47:0/586501185 reap_dead start 2024-04-20T10:29:41.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.409+0000 7f4035000640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4018004c80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4020001330 tx=0x7f4018005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376e4d80778be4cd server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:41.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.409+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020001370 con 0x7f4018004800 2024-04-20T10:29:41.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.409+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40200052d0 con 0x7f4018004800 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.409+0000 7f402fe00640 1 -- 172.21.15.47:0/586501185 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4020003830 con 0x7f4018004800 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.410+0000 7f4038400640 1 -- 172.21.15.47:0/586501185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 msgr2=0x7f4018004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.410+0000 7f4038400640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4018004c80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4020001330 tx=0x7f4018005830 comp rx=0 tx=0).stop 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.410+0000 7f4038400640 1 -- 172.21.15.47:0/586501185 shutdown_connections 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.410+0000 7f4038400640 1 --2- 172.21.15.47:0/586501185 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4018004c80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.410+0000 7f4038400640 1 -- 172.21.15.47:0/586501185 >> 172.21.15.47:0/586501185 conn(0x7f40300b3350 msgr2=0x7f40300b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.410+0000 7f4038400640 1 -- 172.21.15.47:0/586501185 shutdown_connections 2024-04-20T10:29:41.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.410+0000 7f4038400640 1 -- 172.21.15.47:0/586501185 wait complete. 2024-04-20T10:29:41.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.412+0000 7f4038400640 1 Processor -- start 2024-04-20T10:29:41.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.412+0000 7f4038400640 1 -- start start 2024-04-20T10:29:41.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.413+0000 7f4038400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4030154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.413+0000 7f4038400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300af260 0x7f403015d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.413+0000 7f4038400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f403014bfe0 0x7f403014c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.413+0000 7f4038400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f403015bbf0 con 0x7f4018004800 2024-04-20T10:29:41.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.413+0000 7f4038400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f403015bf60 con 0x7f403014bfe0 2024-04-20T10:29:41.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.413+0000 7f4038400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f403015af90 con 0x7f40300af260 2024-04-20T10:29:41.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.413+0000 7f4035a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4030154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4030154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53354/0 (socket says 172.21.15.47:53354) 2024-04-20T10:29:41.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 -- 172.21.15.47:0/652063161 learned_addr learned my addr 172.21.15.47:0/652063161 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035000640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300af260 0x7f403015d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4036400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f403014bfe0 0x7f403014c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 -- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300af260 msgr2=0x7f403015d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300af260 0x7f403015d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 -- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f403014bfe0 msgr2=0x7f403014c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f403014bfe0 0x7f403014c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 -- 172.21.15.47:0/652063161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40301f82f0 con 0x7f4018004800 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4036400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f403014bfe0 0x7f403014c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.414+0000 7f4035a00640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4030154b70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f402400e730 tx=0x7f402400eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.415+0000 7f402d600640 1 -- 172.21.15.47:0/652063161 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402400c6c0 con 0x7f4018004800 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.415+0000 7f402d600640 1 -- 172.21.15.47:0/652063161 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4024018070 con 0x7f4018004800 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.415+0000 7f402d600640 1 -- 172.21.15.47:0/652063161 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402400cac0 con 0x7f4018004800 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.415+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40301f85d0 con 0x7f4018004800 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.415+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40301f8b90 con 0x7f4018004800 2024-04-20T10:29:41.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.416+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4030068980 con 0x7f4018004800 2024-04-20T10:29:41.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.417+0000 7f402d600640 1 -- 172.21.15.47:0/652063161 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f402400cc60 con 0x7f4018004800 2024-04-20T10:29:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.418+0000 7f402d600640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f401406c8f0 0x7f401406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.418+0000 7f402d600640 1 -- 172.21.15.47:0/652063161 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f402408bbb0 con 0x7f4018004800 2024-04-20T10:29:41.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.419+0000 7f4035000640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f401406c8f0 0x7f401406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.420+0000 7f4035000640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f401406c8f0 0x7f401406edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4020036000 tx=0x7f4020009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:41.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.423+0000 7f402d600640 1 -- 172.21.15.47:0/652063161 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4024059760 con 0x7f4018004800 2024-04-20T10:29:41.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.625+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f403006e390 con 0x7f4018004800 2024-04-20T10:29:41.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.626+0000 7f402d600640 1 -- 172.21.15.47:0/652063161 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f402405da60 con 0x7f4018004800 2024-04-20T10:29:41.627 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:29:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f401406c8f0 msgr2=0x7f401406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.629 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f401406c8f0 0x7f401406edb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4020036000 tx=0x7f4020009030 comp rx=0 tx=0).stop 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 msgr2=0x7f4030154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4030154b70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f402400e730 tx=0x7f402400eb30 comp rx=0 tx=0).stop 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 shutdown_connections 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f403014bfe0 0x7f403014c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40300af260 0x7f403015d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f401406c8f0 0x7f401406edb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 --2- 172.21.15.47:0/652063161 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4018004800 0x7f4030154b70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 >> 172.21.15.47:0/652063161 conn(0x7f40300b3350 msgr2=0x7f4030155a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 shutdown_connections 2024-04-20T10:29:41.630 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.629+0000 7f4038400640 1 -- 172.21.15.47:0/652063161 wait complete. 2024-04-20T10:29:41.648 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:29:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.858+0000 7f9ce8000640 1 Processor -- start 2024-04-20T10:29:41.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.858+0000 7f9ce8000640 1 -- start start 2024-04-20T10:29:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce8000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ce0156c20 0x7f9ce006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce00d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce8000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9ce0141c60 con 0x7f9ce015ca40 2024-04-20T10:29:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce8000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9ce00adab0 con 0x7f9ce0156c20 2024-04-20T10:29:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce015cf90 con 0x7f9ce00d0360 2024-04-20T10:29:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce5600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9ce0156c20 0x7f9ce006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce4c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f9ce015ca40 0x7f9ce00ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40232/0 (socket says 172.21.15.47:40232) 2024-04-20T10:29:41.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.860+0000 7f9ce4c00640 1 -- 172.21.15.47:0/1545415573 learned_addr learned my addr 172.21.15.47:0/1545415573 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.861+0000 7f9ce6000640 1 --2- 172.21.15.47:0/1545415573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce00d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.861+0000 7f9ce5600640 1 -- 172.21.15.47:0/1545415573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 msgr2=0x7f9ce00d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.861+0000 7f9ce5600640 1 --2- 172.21.15.47:0/1545415573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce00d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.861+0000 7f9ce5600640 1 -- 172.21.15.47:0/1545415573 >> v1:172.21.15.47:6789/0 conn(0x7f9ce015ca40 legacy=0x7f9ce00ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.861+0000 7f9ce5600640 1 -- 172.21.15.47:0/1545415573 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce015d270 con 0x7f9ce0156c20 2024-04-20T10:29:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.861+0000 7f9ce6000640 1 --2- 172.21.15.47:0/1545415573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce00d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:41.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.861+0000 7f9cdfe00640 1 -- 172.21.15.47:0/1545415573 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3174115377 0 0) 0x7f9ce0141c60 con 0x7f9ce015ca40 2024-04-20T10:29:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.862+0000 7f9ce5600640 1 --2- 172.21.15.47:0/1545415573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce006b900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40097a0 tx=0x7f9cd402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cf2eebc8e81448 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:41.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9cdfe00640 1 -- 172.21.15.47:0/1545415573 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4035020 con 0x7f9ce0156c20 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9cdfe00640 1 -- 172.21.15.47:0/1545415573 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd4031810 con 0x7f9ce0156c20 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9cdfe00640 1 -- 172.21.15.47:0/1545415573 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4031be0 con 0x7f9ce0156c20 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 -- 172.21.15.47:0/1545415573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 msgr2=0x7f9ce006b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1545415573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce006b900 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9cd40097a0 tx=0x7f9cd402b8b0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 -- 172.21.15.47:0/1545415573 shutdown_connections 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1545415573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce00d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1545415573 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce006b900 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 -- 172.21.15.47:0/1545415573 >> 172.21.15.47:0/1545415573 conn(0x7f9ce00b3350 msgr2=0x7f9ce00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 -- 172.21.15.47:0/1545415573 shutdown_connections 2024-04-20T10:29:41.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.863+0000 7f9ce8000640 1 -- 172.21.15.47:0/1545415573 wait complete. 2024-04-20T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.865+0000 7f9ce8000640 1 Processor -- start 2024-04-20T10:29:41.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.865+0000 7f9ce8000640 1 -- start start 2024-04-20T10:29:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.866+0000 7f9ce8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce01e77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.866+0000 7f9ce8000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce01e5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.866+0000 7f9ce8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce015ca40 0x7f9ce01e6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.866+0000 7f9ce8000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce006be40 con 0x7f9ce0156c20 2024-04-20T10:29:41.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.866+0000 7f9ce8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ce015d830 con 0x7f9ce015ca40 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.866+0000 7f9ce8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ce006bfc0 con 0x7f9ce00d0360 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce01e5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce01e5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53374/0 (socket says 172.21.15.47:53374) 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 -- 172.21.15.47:0/1444504103 learned_addr learned my addr 172.21.15.47:0/1444504103 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce5600640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce01e77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 -- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 msgr2=0x7f9ce01e77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce01e77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 -- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce015ca40 msgr2=0x7f9ce01e6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce015ca40 0x7f9ce01e6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 -- 172.21.15.47:0/1444504103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ce01521a0 con 0x7f9ce0156c20 2024-04-20T10:29:41.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.867+0000 7f9ce4c00640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce01e5e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000c4d0 tx=0x7f9cd000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.868+0000 7f9cdd600640 1 -- 172.21.15.47:0/1444504103 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0018070 con 0x7f9ce0156c20 2024-04-20T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.868+0000 7f9cdd600640 1 -- 172.21.15.47:0/1444504103 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9cd0007500 con 0x7f9ce0156c20 2024-04-20T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.868+0000 7f9cdd600640 1 -- 172.21.15.47:0/1444504103 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cd0004a20 con 0x7f9ce0156c20 2024-04-20T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.868+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ce0152480 con 0x7f9ce0156c20 2024-04-20T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.868+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ce0151e30 con 0x7f9ce0156c20 2024-04-20T10:29:41.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.869+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ce0068980 con 0x7f9ce0156c20 2024-04-20T10:29:41.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.870+0000 7f9cdd600640 1 -- 172.21.15.47:0/1444504103 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9cd0004bc0 con 0x7f9ce0156c20 2024-04-20T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.872+0000 7f9cdd600640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.872+0000 7f9cdd600640 1 -- 172.21.15.47:0/1444504103 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f9cd008af40 con 0x7f9ce0156c20 2024-04-20T10:29:41.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.872+0000 7f9ce5600640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:41.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.874+0000 7f9ce5600640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cc406c8f0 0x7f9cc406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4036000 tx=0x7f9cd4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:41.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:41.876+0000 7f9cdd600640 1 -- 172.21.15.47:0/1444504103 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9cd0059270 con 0x7f9ce0156c20 2024-04-20T10:29:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.087+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f9ce00b1810 con 0x7f9ce0156c20 2024-04-20T10:29:42.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.088+0000 7f9cdd600640 1 -- 172.21.15.47:0/1444504103 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7f9cd0058b70 con 0x7f9ce0156c20 2024-04-20T10:29:42.089 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:29:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cc406c8f0 msgr2=0x7f9cc406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cc406c8f0 0x7f9cc406edb0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4036000 tx=0x7f9cd4008030 comp rx=0 tx=0).stop 2024-04-20T10:29:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 msgr2=0x7f9ce01e5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce01e5e00 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cd000c4d0 tx=0x7f9cd000c990 comp rx=0 tx=0).stop 2024-04-20T10:29:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 shutdown_connections 2024-04-20T10:29:42.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ce015ca40 0x7f9ce01e6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9cc406c8f0 0x7f9cc406edb0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9ce0156c20 0x7f9ce01e5e00 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 --2- 172.21.15.47:0/1444504103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ce00d0360 0x7f9ce01e77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 >> 172.21.15.47:0/1444504103 conn(0x7f9ce00b3350 msgr2=0x7f9ce015ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 shutdown_connections 2024-04-20T10:29:42.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.091+0000 7f9ce8000640 1 -- 172.21.15.47:0/1444504103 wait complete. 2024-04-20T10:29:42.111 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:29:42.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.325+0000 7f53ccc00640 1 Processor -- start 2024-04-20T10:29:42.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.325+0000 7f53ccc00640 1 -- start start 2024-04-20T10:29:42.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53ccc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8163070 0x7f53c8071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53ccc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53c80735e0 0x7f53c8163450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53ccc00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f53c8073a40 0x7f53c8163990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53ccc00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f53c8163ed0 con 0x7f53c80735e0 2024-04-20T10:29:42.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53ccc00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f53c806c310 con 0x7f53c8073a40 2024-04-20T10:29:42.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53ccc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53c806c5f0 con 0x7f53c8163070 2024-04-20T10:29:42.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53c6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53c80735e0 0x7f53c8163450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53c6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f53c80735e0 0x7f53c8163450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53380/0 (socket says 172.21.15.47:53380) 2024-04-20T10:29:42.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.327+0000 7f53c6a00640 1 -- 172.21.15.47:0/4244222675 learned_addr learned my addr 172.21.15.47:0/4244222675 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c7400640 1 --2- 172.21.15.47:0/4244222675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8163070 0x7f53c8071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c6a00640 1 -- 172.21.15.47:0/4244222675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8163070 msgr2=0x7f53c8071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c7e00640 1 --2- 172.21.15.47:0/4244222675 >> v2:172.21.15.175:3300/0 conn(0x7f53c8073a40 0x7f53c8163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c6a00640 1 --2- 172.21.15.47:0/4244222675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8163070 0x7f53c8071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c6a00640 1 -- 172.21.15.47:0/4244222675 >> v2:172.21.15.175:3300/0 conn(0x7f53c8073a40 msgr2=0x7f53c8163990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c6a00640 1 --2- 172.21.15.47:0/4244222675 >> v2:172.21.15.175:3300/0 conn(0x7f53c8073a40 0x7f53c8163990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c6a00640 1 -- 172.21.15.47:0/4244222675 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c806c8d0 con 0x7f53c80735e0 2024-04-20T10:29:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c7400640 1 --2- 172.21.15.47:0/4244222675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8163070 0x7f53c8071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.328+0000 7f53c7e00640 1 --2- 172.21.15.47:0/4244222675 >> v2:172.21.15.175:3300/0 conn(0x7f53c8073a40 0x7f53c8163990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:42.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.329+0000 7f53c6a00640 1 --2- 172.21.15.47:0/4244222675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80735e0 0x7f53c8163450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f53c806ff70 tx=0x7f53b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a85728d9431b9397 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53c6000640 1 -- 172.21.15.47:0/4244222675 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b402e070 con 0x7f53c80735e0 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53c6000640 1 -- 172.21.15.47:0/4244222675 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53b4037070 con 0x7f53c80735e0 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53c6000640 1 -- 172.21.15.47:0/4244222675 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53b4031a00 con 0x7f53c80735e0 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53ccc00640 1 -- 172.21.15.47:0/4244222675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80735e0 msgr2=0x7f53c8163450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53ccc00640 1 --2- 172.21.15.47:0/4244222675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80735e0 0x7f53c8163450 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f53c806ff70 tx=0x7f53b402b880 comp rx=0 tx=0).stop 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53ccc00640 1 -- 172.21.15.47:0/4244222675 shutdown_connections 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53ccc00640 1 --2- 172.21.15.47:0/4244222675 >> v2:172.21.15.175:3300/0 conn(0x7f53c8073a40 0x7f53c8163990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53ccc00640 1 --2- 172.21.15.47:0/4244222675 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c80735e0 0x7f53c8163450 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53ccc00640 1 --2- 172.21.15.47:0/4244222675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8163070 0x7f53c8071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.330+0000 7f53ccc00640 1 -- 172.21.15.47:0/4244222675 >> 172.21.15.47:0/4244222675 conn(0x7f53c80d0040 msgr2=0x7f53c80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.331+0000 7f53ccc00640 1 -- 172.21.15.47:0/4244222675 shutdown_connections 2024-04-20T10:29:42.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.331+0000 7f53ccc00640 1 -- 172.21.15.47:0/4244222675 wait complete. 2024-04-20T10:29:42.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.332+0000 7f53ccc00640 1 Processor -- start 2024-04-20T10:29:42.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.332+0000 7f53ccc00640 1 -- start start 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.333+0000 7f53ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c80735e0 0x7f53c80b11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.333+0000 7f53ccc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8073a40 0x7f53c8151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.333+0000 7f53ccc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c8163070 0x7f53c8152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.333+0000 7f53ccc00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53c81662a0 con 0x7f53c8163070 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.333+0000 7f53ccc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53c806ce90 con 0x7f53c80735e0 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.333+0000 7f53ccc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53c81641c0 con 0x7f53c8073a40 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c8163070 0x7f53c8152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c8163070 0x7f53c8152430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53386/0 (socket says 172.21.15.47:53386) 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 -- 172.21.15.47:0/1548669950 learned_addr learned my addr 172.21.15.47:0/1548669950 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:42.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7400640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c80735e0 0x7f53c80b11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c6a00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8073a40 0x7f53c8151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 -- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8073a40 msgr2=0x7f53c8151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8073a40 0x7f53c8151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 -- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c80735e0 msgr2=0x7f53c80b11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c80735e0 0x7f53c80b11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c7e00640 1 -- 172.21.15.47:0/1548669950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53c81fc610 con 0x7f53c8163070 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.334+0000 7f53c6a00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8073a40 0x7f53c8151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.335+0000 7f53c7e00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c8163070 0x7f53c8152430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53c000cef0 tx=0x7f53c000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.335+0000 7f53bf400640 1 -- 172.21.15.47:0/1548669950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c001a070 con 0x7f53c8163070 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.335+0000 7f53bf400640 1 -- 172.21.15.47:0/1548669950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53c000d840 con 0x7f53c8163070 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.335+0000 7f53bf400640 1 -- 172.21.15.47:0/1548669950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c000dc30 con 0x7f53c8163070 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.335+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53c809b920 con 0x7f53c8163070 2024-04-20T10:29:42.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.335+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53c80b1720 con 0x7f53c8163070 2024-04-20T10:29:42.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.337+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53c8075b60 con 0x7f53c8163070 2024-04-20T10:29:42.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.337+0000 7f53bf400640 1 -- 172.21.15.47:0/1548669950 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53c001d070 con 0x7f53c8163070 2024-04-20T10:29:42.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.338+0000 7f53bf400640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53a806c8f0 0x7f53a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.338 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.338+0000 7f53bf400640 1 -- 172.21.15.47:0/1548669950 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f53c008c580 con 0x7f53c8163070 2024-04-20T10:29:42.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.339+0000 7f53c7400640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53a806c8f0 0x7f53a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.340+0000 7f53c7400640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53a806c8f0 0x7f53a806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53b800a950 tx=0x7f53b8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:42.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.343+0000 7f53bf400640 1 -- 172.21.15.47:0/1548669950 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f53c005a780 con 0x7f53c8163070 2024-04-20T10:29:42.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.561+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f53c815fb00 con 0x7f53c8163070 2024-04-20T10:29:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.562+0000 7f53bf400640 1 -- 172.21.15.47:0/1548669950 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f53c005a080 con 0x7f53c8163070 2024-04-20T10:29:42.562 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:29:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53a806c8f0 msgr2=0x7f53a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53a806c8f0 0x7f53a806edb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f53b800a950 tx=0x7f53b8008040 comp rx=0 tx=0).stop 2024-04-20T10:29:42.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c8163070 msgr2=0x7f53c8152430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c8163070 0x7f53c8152430 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f53c000cef0 tx=0x7f53c000a710 comp rx=0 tx=0).stop 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 shutdown_connections 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53a806c8f0 0x7f53a806edb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53c8163070 0x7f53c8152430 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53c8073a40 0x7f53c8151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 --2- 172.21.15.47:0/1548669950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53c80735e0 0x7f53c80b11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 >> 172.21.15.47:0/1548669950 conn(0x7f53c80d0040 msgr2=0x7f53c80cf190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 shutdown_connections 2024-04-20T10:29:42.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.565+0000 7f53ccc00640 1 -- 172.21.15.47:0/1548669950 wait complete. 2024-04-20T10:29:42.586 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:29:42.586 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:29:42.641 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:29:42.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.855+0000 7f6eaee00640 1 Processor -- start 2024-04-20T10:29:42.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.855+0000 7f6eaee00640 1 -- start start 2024-04-20T10:29:42.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.857+0000 7f6eaee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea8156c20 0x7f6ea806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.857+0000 7f6eaee00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ea815ca40 0x7f6ea80ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.857+0000 7f6eaee00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6ea8141c60 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.857+0000 7f6eaee00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6ea815cf90 con 0x7f6ea815ca40 2024-04-20T10:29:42.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.857+0000 7f6eaee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea815d270 con 0x7f6ea8156c20 2024-04-20T10:29:42.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6e9f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ea815ca40 0x7f6ea80ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6e9f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6ea815ca40 0x7f6ea80ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53388/0 (socket says 172.21.15.47:53388) 2024-04-20T10:29:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6e9f400640 1 -- 172.21.15.47:0/3460266380 learned_addr learned my addr 172.21.15.47:0/3460266380 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6ea7e00640 1 --2- 172.21.15.47:0/3460266380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea8156c20 0x7f6ea806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6e9f400640 1 -- 172.21.15.47:0/3460266380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea8156c20 msgr2=0x7f6ea806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6e9f400640 1 --2- 172.21.15.47:0/3460266380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea8156c20 0x7f6ea806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6e9f400640 1 -- 172.21.15.47:0/3460266380 >> v1:172.21.15.175:6789/0 conn(0x7f6ea80ad8e0 legacy=0x7f6ea80d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.858+0000 7f6e9f400640 1 -- 172.21.15.47:0/3460266380 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea815d550 con 0x7f6ea815ca40 2024-04-20T10:29:42.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.860+0000 7f6e9f400640 1 --2- 172.21.15.47:0/3460266380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea815ca40 0x7f6ea80ad3a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009870 tx=0x7f6e9002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24edc9ea5b44987 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:42.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.860+0000 7f6ea7400640 1 -- 172.21.15.47:0/3460266380 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e90035020 con 0x7f6ea815ca40 2024-04-20T10:29:42.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.860+0000 7f6ea7400640 1 -- 172.21.15.47:0/3460266380 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e90031640 con 0x7f6ea815ca40 2024-04-20T10:29:42.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.860+0000 7f6ea7400640 1 -- 172.21.15.47:0/3460266380 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e9002d040 con 0x7f6ea815ca40 2024-04-20T10:29:42.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 -- 172.21.15.47:0/3460266380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea815ca40 msgr2=0x7f6ea80ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 --2- 172.21.15.47:0/3460266380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea815ca40 0x7f6ea80ad3a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f6e90009870 tx=0x7f6e9002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 -- 172.21.15.47:0/3460266380 shutdown_connections 2024-04-20T10:29:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 --2- 172.21.15.47:0/3460266380 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea815ca40 0x7f6ea80ad3a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 --2- 172.21.15.47:0/3460266380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea8156c20 0x7f6ea806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 -- 172.21.15.47:0/3460266380 >> 172.21.15.47:0/3460266380 conn(0x7f6ea80b3350 msgr2=0x7f6ea80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 -- 172.21.15.47:0/3460266380 shutdown_connections 2024-04-20T10:29:42.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.861+0000 7f6eaee00640 1 -- 172.21.15.47:0/3460266380 wait complete. 2024-04-20T10:29:42.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.863+0000 7f6eaee00640 1 Processor -- start 2024-04-20T10:29:42.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.863+0000 7f6eaee00640 1 -- start start 2024-04-20T10:29:42.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6eaee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80ad8e0 0x7f6ea8099a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6eaee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea8156c20 0x7f6ea81e6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6eaee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea815ca40 0x7f6ea80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6eaee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea806bfc0 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6eaee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6ea806be40 con 0x7f6ea8156c20 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6eaee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6ea80d0a70 con 0x7f6ea815ca40 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80ad8e0 0x7f6ea8099a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6ea7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80ad8e0 0x7f6ea8099a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53394/0 (socket says 172.21.15.47:53394) 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6ea7e00640 1 -- 172.21.15.47:0/1923498325 learned_addr learned my addr 172.21.15.47:0/1923498325 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.864+0000 7f6e9f400640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea8156c20 0x7f6ea81e6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.865+0000 7f6eace00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea815ca40 0x7f6ea80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.865+0000 7f6ea7e00640 1 -- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea815ca40 msgr2=0x7f6ea80cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.865+0000 7f6ea7e00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea815ca40 0x7f6ea80cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.865+0000 7f6ea7e00640 1 -- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea8156c20 msgr2=0x7f6ea81e6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:42.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.865+0000 7f6ea7e00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea8156c20 0x7f6ea81e6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:42.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.865+0000 7f6ea7e00640 1 -- 172.21.15.47:0/1923498325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ea81e7100 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.865+0000 7f6ea7e00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80ad8e0 0x7f6ea8099a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6e98007f20 tx=0x7f6e9800e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:42.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.866+0000 7f6ea4c00640 1 -- 172.21.15.47:0/1923498325 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98014620 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.866+0000 7f6ea4c00640 1 -- 172.21.15.47:0/1923498325 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e98019070 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.866+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ea81e73e0 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.866+0000 7f6ea4c00640 1 -- 172.21.15.47:0/1923498325 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e98014a00 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.866+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ea81e79a0 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.867+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ea8068980 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.868+0000 7f6ea4c00640 1 -- 172.21.15.47:0/1923498325 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e98014ba0 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.869+0000 7f6ea4c00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e8406c8f0 0x7f6e8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:42.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.869+0000 7f6ea4c00640 1 -- 172.21.15.47:0/1923498325 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f6e9808c690 con 0x7f6ea80ad8e0 2024-04-20T10:29:42.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.870+0000 7f6e9f400640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e8406c8f0 0x7f6e8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:42.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.871+0000 7f6e9f400640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e8406c8f0 0x7f6e8406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6e90036000 tx=0x7f6e9003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:42.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:42.874+0000 7f6ea4c00640 1 -- 172.21.15.47:0/1923498325 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e9805a890 con 0x7f6ea80ad8e0 2024-04-20T10:29:43.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.082+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6ea815c050 con 0x7f6ea80ad8e0 2024-04-20T10:29:43.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.084+0000 7f6ea4c00640 1 -- 172.21.15.47:0/1923498325 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) ==== 74+0+16954 (secure 0 0 0) 0x7f6e9805a190 con 0x7f6ea80ad8e0 2024-04-20T10:29:43.085 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:43.085 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":42,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:26.904749+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:29:43.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e8406c8f0 msgr2=0x7f6e8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e8406c8f0 0x7f6e8406edb0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6e90036000 tx=0x7f6e9003c040 comp rx=0 tx=0).stop 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80ad8e0 msgr2=0x7f6ea8099a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80ad8e0 0x7f6ea8099a20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6e98007f20 tx=0x7f6e9800e570 comp rx=0 tx=0).stop 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 shutdown_connections 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6ea815ca40 0x7f6ea80cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6ea8156c20 0x7f6ea81e6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6e8406c8f0 0x7f6e8406edb0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 --2- 172.21.15.47:0/1923498325 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6ea80ad8e0 0x7f6ea8099a20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 >> 172.21.15.47:0/1923498325 conn(0x7f6ea80b3350 msgr2=0x7f6ea80afce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 shutdown_connections 2024-04-20T10:29:43.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.087+0000 7f6eaee00640 1 -- 172.21.15.47:0/1923498325 wait complete. 2024-04-20T10:29:43.109 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:29:23.363565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:29:24.330114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:29:43.109 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:29:43.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.321+0000 7fa896400640 1 Processor -- start 2024-04-20T10:29:43.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.321+0000 7fa896400640 1 -- start start 2024-04-20T10:29:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.323+0000 7fa896400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa890071a60 0x7fa890167390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.323+0000 7fa896400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89015cc20 con 0x7fa890073b80 2024-04-20T10:29:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.323+0000 7fa896400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa89015cec0 con 0x7fa89015a130 2024-04-20T10:29:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.323+0000 7fa896400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa89015d7f0 con 0x7fa890071a60 2024-04-20T10:29:43.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.323+0000 7fa88fe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa890073b80 0x7fa89006f160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:40236/0 (socket says 172.21.15.47:40236) 2024-04-20T10:29:43.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.323+0000 7fa88fe00640 1 -- 172.21.15.47:0/2797916478 learned_addr learned my addr 172.21.15.47:0/2797916478 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:43.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4177901736 0 0) 0x7fa89015cc20 con 0x7fa890073b80 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa878003520 con 0x7fa890073b80 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa895a00640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa890071a60 0x7fa890167390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2647486512 0 0) 0x7fa878003520 con 0x7fa890073b80 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa890071a60 msgr2=0x7fa890167390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa890071a60 0x7fa890167390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 >> v1:172.21.15.175:6789/0 conn(0x7fa89015a130 legacy=0x7fa890073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89015d1a0 con 0x7fa890073b80 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fa87c001670 con 0x7fa890073b80 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 >> v1:172.21.15.47:6789/0 conn(0x7fa890073b80 legacy=0x7fa89006f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa890071a60 0x7fa8780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878003fb0 0x7fa878004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa878004d40 0x7fa8780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa89015d7f0 con 0x7fa878004d40 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa87800bbb0 con 0x7fa878003fb0 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.324+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa87800bdb0 con 0x7fa890071a60 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.325+0000 7fa895a00640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa878004d40 0x7fa8780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.325+0000 7fa895a00640 1 -- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa890071a60 msgr2=0x7fa8780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.325+0000 7fa895a00640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa890071a60 0x7fa8780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.325+0000 7fa895a00640 1 -- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878003fb0 msgr2=0x7fa878004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.325+0000 7fa895a00640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878003fb0 0x7fa878004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.325+0000 7fa895a00640 1 -- 172.21.15.47:0/2797916478 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa878003520 con 0x7fa878004d40 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.326+0000 7fa895a00640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa878004d40 0x7fa8780051c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa888009fc0 tx=0x7fa878005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28e2d03be21f9295 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.326+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888039070 con 0x7fa878004d40 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa88802cc60 con 0x7fa878004d40 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa88f400640 1 -- 172.21.15.47:0/2797916478 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888009c10 con 0x7fa878004d40 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa896400640 1 -- 172.21.15.47:0/2797916478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa878004d40 msgr2=0x7fa8780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa896400640 1 --2- 172.21.15.47:0/2797916478 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa878004d40 0x7fa8780051c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa888009fc0 tx=0x7fa878005830 comp rx=0 tx=0).stop 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa895a00640 1 -- 172.21.15.47:0/2797916478 reap_dead start 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa896400640 1 -- 172.21.15.47:0/2797916478 shutdown_connections 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa896400640 1 -- 172.21.15.47:0/2797916478 >> 172.21.15.47:0/2797916478 conn(0x7fa8900cffa0 msgr2=0x7fa8900d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:43.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa896400640 1 -- 172.21.15.47:0/2797916478 shutdown_connections 2024-04-20T10:29:43.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.327+0000 7fa896400640 1 -- 172.21.15.47:0/2797916478 wait complete. 2024-04-20T10:29:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.329+0000 7fa896400640 1 Processor -- start 2024-04-20T10:29:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.330+0000 7fa896400640 1 -- start start 2024-04-20T10:29:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa896400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878004d40 0x7fa890152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa896400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa890153390 0x7fa89015a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa896400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8900c92f0 0x7fa8900c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa896400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa89006f9f0 con 0x7fa890153390 2024-04-20T10:29:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa896400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa89006f870 con 0x7fa878004d40 2024-04-20T10:29:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa896400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa89006fb70 con 0x7fa8900c92f0 2024-04-20T10:29:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa88fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa890153390 0x7fa89015a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa88fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa890153390 0x7fa89015a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53418/0 (socket says 172.21.15.47:53418) 2024-04-20T10:29:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa88fe00640 1 -- 172.21.15.47:0/3269166021 learned_addr learned my addr 172.21.15.47:0/3269166021 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa895000640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878004d40 0x7fa890152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.331+0000 7fa895a00640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8900c92f0 0x7fa8900c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88fe00640 1 -- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8900c92f0 msgr2=0x7fa8900c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88fe00640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8900c92f0 0x7fa8900c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88fe00640 1 -- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878004d40 msgr2=0x7fa890152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88fe00640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878004d40 0x7fa890152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88fe00640 1 -- 172.21.15.47:0/3269166021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa890156eb0 con 0x7fa890153390 2024-04-20T10:29:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88fe00640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa890153390 0x7fa89015a510 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00c7a0 tx=0x7fa87c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88cc00640 1 -- 172.21.15.47:0/3269166021 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c018070 con 0x7fa890153390 2024-04-20T10:29:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88cc00640 1 -- 172.21.15.47:0/3269166021 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa87c007500 con 0x7fa890153390 2024-04-20T10:29:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa890157150 con 0x7fa890153390 2024-04-20T10:29:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.332+0000 7fa88cc00640 1 -- 172.21.15.47:0/3269166021 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87c004a20 con 0x7fa890153390 2024-04-20T10:29:43.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.333+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa890157710 con 0x7fa890153390 2024-04-20T10:29:43.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.334+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa890075b30 con 0x7fa890153390 2024-04-20T10:29:43.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.335+0000 7fa88cc00640 1 -- 172.21.15.47:0/3269166021 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa87c01b070 con 0x7fa890153390 2024-04-20T10:29:43.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.336+0000 7fa88cc00640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa87006c8f0 0x7fa87006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.336+0000 7fa88cc00640 1 -- 172.21.15.47:0/3269166021 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7fa87c08b2b0 con 0x7fa890153390 2024-04-20T10:29:43.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.336+0000 7fa895000640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa87006c8f0 0x7fa87006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.338+0000 7fa895000640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa87006c8f0 0x7fa87006edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa8840017e0 tx=0x7fa884009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:43.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.340+0000 7fa88cc00640 1 -- 172.21.15.47:0/3269166021 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa87c058e60 con 0x7fa890153390 2024-04-20T10:29:43.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.546+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa89006f650 con 0x7fa890153390 2024-04-20T10:29:43.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.547+0000 7fa88cc00640 1 -- 172.21.15.47:0/3269166021 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) ==== 93+0+10 (secure 0 0 0) 0x7fa87c05d160 con 0x7fa890153390 2024-04-20T10:29:43.547 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:29:43.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.549+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa87006c8f0 msgr2=0x7fa87006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.549+0000 7fa896400640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa87006c8f0 0x7fa87006edb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa8840017e0 tx=0x7fa884009040 comp rx=0 tx=0).stop 2024-04-20T10:29:43.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa890153390 msgr2=0x7fa89015a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa890153390 0x7fa89015a510 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa87c00c7a0 tx=0x7fa87c00cc60 comp rx=0 tx=0).stop 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 shutdown_connections 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8900c92f0 0x7fa8900c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa87006c8f0 0x7fa87006edb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa890153390 0x7fa89015a510 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 --2- 172.21.15.47:0/3269166021 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa878004d40 0x7fa890152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 >> 172.21.15.47:0/3269166021 conn(0x7fa8900cffa0 msgr2=0x7fa8900d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 shutdown_connections 2024-04-20T10:29:43.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.550+0000 7fa896400640 1 -- 172.21.15.47:0/3269166021 wait complete. 2024-04-20T10:29:43.570 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:29:43.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.778+0000 7f8fc2400640 1 Processor -- start 2024-04-20T10:29:43.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.778+0000 7f8fc2400640 1 -- start start 2024-04-20T10:29:43.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.779+0000 7f8fc2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.779+0000 7f8fc2400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8fbc0d0a70 0x7f8fbc0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.779+0000 7f8fc2400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8fbc06f060 con 0x7f8fbc06dbb0 2024-04-20T10:29:43.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.779+0000 7f8fc2400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8fbc06e7c0 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.779+0000 7f8fc2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fbc06eaa0 con 0x7f8fbc15e8a0 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.780+0000 7f8fbbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8fbc0d0a70 0x7f8fbc0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.780+0000 7f8fbbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8fbc0d0a70 0x7f8fbc0ad680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53420/0 (socket says 172.21.15.47:53420) 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.780+0000 7f8fbbe00640 1 -- 172.21.15.47:0/2537645646 learned_addr learned my addr 172.21.15.47:0/2537645646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.780+0000 7f8fbb400640 1 --2- 172.21.15.47:0/2537645646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.781+0000 7f8fbbe00640 1 -- 172.21.15.47:0/2537645646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 msgr2=0x7f8fbc06e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.781+0000 7f8fbbe00640 1 --2- 172.21.15.47:0/2537645646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc06e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.781+0000 7f8fbbe00640 1 -- 172.21.15.47:0/2537645646 >> v1:172.21.15.175:6789/0 conn(0x7f8fbc06dbb0 legacy=0x7f8fbc0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.781+0000 7f8fbbe00640 1 -- 172.21.15.47:0/2537645646 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc06ed80 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.783+0000 7f8fbbe00640 1 --2- 172.21.15.47:0/2537645646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc0ad680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009870 tx=0x7f8fb402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e677181a0fc6040d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:43.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.783+0000 7f8fba000640 1 -- 172.21.15.47:0/2537645646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4034070 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.783+0000 7f8fba000640 1 -- 172.21.15.47:0/2537645646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fb4030840 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.783+0000 7f8fba000640 1 -- 172.21.15.47:0/2537645646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4030c10 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 -- 172.21.15.47:0/2537645646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 msgr2=0x7f8fbc0ad680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 --2- 172.21.15.47:0/2537645646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc0ad680 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009870 tx=0x7f8fb402b980 comp rx=0 tx=0).stop 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 -- 172.21.15.47:0/2537645646 shutdown_connections 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 --2- 172.21.15.47:0/2537645646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc0ad680 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 --2- 172.21.15.47:0/2537645646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc06e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 -- 172.21.15.47:0/2537645646 >> 172.21.15.47:0/2537645646 conn(0x7f8fbc0b3350 msgr2=0x7f8fbc0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 -- 172.21.15.47:0/2537645646 shutdown_connections 2024-04-20T10:29:43.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.784+0000 7f8fc2400640 1 -- 172.21.15.47:0/2537645646 wait complete. 2024-04-20T10:29:43.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.786+0000 7f8fc2400640 1 Processor -- start 2024-04-20T10:29:43.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.786+0000 7f8fc2400640 1 -- start start 2024-04-20T10:29:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fc2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fbc06dbb0 0x7f8fbc159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fc2400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc1f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fc2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fc2400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc06fef0 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fc2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc06fd70 con 0x7f8fbc06dbb0 2024-04-20T10:29:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fc2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8fbc0adbc0 con 0x7f8fbc15e8a0 2024-04-20T10:29:43.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc1f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc1f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53436/0 (socket says 172.21.15.47:53436) 2024-04-20T10:29:43.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.787+0000 7f8fbaa00640 1 -- 172.21.15.47:0/611828961 learned_addr learned my addr 172.21.15.47:0/611828961 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbb400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fbc06dbb0 0x7f8fbc159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbbe00640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbaa00640 1 -- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 msgr2=0x7f8fbc15d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbaa00640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbaa00640 1 -- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fbc06dbb0 msgr2=0x7f8fbc159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbaa00640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fbc06dbb0 0x7f8fbc159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbaa00640 1 -- 172.21.15.47:0/611828961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc159670 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fbaa00640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc1f4e00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400c790 tx=0x7f8fa400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:43.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fb3400640 1 -- 172.21.15.47:0/611828961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4016020 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fb3400640 1 -- 172.21.15.47:0/611828961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fa4006d50 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc14e280 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.788+0000 7f8fb3400640 1 -- 172.21.15.47:0/611828961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4003bb0 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.789+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc14e840 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.790+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc0ad680 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.791+0000 7f8fb3400640 1 -- 172.21.15.47:0/611828961 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fa400a850 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.792+0000 7f8fb3400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f9806c8f0 0x7f8f9806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:43.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.792+0000 7f8fb3400640 1 -- 172.21.15.47:0/611828961 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f8fa408ba10 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.793+0000 7f8fbb400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f9806c8f0 0x7f8f9806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:43.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.794+0000 7f8fbb400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f9806c8f0 0x7f8f9806edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b06f0 tx=0x7f8fac008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:43.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.796+0000 7f8fb3400640 1 -- 172.21.15.47:0/611828961 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8fbc0ad680 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.998+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8fbc068980 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:43.999+0000 7f8fb3400640 1 -- 172.21.15.47:0/611828961 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) ==== 104+0+11 (secure 0 0 0) 0x7f8fa4020020 con 0x7f8fbc0d0a70 2024-04-20T10:29:43.999 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:29:44.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f9806c8f0 msgr2=0x7f8f9806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f9806c8f0 0x7f8f9806edb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b06f0 tx=0x7f8fac008040 comp rx=0 tx=0).stop 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 msgr2=0x7f8fbc1f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc1f4e00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400c790 tx=0x7f8fa400cc50 comp rx=0 tx=0).stop 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 shutdown_connections 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8fbc15e8a0 0x7f8fbc15d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f9806c8f0 0x7f8f9806edb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8fbc0d0a70 0x7f8fbc1f4e00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 --2- 172.21.15.47:0/611828961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8fbc06dbb0 0x7f8fbc159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.001+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 >> 172.21.15.47:0/611828961 conn(0x7f8fbc0b3350 msgr2=0x7f8fbc0b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.002+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 shutdown_connections 2024-04-20T10:29:44.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.002+0000 7f8fc2400640 1 -- 172.21.15.47:0/611828961 wait complete. 2024-04-20T10:29:44.021 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:29:44.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.231+0000 7f7d64e00640 1 Processor -- start 2024-04-20T10:29:44.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.232+0000 7f7d64e00640 1 -- start start 2024-04-20T10:29:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.233+0000 7f7d64e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f7d60156c20 0x7f7d6006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.233+0000 7f7d64e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 0x7f7d600d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.233+0000 7f7d64e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7d60141c60 con 0x7f7d6015ca40 2024-04-20T10:29:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.233+0000 7f7d64e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f7d600adab0 con 0x7f7d60156c20 2024-04-20T10:29:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.233+0000 7f7d64e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6015cf90 con 0x7f7d600d0360 2024-04-20T10:29:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.233+0000 7f7d5d400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f7d6015ca40 0x7f7d600ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47918/0 (socket says 172.21.15.47:47918) 2024-04-20T10:29:44.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.233+0000 7f7d5d400640 1 -- 172.21.15.47:0/259817950 learned_addr learned my addr 172.21.15.47:0/259817950 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:44.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5de00640 1 --2- 172.21.15.47:0/259817950 >> v2:172.21.15.175:3300/0 conn(0x7f7d60156c20 0x7f7d6006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5e800640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 0x7f7d600d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2110668687 0 0) 0x7f7d60141c60 con 0x7f7d6015ca40 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7d44003520 con 0x7f7d6015ca40 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1141304900 0 0) 0x7f7d44003520 con 0x7f7d6015ca40 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 msgr2=0x7f7d600d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 0x7f7d600d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 >> v2:172.21.15.175:3300/0 conn(0x7f7d60156c20 msgr2=0x7f7d6006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 --2- 172.21.15.47:0/259817950 >> v2:172.21.15.175:3300/0 conn(0x7f7d60156c20 0x7f7d6006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6015d270 con 0x7f7d6015ca40 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f7d4c001670 con 0x7f7d6015ca40 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 >> v1:172.21.15.47:6789/0 conn(0x7f7d6015ca40 legacy=0x7f7d600ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d440039b0 0x7f7d44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 0x7f7d44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d44004d40 0x7f7d440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7d600adab0 con 0x7f7d44004d40 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6015cf90 con 0x7f7d440039b0 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d4400ba40 con 0x7f7d600d0360 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.234+0000 7f7d5de00640 1 --2- 172.21.15.47:0/259817950 >> v2:172.21.15.175:3300/0 conn(0x7f7d60156c20 0x7f7d6006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5e800640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d44004d40 0x7f7d440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5d400640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 0x7f7d44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5de00640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d440039b0 0x7f7d44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5e800640 1 -- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 msgr2=0x7f7d44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5e800640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d600d0360 0x7f7d44004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5e800640 1 -- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d440039b0 msgr2=0x7f7d44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5e800640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d440039b0 0x7f7d44003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.235+0000 7f7d5e800640 1 -- 172.21.15.47:0/259817950 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d44003520 con 0x7f7d44004d40 2024-04-20T10:29:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.236+0000 7f7d5e800640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d44004d40 0x7f7d440051c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009fc0 tx=0x7f7d44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e635514bd41cee6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58039070 con 0x7f7d44004d40 2024-04-20T10:29:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d5802cc20 con 0x7f7d44004d40 2024-04-20T10:29:44.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d5ca00640 1 -- 172.21.15.47:0/259817950 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d58009c10 con 0x7f7d44004d40 2024-04-20T10:29:44.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d64e00640 1 -- 172.21.15.47:0/259817950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d44004d40 msgr2=0x7f7d440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d64e00640 1 --2- 172.21.15.47:0/259817950 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d44004d40 0x7f7d440051c0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009fc0 tx=0x7f7d44005830 comp rx=0 tx=0).stop 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d5e800640 1 -- 172.21.15.47:0/259817950 reap_dead start 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d64e00640 1 -- 172.21.15.47:0/259817950 shutdown_connections 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d64e00640 1 -- 172.21.15.47:0/259817950 >> 172.21.15.47:0/259817950 conn(0x7f7d600b3350 msgr2=0x7f7d600b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d64e00640 1 -- 172.21.15.47:0/259817950 shutdown_connections 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.237+0000 7f7d64e00640 1 -- 172.21.15.47:0/259817950 wait complete. 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.239+0000 7f7d64e00640 1 Processor -- start 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.239+0000 7f7d64e00640 1 -- start start 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.240+0000 7f7d64e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d44004d40 0x7f7d6006e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.240+0000 7f7d64e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d600cec40 0x7f7d601f47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.240+0000 7f7d64e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d600731e0 0x7f7d60073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.240+0000 7f7d64e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6006be40 con 0x7f7d600cec40 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.240+0000 7f7d64e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d6015d830 con 0x7f7d600731e0 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.240+0000 7f7d64e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d6006bfc0 con 0x7f7d44004d40 2024-04-20T10:29:44.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d600cec40 0x7f7d601f47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d600cec40 0x7f7d601f47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60654/0 (socket says 172.21.15.47:60654) 2024-04-20T10:29:44.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 -- 172.21.15.47:0/2695069842 learned_addr learned my addr 172.21.15.47:0/2695069842 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5de00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d44004d40 0x7f7d6006e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5e800640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d600731e0 0x7f7d60073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 -- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d44004d40 msgr2=0x7f7d6006e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d44004d40 0x7f7d6006e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 -- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d600731e0 msgr2=0x7f7d60073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d600731e0 0x7f7d60073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 -- 172.21.15.47:0/2695069842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d601f4d00 con 0x7f7d600cec40 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5e800640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d600731e0 0x7f7d60073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.241+0000 7f7d5d400640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d600cec40 0x7f7d601f47c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e7f0 tx=0x7f7d4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.242+0000 7f7d56000640 1 -- 172.21.15.47:0/2695069842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00c870 con 0x7f7d600cec40 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.242+0000 7f7d56000640 1 -- 172.21.15.47:0/2695069842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7d4c018070 con 0x7f7d600cec40 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.242+0000 7f7d56000640 1 -- 172.21.15.47:0/2695069842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00cc50 con 0x7f7d600cec40 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.242+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6015b7b0 con 0x7f7d600cec40 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.242+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d6015bd70 con 0x7f7d600cec40 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.243+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d60068980 con 0x7f7d600cec40 2024-04-20T10:29:44.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.244+0000 7f7d56000640 1 -- 172.21.15.47:0/2695069842 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7d4c01d460 con 0x7f7d600cec40 2024-04-20T10:29:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.245+0000 7f7d56000640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d4006c920 0x7f7d4006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.245+0000 7f7d56000640 1 -- 172.21.15.47:0/2695069842 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f7d4c08bd20 con 0x7f7d600cec40 2024-04-20T10:29:44.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.246+0000 7f7d5de00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d4006c920 0x7f7d4006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.247+0000 7f7d5de00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d4006c920 0x7f7d4006ede0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006fd0 tx=0x7f7d50008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:44.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.251+0000 7f7d56000640 1 -- 172.21.15.47:0/2695069842 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f7d4c059f20 con 0x7f7d600cec40 2024-04-20T10:29:44.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.456+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f7d6015f090 con 0x7f7d600cec40 2024-04-20T10:29:44.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.456+0000 7f7d56000640 1 -- 172.21.15.47:0/2695069842 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) ==== 100+0+11 (secure 0 0 0) 0x7f7d4c059820 con 0x7f7d600cec40 2024-04-20T10:29:44.457 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d4006c920 msgr2=0x7f7d4006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d4006c920 0x7f7d4006ede0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7d50006fd0 tx=0x7f7d50008040 comp rx=0 tx=0).stop 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d600cec40 msgr2=0x7f7d601f47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d600cec40 0x7f7d601f47c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e7f0 tx=0x7f7d4c00ecb0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 shutdown_connections 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d600731e0 0x7f7d60073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f7d4006c920 0x7f7d4006ede0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f7d600cec40 0x7f7d601f47c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 --2- 172.21.15.47:0/2695069842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d44004d40 0x7f7d6006e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 >> 172.21.15.47:0/2695069842 conn(0x7f7d600b3350 msgr2=0x7f7d60154de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:44.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 shutdown_connections 2024-04-20T10:29:44.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.459+0000 7f7d64e00640 1 -- 172.21.15.47:0/2695069842 wait complete. 2024-04-20T10:29:44.478 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:29:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.686+0000 7f246d800640 1 Processor -- start 2024-04-20T10:29:44.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.687+0000 7f246d800640 1 -- start start 2024-04-20T10:29:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.688+0000 7f246d800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f246815a1f0 0x7f2468072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.688+0000 7f246d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2468072190 0x7f2468073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.688+0000 7f246d800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f246815cc90 con 0x7f2468159e10 2024-04-20T10:29:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.688+0000 7f246d800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2468072900 con 0x7f2468072190 2024-04-20T10:29:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.688+0000 7f246d800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f246815ca40 con 0x7f246815a1f0 2024-04-20T10:29:44.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f246ce00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2468072190 0x7f2468073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f246ce00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2468072190 0x7f2468073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60656/0 (socket says 172.21.15.47:60656) 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f246ce00640 1 -- 172.21.15.47:0/1238037353 learned_addr learned my addr 172.21.15.47:0/1238037353 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f245f400640 1 --2- 172.21.15.47:0/1238037353 >> v2:172.21.15.175:3300/0 conn(0x7f246815a1f0 0x7f2468072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f246ce00640 1 -- 172.21.15.47:0/1238037353 >> v2:172.21.15.175:3300/0 conn(0x7f246815a1f0 msgr2=0x7f2468072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f246ce00640 1 --2- 172.21.15.47:0/1238037353 >> v2:172.21.15.175:3300/0 conn(0x7f246815a1f0 0x7f2468072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f246ce00640 1 -- 172.21.15.47:0/1238037353 >> v1:172.21.15.175:6789/0 conn(0x7f2468159e10 legacy=0x7f2468071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.689+0000 7f246ce00640 1 -- 172.21.15.47:0/1238037353 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f246815cf70 con 0x7f2468072190 2024-04-20T10:29:44.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.691+0000 7f246ce00640 1 --2- 172.21.15.47:0/1238037353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2468072190 0x7f2468073420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2468050400 tx=0x7f246002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72d43db9cd1b908 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.691+0000 7f2467400640 1 -- 172.21.15.47:0/1238037353 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460035020 con 0x7f2468072190 2024-04-20T10:29:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.691+0000 7f2467400640 1 -- 172.21.15.47:0/1238037353 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2460031870 con 0x7f2468072190 2024-04-20T10:29:44.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.691+0000 7f2467400640 1 -- 172.21.15.47:0/1238037353 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460031c40 con 0x7f2468072190 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 -- 172.21.15.47:0/1238037353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2468072190 msgr2=0x7f2468073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 --2- 172.21.15.47:0/1238037353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2468072190 0x7f2468073420 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2468050400 tx=0x7f246002b910 comp rx=0 tx=0).stop 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 -- 172.21.15.47:0/1238037353 shutdown_connections 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 --2- 172.21.15.47:0/1238037353 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2468072190 0x7f2468073420 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 --2- 172.21.15.47:0/1238037353 >> v2:172.21.15.175:3300/0 conn(0x7f246815a1f0 0x7f2468072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 -- 172.21.15.47:0/1238037353 >> 172.21.15.47:0/1238037353 conn(0x7f24680d0040 msgr2=0x7f24680d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 -- 172.21.15.47:0/1238037353 shutdown_connections 2024-04-20T10:29:44.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.692+0000 7f246d800640 1 -- 172.21.15.47:0/1238037353 wait complete. 2024-04-20T10:29:44.693 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.694+0000 7f246d800640 1 Processor -- start 2024-04-20T10:29:44.694 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.694+0000 7f246d800640 1 -- start start 2024-04-20T10:29:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2468072190 0x7f2468156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2468159e10 0x7f24681f7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246d800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f246815a1f0 0x7f24680b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246d800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f246815d810 con 0x7f246815a1f0 2024-04-20T10:29:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f246815d530 con 0x7f2468159e10 2024-04-20T10:29:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f246806da90 con 0x7f2468072190 2024-04-20T10:29:44.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f246815a1f0 0x7f24680b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f246815a1f0 0x7f24680b0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60662/0 (socket says 172.21.15.47:60662) 2024-04-20T10:29:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f246ce00640 1 -- 172.21.15.47:0/2582570679 learned_addr learned my addr 172.21.15.47:0/2582570679 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:44.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.695+0000 7f2467e00640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2468072190 0x7f2468156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.696+0000 7f245f400640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2468159e10 0x7f24681f7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.696+0000 7f246ce00640 1 -- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2468072190 msgr2=0x7f2468156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.696+0000 7f246ce00640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2468072190 0x7f2468156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.696+0000 7f246ce00640 1 -- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2468159e10 msgr2=0x7f24681f7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.696+0000 7f246ce00640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2468159e10 0x7f24681f7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.696+0000 7f246ce00640 1 -- 172.21.15.47:0/2582570679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24681f84d0 con 0x7f246815a1f0 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.696+0000 7f246ce00640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f246815a1f0 0x7f24680b0ca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2460009960 tx=0x7f24600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.697+0000 7f2464c00640 1 -- 172.21.15.47:0/2582570679 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460040070 con 0x7f246815a1f0 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.697+0000 7f2464c00640 1 -- 172.21.15.47:0/2582570679 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2460030d00 con 0x7f246815a1f0 2024-04-20T10:29:44.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.697+0000 7f2464c00640 1 -- 172.21.15.47:0/2582570679 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2460035040 con 0x7f246815a1f0 2024-04-20T10:29:44.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.697+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24681f7f10 con 0x7f246815a1f0 2024-04-20T10:29:44.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.697+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f246815aef0 con 0x7f246815a1f0 2024-04-20T10:29:44.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.698+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2468075b60 con 0x7f246815a1f0 2024-04-20T10:29:44.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.699+0000 7f2464c00640 1 -- 172.21.15.47:0/2582570679 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2460038020 con 0x7f246815a1f0 2024-04-20T10:29:44.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.700+0000 7f2464c00640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f244406c8f0 0x7f244406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:44.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.700+0000 7f2464c00640 1 -- 172.21.15.47:0/2582570679 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f24600adae0 con 0x7f246815a1f0 2024-04-20T10:29:44.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.701+0000 7f2467e00640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f244406c8f0 0x7f244406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:44.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.702+0000 7f2467e00640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f244406c8f0 0x7f244406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f24680b1c40 tx=0x7f2458003d90 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:44.705 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.705+0000 7f2464c00640 1 -- 172.21.15.47:0/2582570679 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f246007b690 con 0x7f246815a1f0 2024-04-20T10:29:44.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.923+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f246806c060 con 0x7f246815a1f0 2024-04-20T10:29:44.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.924+0000 7f2464c00640 1 -- 172.21.15.47:0/2582570679 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v42) ==== 80+0+0 (secure 0 0 0) 0x7f246007f990 con 0x7f246815a1f0 2024-04-20T10:29:44.924 INFO:teuthology.orchestra.run.smithi047.stderr:listed 0 entries 2024-04-20T10:29:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f244406c8f0 msgr2=0x7f244406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f244406c8f0 0x7f244406edb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f24680b1c40 tx=0x7f2458003d90 comp rx=0 tx=0).stop 2024-04-20T10:29:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f246815a1f0 msgr2=0x7f24680b0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f246815a1f0 0x7f24680b0ca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2460009960 tx=0x7f24600046c0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 shutdown_connections 2024-04-20T10:29:44.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f244406c8f0 0x7f244406edb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f246815a1f0 0x7f24680b0ca0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2468159e10 0x7f24681f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 --2- 172.21.15.47:0/2582570679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2468072190 0x7f2468156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 >> 172.21.15.47:0/2582570679 conn(0x7f24680d0040 msgr2=0x7f246806cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 shutdown_connections 2024-04-20T10:29:44.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:44.927+0000 7f246d800640 1 -- 172.21.15.47:0/2582570679 wait complete. 2024-04-20T10:29:44.947 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-04-20T10:29:44.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:44.947 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:45.027 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[37648]: fuse finished with error 0 and tester_r 0 2024-04-20T10:29:45.034 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:29:51.136 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:29:51.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:51.136 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T10:29:51.198 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:51.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.415+0000 7efee3e00640 1 Processor -- start 2024-04-20T10:29:51.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.415+0000 7efee3e00640 1 -- start start 2024-04-20T10:29:51.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.416+0000 7efee3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efee415e550 0x7efee40d06b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:51.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.416+0000 7efee3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.416+0000 7efee3e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efee40cf950 con 0x7efee41621d0 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.416+0000 7efee3e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efee4162e90 con 0x7efee415e550 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.416+0000 7efee3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efee40cfbf0 con 0x7efee40d4380 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee2a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efee415e550 0x7efee40d06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee3400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7efee41621d0 0x7efee4162640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47928/0 (socket says 172.21.15.47:47928) 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee3400640 1 -- 172.21.15.47:0/3538274655 learned_addr learned my addr 172.21.15.47:0/3538274655 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee2000640 1 --2- 172.21.15.47:0/3538274655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee2a00640 1 -- 172.21.15.47:0/3538274655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 msgr2=0x7efee40d4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee2a00640 1 --2- 172.21.15.47:0/3538274655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40d4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee2a00640 1 -- 172.21.15.47:0/3538274655 >> v1:172.21.15.47:6789/0 conn(0x7efee41621d0 legacy=0x7efee4162640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee2a00640 1 -- 172.21.15.47:0/3538274655 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee40cfed0 con 0x7efee415e550 2024-04-20T10:29:51.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.417+0000 7efee1600640 1 -- 172.21.15.47:0/3538274655 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 16760205 0 0) 0x7efee40cf950 con 0x7efee41621d0 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.419+0000 7efee2a00640 1 --2- 172.21.15.47:0/3538274655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40d06b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efed40097a0 tx=0x7efed402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224c60d07f9f5440 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.419+0000 7efee1600640 1 -- 172.21.15.47:0/3538274655 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed40315a0 con 0x7efee415e550 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.419+0000 7efee1600640 1 -- 172.21.15.47:0/3538274655 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efed4037070 con 0x7efee415e550 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.419+0000 7efee1600640 1 -- 172.21.15.47:0/3538274655 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4031960 con 0x7efee415e550 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.419+0000 7efee3e00640 1 -- 172.21.15.47:0/3538274655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 msgr2=0x7efee40d06b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.419+0000 7efee3e00640 1 --2- 172.21.15.47:0/3538274655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40d06b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efed40097a0 tx=0x7efed402b840 comp rx=0 tx=0).stop 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.420+0000 7efee3e00640 1 -- 172.21.15.47:0/3538274655 shutdown_connections 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.420+0000 7efee3e00640 1 --2- 172.21.15.47:0/3538274655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40d4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.420+0000 7efee3e00640 1 --2- 172.21.15.47:0/3538274655 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40d06b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.420+0000 7efee3e00640 1 -- 172.21.15.47:0/3538274655 >> 172.21.15.47:0/3538274655 conn(0x7efee408bf80 msgr2=0x7efee409d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.420+0000 7efee3e00640 1 -- 172.21.15.47:0/3538274655 shutdown_connections 2024-04-20T10:29:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.420+0000 7efee3e00640 1 -- 172.21.15.47:0/3538274655 wait complete. 2024-04-20T10:29:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.421+0000 7efee3e00640 1 Processor -- start 2024-04-20T10:29:51.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.422+0000 7efee3e00640 1 -- start start 2024-04-20T10:29:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.422+0000 7efee3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.422+0000 7efee3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:51.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee41621d0 0x7efee41f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee3e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efee415eab0 con 0x7efee415e550 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efee415e930 con 0x7efee41621d0 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efee40d1fc0 con 0x7efee40d4380 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40b4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60684/0 (socket says 172.21.15.47:60684) 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 -- 172.21.15.47:0/139123559 learned_addr learned my addr 172.21.15.47:0/139123559 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2a00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 -- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 msgr2=0x7efee40b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee3400640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee41621d0 0x7efee41f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 -- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee41621d0 msgr2=0x7efee41f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee41621d0 0x7efee41f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.423+0000 7efee2000640 1 -- 172.21.15.47:0/139123559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee40b5330 con 0x7efee415e550 2024-04-20T10:29:51.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.424+0000 7efee2000640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40b4df0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efecc00d800 tx=0x7efecc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.424+0000 7efedaa00640 1 -- 172.21.15.47:0/139123559 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00b6c0 con 0x7efee415e550 2024-04-20T10:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.424+0000 7efedaa00640 1 -- 172.21.15.47:0/139123559 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efecc019070 con 0x7efee415e550 2024-04-20T10:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.424+0000 7efedaa00640 1 -- 172.21.15.47:0/139123559 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc00ba80 con 0x7efee415e550 2024-04-20T10:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.424+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee41f43d0 con 0x7efee415e550 2024-04-20T10:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.424+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee41f4670 con 0x7efee415e550 2024-04-20T10:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.426+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee4075b30 con 0x7efee415e550 2024-04-20T10:29:51.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.426+0000 7efedaa00640 1 -- 172.21.15.47:0/139123559 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efecc00bc20 con 0x7efee415e550 2024-04-20T10:29:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.427+0000 7efedaa00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efec006c8f0 0x7efec006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:51.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.427+0000 7efedaa00640 1 -- 172.21.15.47:0/139123559 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7efecc08c270 con 0x7efee415e550 2024-04-20T10:29:51.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.428+0000 7efee2a00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efec006c8f0 0x7efec006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:51.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.429+0000 7efee2a00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efec006c8f0 0x7efec006edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efed4036000 tx=0x7efed4008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:51.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.432+0000 7efedaa00640 1 -- 172.21.15.47:0/139123559 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efecc01c360 con 0x7efee415e550 2024-04-20T10:29:51.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.679+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efee40b9730 con 0x7efee415e550 2024-04-20T10:29:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.681+0000 7efedaa00640 1 -- 172.21.15.47:0/139123559 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+5205 (secure 0 0 0) 0x7efecc05a470 con 0x7efee415e550 2024-04-20T10:29:51.681 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:51.681 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5475,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:28.440368+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5481},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5481":{"gid":5481,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6835/1013167846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":1013167846},{"type":"v1","addr":"172.21.15.47:6835","nonce":1013167846}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5481,"qdb_cluster":[5481]},"id":2}]} 2024-04-20T10:29:51.681 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 19 2024-04-20T10:29:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.683+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efec006c8f0 msgr2=0x7efec006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.683+0000 7efee3e00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efec006c8f0 0x7efec006edb0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efed4036000 tx=0x7efed4008030 comp rx=0 tx=0).stop 2024-04-20T10:29:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.683+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 msgr2=0x7efee40b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.683+0000 7efee3e00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40b4df0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efecc00d800 tx=0x7efecc00dcc0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 shutdown_connections 2024-04-20T10:29:51.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee41621d0 0x7efee41f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efec006c8f0 0x7efec006edb0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efee415e550 0x7efee40b4df0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 --2- 172.21.15.47:0/139123559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee40d4380 0x7efee40b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 >> 172.21.15.47:0/139123559 conn(0x7efee408bf80 msgr2=0x7efee40b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 shutdown_connections 2024-04-20T10:29:51.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:51.684+0000 7efee3e00640 1 -- 172.21.15.47:0/139123559 wait complete. 2024-04-20T10:29:51.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-20T10:29:51.955 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-04-20T10:29:51.956 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T10:29:51.956 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail 2:0 2024-04-20T10:29:52.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.169+0000 7f36c7200640 1 Processor -- start 2024-04-20T10:29:52.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.169+0000 7f36c7200640 1 -- start start 2024-04-20T10:29:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.171+0000 7f36c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 0x7f36c80d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.171+0000 7f36c7200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f36c80b3330 0x7f36c80b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.171+0000 7f36c7200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f36c806c190 con 0x7f36c806c6d0 2024-04-20T10:29:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.171+0000 7f36c7200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f36c8162130 con 0x7f36c80b3330 2024-04-20T10:29:52.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.171+0000 7f36c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36c806c470 con 0x7f36c8161d50 2024-04-20T10:29:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.171+0000 7f36c6800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f36c806c6d0 0x7f36c80cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47936/0 (socket says 172.21.15.47:47936) 2024-04-20T10:29:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.171+0000 7f36c6800640 1 -- 172.21.15.47:0/3520524782 learned_addr learned my addr 172.21.15.47:0/3520524782 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5e00640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 0x7f36c80d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36bd400640 1 --2- 172.21.15.47:0/3520524782 >> v2:172.21.15.175:3300/0 conn(0x7f36c80b3330 0x7f36c80b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1000024872 0 0) 0x7f36c806c190 con 0x7f36c806c6d0 2024-04-20T10:29:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f36ac003520 con 0x7f36c806c6d0 2024-04-20T10:29:52.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1311212094 0 0) 0x7f36ac003520 con 0x7f36c806c6d0 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 msgr2=0x7f36c80d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 0x7f36c80d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 >> v2:172.21.15.175:3300/0 conn(0x7f36c80b3330 msgr2=0x7f36c80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 --2- 172.21.15.47:0/3520524782 >> v2:172.21.15.175:3300/0 conn(0x7f36c80b3330 0x7f36c80b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.172+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c80d2e00 con 0x7f36c806c6d0 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f36c00019f0 con 0x7f36c806c6d0 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 >> v1:172.21.15.47:6789/0 conn(0x7f36c806c6d0 legacy=0x7f36c80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 0x7f36ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36ac003f30 0x7f36ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ac004d40 0x7f36ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36c806c470 con 0x7f36ac004d40 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac00b9c0 con 0x7f36ac003f30 2024-04-20T10:29:52.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac00bb40 con 0x7f36c8161d50 2024-04-20T10:29:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c6800640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ac004d40 0x7f36ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36c5e00640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 0x7f36ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.173+0000 7f36bd400640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36ac003f30 0x7f36ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.174+0000 7f36c6800640 1 -- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 msgr2=0x7f36ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.174+0000 7f36c6800640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8161d50 0x7f36ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.174+0000 7f36c6800640 1 -- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36ac003f30 msgr2=0x7f36ac004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:52.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.174+0000 7f36c6800640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36ac003f30 0x7f36ac004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.174+0000 7f36c6800640 1 -- 172.21.15.47:0/3520524782 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac003520 con 0x7f36ac004d40 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.174+0000 7f36bd400640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36ac003f30 0x7f36ac004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.175+0000 7f36c6800640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ac004d40 0x7f36ac0051c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f36c000a870 tx=0x7f36ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70a7047c8794bd49 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c000a4c0 con 0x7f36ac004d40 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36c0034630 con 0x7f36ac004d40 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c5400640 1 -- 172.21.15.47:0/3520524782 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c0034a20 con 0x7f36ac004d40 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c7200640 1 -- 172.21.15.47:0/3520524782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ac004d40 msgr2=0x7f36ac0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c7200640 1 --2- 172.21.15.47:0/3520524782 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36ac004d40 0x7f36ac0051c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f36c000a870 tx=0x7f36ac005830 comp rx=0 tx=0).stop 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c6800640 1 -- 172.21.15.47:0/3520524782 reap_dead start 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c7200640 1 -- 172.21.15.47:0/3520524782 shutdown_connections 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c7200640 1 -- 172.21.15.47:0/3520524782 >> 172.21.15.47:0/3520524782 conn(0x7f36c80d4450 msgr2=0x7f36c80d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c7200640 1 -- 172.21.15.47:0/3520524782 shutdown_connections 2024-04-20T10:29:52.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.176+0000 7f36c7200640 1 -- 172.21.15.47:0/3520524782 wait complete. 2024-04-20T10:29:52.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.178+0000 7f36c7200640 1 Processor -- start 2024-04-20T10:29:52.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.178+0000 7f36c7200640 1 -- start start 2024-04-20T10:29:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.179+0000 7f36c7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36c8156bc0 0x7f36c81f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.179+0000 7f36c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c814bda0 0x7f36c80cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.179+0000 7f36c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8156fa0 0x7f36c80b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.179+0000 7f36c7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f36c80d38e0 con 0x7f36c8156bc0 2024-04-20T10:29:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.179+0000 7f36c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36c80d33c0 con 0x7f36c814bda0 2024-04-20T10:29:52.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.179+0000 7f36c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36c80d3a60 con 0x7f36c8156fa0 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.179+0000 7f36c5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36c8156bc0 0x7f36c81f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36c8156bc0 0x7f36c81f87c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60700/0 (socket says 172.21.15.47:60700) 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 -- 172.21.15.47:0/1904926594 learned_addr learned my addr 172.21.15.47:0/1904926594 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36bd400640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c814bda0 0x7f36c80cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c6800640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8156fa0 0x7f36c80b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 -- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8156fa0 msgr2=0x7f36c80b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8156fa0 0x7f36c80b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 -- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c814bda0 msgr2=0x7f36c80cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c814bda0 0x7f36c80cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 -- 172.21.15.47:0/1904926594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c80cbe90 con 0x7f36c8156bc0 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c6800640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8156fa0 0x7f36c80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:52.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.180+0000 7f36c5e00640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36c8156bc0 0x7f36c81f87c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f36b8011630 tx=0x7f36b8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:52.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.181+0000 7f36bea00640 1 -- 172.21.15.47:0/1904926594 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b800f920 con 0x7f36c8156bc0 2024-04-20T10:29:52.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.181+0000 7f36bea00640 1 -- 172.21.15.47:0/1904926594 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f36b800fac0 con 0x7f36c8156bc0 2024-04-20T10:29:52.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.181+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c80cc170 con 0x7f36c8156bc0 2024-04-20T10:29:52.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.181+0000 7f36bea00640 1 -- 172.21.15.47:0/1904926594 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b800fe70 con 0x7f36c8156bc0 2024-04-20T10:29:52.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.181+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c81f8d00 con 0x7f36c8156bc0 2024-04-20T10:29:52.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.182+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36c8075b60 con 0x7f36c8156bc0 2024-04-20T10:29:52.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.183+0000 7f36bea00640 1 -- 172.21.15.47:0/1904926594 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f36b8015460 con 0x7f36c8156bc0 2024-04-20T10:29:52.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.184+0000 7f36bea00640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f36a406c8f0 0x7f36a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.184+0000 7f36bea00640 1 -- 172.21.15.47:0/1904926594 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 6101+0+0 (secure 0 0 0) 0x7f36b808c700 con 0x7f36c8156bc0 2024-04-20T10:29:52.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.185+0000 7f36bd400640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f36a406c8f0 0x7f36a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.186+0000 7f36bd400640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f36a406c8f0 0x7f36a406edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cc820 tx=0x7f36b0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:52.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.189+0000 7f36bea00640 1 -- 172.21.15.47:0/1904926594 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f36b8019030 con 0x7f36c8156bc0 2024-04-20T10:29:52.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.473+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "2:0"} v 0) -- 0x7f36c815cbc0 con 0x7f36c8156bc0 2024-04-20T10:29:52.474 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-20T10:29:52.475+0000 7f06e1000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:29:52.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.479+0000 7f36bea00640 1 -- 172.21.15.47:0/1904926594 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "2:0"}]=0 failed mds gid 5481 v20) ==== 97+0+0 (secure 0 0 0) 0x7f36b808ae20 con 0x7f36c8156bc0 2024-04-20T10:29:52.479 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 5481 2024-04-20T10:29:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f36a406c8f0 msgr2=0x7f36a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f36a406c8f0 0x7f36a406edb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f36c80cc820 tx=0x7f36b0008040 comp rx=0 tx=0).stop 2024-04-20T10:29:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36c8156bc0 msgr2=0x7f36c81f87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36c8156bc0 0x7f36c81f87c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f36b8011630 tx=0x7f36b8011af0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 shutdown_connections 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36c8156fa0 0x7f36c80b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36c814bda0 0x7f36c80cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f36a406c8f0 0x7f36a406edb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 --2- 172.21.15.47:0/1904926594 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f36c8156bc0 0x7f36c81f87c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.482+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 >> 172.21.15.47:0/1904926594 conn(0x7f36c80d4450 msgr2=0x7f36c8160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.483+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 shutdown_connections 2024-04-20T10:29:52.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.483+0000 7f36c7200640 1 -- 172.21.15.47:0/1904926594 wait complete. 2024-04-20T10:29:52.495 INFO:tasks.ceph.mds.a.smithi047.stderr: -3346> 2024-04-20T10:29:31.764+0000 7f3677200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:29:52.504 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:52.556 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-20T10:29:52.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.677+0000 7f833ca00640 1 Processor -- start 2024-04-20T10:29:52.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.678+0000 7f833ca00640 1 -- start start 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.679+0000 7f833ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 0x7f833815f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.679+0000 7f833ca00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f833815f6d0 con 0x7f833815ebc0 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.679+0000 7f833ca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f833815f9b0 con 0x7f833815c260 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.679+0000 7f833ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f833815c640 con 0x7f833806fae0 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.679+0000 7f8336a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f833815ebc0 0x7f833806f3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:47940/0 (socket says 172.21.15.47:47940) 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.679+0000 7f8336a00640 1 -- 172.21.15.47:0/1635582856 learned_addr learned my addr 172.21.15.47:0/1635582856 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8337e00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 0x7f833815f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425496322 0 0) 0x7f833815f6d0 con 0x7f833815ebc0 2024-04-20T10:29:52.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8318003520 con 0x7f833815ebc0 2024-04-20T10:29:52.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4115448349 0 0) 0x7f8318003520 con 0x7f833815ebc0 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 msgr2=0x7f833815f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 0x7f833815f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 >> v1:172.21.15.175:6789/0 conn(0x7f833815c260 legacy=0x7f83380b4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83380b43f0 con 0x7f833815ebc0 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f8320001670 con 0x7f833815ebc0 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 >> v1:172.21.15.47:6789/0 conn(0x7f833815ebc0 legacy=0x7f833806f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 0x7f83180039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318003fb0 0x7f8318004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.680+0000 7f8336000640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8318004d40 0x7f83180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f833815c640 con 0x7f8318004d40 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f831800bb00 con 0x7f8318003fb0 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f831800bd00 con 0x7f833806fae0 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337e00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8318004d40 0x7f83180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337e00640 1 -- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 msgr2=0x7f83180039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337400640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 0x7f83180039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8336a00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318003fb0 0x7f8318004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337e00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 0x7f83180039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337e00640 1 -- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318003fb0 msgr2=0x7f8318004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337e00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318003fb0 0x7f8318004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337e00640 1 -- 172.21.15.47:0/1635582856 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8318003520 con 0x7f8318004d40 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8337400640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f833806fae0 0x7f83180039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.681+0000 7f8336a00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8318003fb0 0x7f8318004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.682+0000 7f8337e00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8318004d40 0x7f83180051c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8330009f90 tx=0x7f8318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c98d51f194d929 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330039070 con 0x7f8318004d40 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f833002cbd0 con 0x7f8318004d40 2024-04-20T10:29:52.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f8336000640 1 -- 172.21.15.47:0/1635582856 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330009c10 con 0x7f8318004d40 2024-04-20T10:29:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f833ca00640 1 -- 172.21.15.47:0/1635582856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8318004d40 msgr2=0x7f83180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f833ca00640 1 --2- 172.21.15.47:0/1635582856 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8318004d40 0x7f83180051c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8330009f90 tx=0x7f8318005830 comp rx=0 tx=0).stop 2024-04-20T10:29:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f8337e00640 1 -- 172.21.15.47:0/1635582856 reap_dead start 2024-04-20T10:29:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f833ca00640 1 -- 172.21.15.47:0/1635582856 shutdown_connections 2024-04-20T10:29:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f833ca00640 1 -- 172.21.15.47:0/1635582856 >> 172.21.15.47:0/1635582856 conn(0x7f83380b3300 msgr2=0x7f83380b3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:52.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f833ca00640 1 -- 172.21.15.47:0/1635582856 shutdown_connections 2024-04-20T10:29:52.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.683+0000 7f833ca00640 1 -- 172.21.15.47:0/1635582856 wait complete. 2024-04-20T10:29:52.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.685+0000 7f833ca00640 1 Processor -- start 2024-04-20T10:29:52.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.685+0000 7f833ca00640 1 -- start start 2024-04-20T10:29:52.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.686+0000 7f833ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8318004d40 0x7f83381429d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.686+0000 7f833ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8338161330 0x7f833815d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.686+0000 7f833ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83380cf5e0 0x7f83380cfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.686+0000 7f833ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f833815f040 con 0x7f8338161330 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.686+0000 7f833ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83380b4760 con 0x7f83380cf5e0 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.686+0000 7f833ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f833815f1c0 con 0x7f8318004d40 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8338161330 0x7f833815d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8338161330 0x7f833815d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60718/0 (socket says 172.21.15.47:60718) 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 -- 172.21.15.47:0/1469504407 learned_addr learned my addr 172.21.15.47:0/1469504407 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8337400640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8318004d40 0x7f83381429d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 -- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8318004d40 msgr2=0x7f83381429d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8318004d40 0x7f83381429d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8337e00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83380cf5e0 0x7f83380cfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 -- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83380cf5e0 msgr2=0x7f83380cfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83380cf5e0 0x7f83380cfa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 -- 172.21.15.47:0/1469504407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8338160290 con 0x7f8338161330 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f8336a00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8338161330 0x7f833815d110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f832000d8b0 tx=0x7f832000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.687+0000 7f832f400640 1 -- 172.21.15.47:0/1469504407 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8320019070 con 0x7f8338161330 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.688+0000 7f832f400640 1 -- 172.21.15.47:0/1469504407 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83200074e0 con 0x7f8338161330 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.688+0000 7f832f400640 1 -- 172.21.15.47:0/1469504407 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83200049e0 con 0x7f8338161330 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.688+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8338160570 con 0x7f8338161330 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.688+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8338160b30 con 0x7f8338161330 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.689+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8338075b30 con 0x7f8338161330 2024-04-20T10:29:52.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.690+0000 7f832f400640 1 -- 172.21.15.47:0/1469504407 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8320004b80 con 0x7f8338161330 2024-04-20T10:29:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.690+0000 7f832f400640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f831406c920 0x7f831406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:52.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.691+0000 7f832f400640 1 -- 172.21.15.47:0/1469504407 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f832008b840 con 0x7f8338161330 2024-04-20T10:29:52.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.691+0000 7f8337400640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f831406c920 0x7f831406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:52.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.692+0000 7f8337400640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f831406c920 0x7f831406ede0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8328000f60 tx=0x7f8328009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:52.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.695+0000 7f832f400640 1 -- 172.21.15.47:0/1469504407 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8320059a90 con 0x7f8338161330 2024-04-20T10:29:52.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.938+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8338158420 con 0x7f8338161330 2024-04-20T10:29:52.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.939+0000 7f832f400640 1 -- 172.21.15.47:0/1469504407 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 21 v21) ==== 94+0+4374 (secure 0 0 0) 0x7f8320059390 con 0x7f8338161330 2024-04-20T10:29:52.940 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:52.940 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":21,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:52.486441+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-20T10:29:52.940 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 21 2024-04-20T10:29:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f831406c920 msgr2=0x7f831406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f831406c920 0x7f831406ede0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8328000f60 tx=0x7f8328009040 comp rx=0 tx=0).stop 2024-04-20T10:29:52.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8338161330 msgr2=0x7f833815d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8338161330 0x7f833815d110 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f832000d8b0 tx=0x7f832000dd70 comp rx=0 tx=0).stop 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 shutdown_connections 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83380cf5e0 0x7f83380cfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f831406c920 0x7f831406ede0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8338161330 0x7f833815d110 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 --2- 172.21.15.47:0/1469504407 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8318004d40 0x7f83381429d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 >> 172.21.15.47:0/1469504407 conn(0x7f83380b3300 msgr2=0x7f83380b4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 shutdown_connections 2024-04-20T10:29:52.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:52.942+0000 7f833ca00640 1 -- 172.21.15.47:0/1469504407 wait complete. 2024-04-20T10:29:53.364 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 21, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:29:25.353181+0000', 'modified': '2024-04-20T10:29:52.486441+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5475': {'gid': 5475, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6833/2735464998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 2735464998}, {'type': 'v1', 'addr': '172.21.15.47:6833', 'nonce': 2735464998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T10:29:53.364 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5475:up:replay 2024-04-20T10:29:54.365 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:54.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.588+0000 7f821f200640 1 Processor -- start 2024-04-20T10:29:54.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.589+0000 7f821f200640 1 -- start start 2024-04-20T10:29:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.590+0000 7f821f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8218161d50 0x7f82180d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.590+0000 7f821f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f82180b3330 0x7f82180b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.590+0000 7f821f200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f821806c6d0 0x7f821806cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.590+0000 7f821f200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8218162130 con 0x7f821806c6d0 2024-04-20T10:29:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.590+0000 7f821f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f821806c190 con 0x7f82180b3330 2024-04-20T10:29:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.590+0000 7f821f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f821806c470 con 0x7f8218161d50 2024-04-20T10:29:54.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.590+0000 7f821e800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f821806c6d0 0x7f821806cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821e800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f821806c6d0 0x7f821806cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38000/0 (socket says 172.21.15.47:38000) 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821e800640 1 -- 172.21.15.47:0/390632227 learned_addr learned my addr 172.21.15.47:0/390632227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821de00640 1 --2- 172.21.15.47:0/390632227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8218161d50 0x7f82180d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821d400640 1 --2- 172.21.15.47:0/390632227 >> v2:172.21.15.175:3300/0 conn(0x7f82180b3330 0x7f82180b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821e800640 1 -- 172.21.15.47:0/390632227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8218161d50 msgr2=0x7f82180d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821e800640 1 --2- 172.21.15.47:0/390632227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8218161d50 0x7f82180d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821e800640 1 -- 172.21.15.47:0/390632227 >> v2:172.21.15.175:3300/0 conn(0x7f82180b3330 msgr2=0x7f82180b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821e800640 1 --2- 172.21.15.47:0/390632227 >> v2:172.21.15.175:3300/0 conn(0x7f82180b3330 0x7f82180b3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.591+0000 7f821e800640 1 -- 172.21.15.47:0/390632227 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82180d2e00 con 0x7f821806c6d0 2024-04-20T10:29:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.592+0000 7f821e800640 1 --2- 172.21.15.47:0/390632227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821806c6d0 0x7f821806cb60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f82180b7000 tx=0x7f821402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd58ffa3c68ea76a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.593+0000 7f821ca00640 1 -- 172.21.15.47:0/390632227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821402ca70 con 0x7f821806c6d0 2024-04-20T10:29:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.593+0000 7f821ca00640 1 -- 172.21.15.47:0/390632227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821402cc10 con 0x7f821806c6d0 2024-04-20T10:29:54.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.593+0000 7f821ca00640 1 -- 172.21.15.47:0/390632227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82140343f0 con 0x7f821806c6d0 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.593+0000 7f821f200640 1 -- 172.21.15.47:0/390632227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821806c6d0 msgr2=0x7f821806cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.593+0000 7f821f200640 1 --2- 172.21.15.47:0/390632227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821806c6d0 0x7f821806cb60 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f82180b7000 tx=0x7f821402bed0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.594+0000 7f821f200640 1 -- 172.21.15.47:0/390632227 shutdown_connections 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.594+0000 7f821f200640 1 --2- 172.21.15.47:0/390632227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f821806c6d0 0x7f821806cb60 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.594+0000 7f821f200640 1 --2- 172.21.15.47:0/390632227 >> v2:172.21.15.175:3300/0 conn(0x7f82180b3330 0x7f82180b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.594+0000 7f821f200640 1 --2- 172.21.15.47:0/390632227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8218161d50 0x7f82180d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.594+0000 7f821f200640 1 -- 172.21.15.47:0/390632227 >> 172.21.15.47:0/390632227 conn(0x7f82180d4450 msgr2=0x7f82180d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.594+0000 7f821f200640 1 -- 172.21.15.47:0/390632227 shutdown_connections 2024-04-20T10:29:54.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.594+0000 7f821f200640 1 -- 172.21.15.47:0/390632227 wait complete. 2024-04-20T10:29:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.596+0000 7f821f200640 1 Processor -- start 2024-04-20T10:29:54.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.596+0000 7f821f200640 1 -- start start 2024-04-20T10:29:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.597+0000 7f821f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f821806c6d0 0x7f82180ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.597+0000 7f821f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82180b3330 0x7f821814e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.597+0000 7f821f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8218161d50 0x7f821815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.597+0000 7f821f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f82180cfa00 con 0x7f8218161d50 2024-04-20T10:29:54.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82180d33c0 con 0x7f821806c6d0 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82180cfb80 con 0x7f82180b3330 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8218161d50 0x7f821815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8218161d50 0x7f821815d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38006/0 (socket says 172.21.15.47:38006) 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 -- 172.21.15.47:0/2547012548 learned_addr learned my addr 172.21.15.47:0/2547012548 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821d400640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82180b3330 0x7f821814e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821de00640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f821806c6d0 0x7f82180ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 -- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82180b3330 msgr2=0x7f821814e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82180b3330 0x7f821814e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 -- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f821806c6d0 msgr2=0x7f82180ce060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f821806c6d0 0x7f82180ce060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.598+0000 7f821e800640 1 -- 172.21.15.47:0/2547012548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f821814eca0 con 0x7f8218161d50 2024-04-20T10:29:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.599+0000 7f821e800640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8218161d50 0x7f821815d790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8214036040 tx=0x7f8214004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.599+0000 7f8212000640 1 -- 172.21.15.47:0/2547012548 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214043020 con 0x7f8218161d50 2024-04-20T10:29:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.599+0000 7f8212000640 1 -- 172.21.15.47:0/2547012548 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f821403e450 con 0x7f8218161d50 2024-04-20T10:29:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.599+0000 7f8212000640 1 -- 172.21.15.47:0/2547012548 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f821403e840 con 0x7f8218161d50 2024-04-20T10:29:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.599+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82181f86a0 con 0x7f8218161d50 2024-04-20T10:29:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.599+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82180cd860 con 0x7f8218161d50 2024-04-20T10:29:54.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.600+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8218075b60 con 0x7f8218161d50 2024-04-20T10:29:54.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.601+0000 7f8212000640 1 -- 172.21.15.47:0/2547012548 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8214046020 con 0x7f8218161d50 2024-04-20T10:29:54.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.603+0000 7f8212000640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f81fc06c920 0x7f81fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:54.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.603+0000 7f8212000640 1 -- 172.21.15.47:0/2547012548 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f82140ae530 con 0x7f8218161d50 2024-04-20T10:29:54.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.603+0000 7f821de00640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f81fc06c920 0x7f81fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:54.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.604+0000 7f821de00640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f81fc06c920 0x7f81fc06ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f820c006fd0 tx=0x7f820c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:54.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.608+0000 7f8212000640 1 -- 172.21.15.47:0/2547012548 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f821404d030 con 0x7f8218161d50 2024-04-20T10:29:54.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.853+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82180cb2e0 con 0x7f8218161d50 2024-04-20T10:29:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.855+0000 7f8212000640 1 -- 172.21.15.47:0/2547012548 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) ==== 94+0+5200 (secure 0 0 0) 0x7f821407c770 con 0x7f8218161d50 2024-04-20T10:29:54.855 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:54.855 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:53.501380+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:rejoin","state_seq":13,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":2}]} 2024-04-20T10:29:54.855 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 23 2024-04-20T10:29:54.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f81fc06c920 msgr2=0x7f81fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:54.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f81fc06c920 0x7f81fc06ede0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f820c006fd0 tx=0x7f820c008040 comp rx=0 tx=0).stop 2024-04-20T10:29:54.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8218161d50 msgr2=0x7f821815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:54.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8218161d50 0x7f821815d790 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f8214036040 tx=0x7f8214004a40 comp rx=0 tx=0).stop 2024-04-20T10:29:54.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 shutdown_connections 2024-04-20T10:29:54.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f81fc06c920 0x7f81fc06ede0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8218161d50 0x7f821815d790 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82180b3330 0x7f821814e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 --2- 172.21.15.47:0/2547012548 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f821806c6d0 0x7f82180ce060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 >> 172.21.15.47:0/2547012548 conn(0x7f82180d4450 msgr2=0x7f82180d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 shutdown_connections 2024-04-20T10:29:54.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:54.858+0000 7f821f200640 1 -- 172.21.15.47:0/2547012548 wait complete. 2024-04-20T10:29:55.279 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:29:25.353181+0000', 'modified': '2024-04-20T10:29:53.501380+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5475': {'gid': 5475, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:rejoin', 'state_seq': 13, 'addr': '172.21.15.47:6833/2735464998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 2735464998}, {'type': 'v1', 'addr': '172.21.15.47:6833', 'nonce': 2735464998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T10:29:55.279 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5475:up:rejoin 2024-04-20T10:29:56.280 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:29:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.504+0000 7f28d2600640 1 Processor -- start 2024-04-20T10:29:56.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.504+0000 7f28d2600640 1 -- start start 2024-04-20T10:29:56.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.506+0000 7f28d2600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f28cc1651d0 0x7f28cc071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:56.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.506+0000 7f28d2600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28cc163450 0x7f28cc1638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:56.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.506+0000 7f28d2600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28cc06c310 con 0x7f28cc163070 2024-04-20T10:29:56.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.506+0000 7f28d2600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc073cf0 con 0x7f28cc163450 2024-04-20T10:29:56.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.506+0000 7f28d2600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f28cc06c5f0 con 0x7f28cc1651d0 2024-04-20T10:29:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.506+0000 7f28d1c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28cc163450 0x7f28cc1638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.507+0000 7f28d1c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28cc163450 0x7f28cc1638e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38022/0 (socket says 172.21.15.47:38022) 2024-04-20T10:29:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.507+0000 7f28d1c00640 1 -- 172.21.15.47:0/3489016699 learned_addr learned my addr 172.21.15.47:0/3489016699 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.507+0000 7f28c3e00640 1 --2- 172.21.15.47:0/3489016699 >> v2:172.21.15.175:3300/0 conn(0x7f28cc1651d0 0x7f28cc071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.507+0000 7f28d1c00640 1 -- 172.21.15.47:0/3489016699 >> v2:172.21.15.175:3300/0 conn(0x7f28cc1651d0 msgr2=0x7f28cc071d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:56.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.507+0000 7f28d1c00640 1 --2- 172.21.15.47:0/3489016699 >> v2:172.21.15.175:3300/0 conn(0x7f28cc1651d0 0x7f28cc071d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.507+0000 7f28d1c00640 1 -- 172.21.15.47:0/3489016699 >> v1:172.21.15.175:6789/0 conn(0x7f28cc163070 legacy=0x7f28cc0735e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.507+0000 7f28d1c00640 1 -- 172.21.15.47:0/3489016699 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc06c8d0 con 0x7f28cc163450 2024-04-20T10:29:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.508+0000 7f28d1c00640 1 --2- 172.21.15.47:0/3489016699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc163450 0x7f28cc1638e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06ff70 tx=0x7f28c402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9245e3840be080 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.508+0000 7f28cbe00640 1 -- 172.21.15.47:0/3489016699 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c402c9f0 con 0x7f28cc163450 2024-04-20T10:29:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.508+0000 7f28cbe00640 1 -- 172.21.15.47:0/3489016699 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c402cb90 con 0x7f28cc163450 2024-04-20T10:29:56.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.508+0000 7f28cbe00640 1 -- 172.21.15.47:0/3489016699 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c40343f0 con 0x7f28cc163450 2024-04-20T10:29:56.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 -- 172.21.15.47:0/3489016699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc163450 msgr2=0x7f28cc1638e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:56.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 --2- 172.21.15.47:0/3489016699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc163450 0x7f28cc1638e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f28cc06ff70 tx=0x7f28c402be50 comp rx=0 tx=0).stop 2024-04-20T10:29:56.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 -- 172.21.15.47:0/3489016699 shutdown_connections 2024-04-20T10:29:56.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 --2- 172.21.15.47:0/3489016699 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc163450 0x7f28cc1638e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 --2- 172.21.15.47:0/3489016699 >> v2:172.21.15.175:3300/0 conn(0x7f28cc1651d0 0x7f28cc071d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 -- 172.21.15.47:0/3489016699 >> 172.21.15.47:0/3489016699 conn(0x7f28cc0d0040 msgr2=0x7f28cc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:56.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 -- 172.21.15.47:0/3489016699 shutdown_connections 2024-04-20T10:29:56.510 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.509+0000 7f28d2600640 1 -- 172.21.15.47:0/3489016699 wait complete. 2024-04-20T10:29:56.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.511+0000 7f28d2600640 1 Processor -- start 2024-04-20T10:29:56.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.511+0000 7f28d2600640 1 -- start start 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.512+0000 7f28d2600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc163070 0x7f28cc09bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.512+0000 7f28d2600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc163450 0x7f28cc09d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.512+0000 7f28d2600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc1651d0 0x7f28cc1594e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.512+0000 7f28d2600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc163e20 con 0x7f28cc1651d0 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.512+0000 7f28d2600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc06ce90 con 0x7f28cc163450 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.512+0000 7f28d2600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc1662a0 con 0x7f28cc163070 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc1651d0 0x7f28cc1594e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:56.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc1651d0 0x7f28cc1594e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38036/0 (socket says 172.21.15.47:38036) 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 -- 172.21.15.47:0/3131276698 learned_addr learned my addr 172.21.15.47:0/3131276698 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28c3e00640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc163450 0x7f28cc09d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1200640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc163070 0x7f28cc09bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 -- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc163070 msgr2=0x7f28cc09bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc163070 0x7f28cc09bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 -- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc163450 msgr2=0x7f28cc09d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc163450 0x7f28cc09d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 -- 172.21.15.47:0/3131276698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc09d910 con 0x7f28cc1651d0 2024-04-20T10:29:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.513+0000 7f28d1c00640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc1651d0 0x7f28cc1594e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f28c4036040 tx=0x7f28c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.514+0000 7f28c9600640 1 -- 172.21.15.47:0/3131276698 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4043020 con 0x7f28cc1651d0 2024-04-20T10:29:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.514+0000 7f28c9600640 1 -- 172.21.15.47:0/3131276698 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c403e450 con 0x7f28cc1651d0 2024-04-20T10:29:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.514+0000 7f28c9600640 1 -- 172.21.15.47:0/3131276698 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c403e860 con 0x7f28cc1651d0 2024-04-20T10:29:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.514+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc09c330 con 0x7f28cc1651d0 2024-04-20T10:29:56.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.514+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc09c8f0 con 0x7f28cc1651d0 2024-04-20T10:29:56.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.515+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc075b60 con 0x7f28cc1651d0 2024-04-20T10:29:56.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.516+0000 7f28c9600640 1 -- 172.21.15.47:0/3131276698 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28c4046020 con 0x7f28cc1651d0 2024-04-20T10:29:56.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.517+0000 7f28c9600640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a806c8f0 0x7f28a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:56.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.517+0000 7f28c9600640 1 -- 172.21.15.47:0/3131276698 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f28c40ae580 con 0x7f28cc1651d0 2024-04-20T10:29:56.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.518+0000 7f28d1200640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a806c8f0 0x7f28a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:56.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.519+0000 7f28d1200640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a806c8f0 0x7f28a806edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006fd0 tx=0x7f28bc008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:56.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.522+0000 7f28c9600640 1 -- 172.21.15.47:0/3131276698 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28c404d030 con 0x7f28cc1651d0 2024-04-20T10:29:56.737 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-04-20T10:29:56.736+0000 7f9ab9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:29:56.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.777+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28cc0cf230 con 0x7f28cc1651d0 2024-04-20T10:29:56.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.778+0000 7f28c9600640 1 -- 172.21.15.47:0/3131276698 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5207 (secure 0 0 0) 0x7f28c407c7d0 con 0x7f28cc1651d0 2024-04-20T10:29:56.779 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:29:56.779 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:55.499328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5475,"qdb_cluster":[5475]},"id":2}]} 2024-04-20T10:29:56.779 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-04-20T10:29:56.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.781+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a806c8f0 msgr2=0x7f28a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:56.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.781+0000 7f28d2600640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a806c8f0 0x7f28a806edb0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f28bc006fd0 tx=0x7f28bc008040 comp rx=0 tx=0).stop 2024-04-20T10:29:56.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.781+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc1651d0 msgr2=0x7f28cc1594e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:56.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.781+0000 7f28d2600640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc1651d0 0x7f28cc1594e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f28c4036040 tx=0x7f28c4004a40 comp rx=0 tx=0).stop 2024-04-20T10:29:56.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 shutdown_connections 2024-04-20T10:29:56.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a806c8f0 0x7f28a806edb0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28cc1651d0 0x7f28cc1594e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc163450 0x7f28cc09d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 --2- 172.21.15.47:0/3131276698 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc163070 0x7f28cc09bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 >> 172.21.15.47:0/3131276698 conn(0x7f28cc0d0040 msgr2=0x7f28cc06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 shutdown_connections 2024-04-20T10:29:56.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:56.782+0000 7f28d2600640 1 -- 172.21.15.47:0/3131276698 wait complete. 2024-04-20T10:29:57.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:29:25.353181+0000', 'modified': '2024-04-20T10:29:55.499328+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 43, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5475}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5475': {'gid': 5475, 'name': 'c', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.47:6833/2735464998', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 2735464998}, {'type': 'v1', 'addr': '172.21.15.47:6833', 'nonce': 2735464998}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5475, 'qdb_cluster': [5475]} 2024-04-20T10:29:57.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:29:57.203 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-20T10:29:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.424+0000 7fa3cd000640 1 Processor -- start 2024-04-20T10:29:57.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.424+0000 7fa3cd000640 1 -- start start 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3cd000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa3c815b7e0 0x7fa3c815bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3cd000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3c815c190 0x7fa3c806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3cd000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa3c806ccc0 con 0x7fa3c815b400 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3cd000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c806d280 con 0x7fa3c815c190 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3cd000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa3c806cfa0 con 0x7fa3c815b7e0 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3c7e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3c815c190 0x7fa3c806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3c7e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa3c815c190 0x7fa3c806ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38038/0 (socket says 172.21.15.47:38038) 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.426+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1995839997 learned_addr learned my addr 172.21.15.47:0/1995839997 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.427+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/1995839997 >> v2:172.21.15.175:3300/0 conn(0x7fa3c815b7e0 0x7fa3c815bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.427+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1995839997 >> v2:172.21.15.175:3300/0 conn(0x7fa3c815b7e0 msgr2=0x7fa3c815bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.427+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/1995839997 >> v2:172.21.15.175:3300/0 conn(0x7fa3c815b7e0 0x7fa3c815bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.427+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1995839997 >> v1:172.21.15.175:6789/0 conn(0x7fa3c815b400 legacy=0x7fa3c8072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.427+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1995839997 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c806e440 con 0x7fa3c815c190 2024-04-20T10:29:57.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.428+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/1995839997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815c190 0x7fa3c806ec40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8074a40 tx=0x7fa3c002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf2f9dd2a1b6082 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3c6000640 1 -- 172.21.15.47:0/1995839997 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c002ca80 con 0x7fa3c815c190 2024-04-20T10:29:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3c6000640 1 -- 172.21.15.47:0/1995839997 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c002cc20 con 0x7fa3c815c190 2024-04-20T10:29:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3c6000640 1 -- 172.21.15.47:0/1995839997 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c00343f0 con 0x7fa3c815c190 2024-04-20T10:29:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 -- 172.21.15.47:0/1995839997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815c190 msgr2=0x7fa3c806ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 --2- 172.21.15.47:0/1995839997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815c190 0x7fa3c806ec40 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8074a40 tx=0x7fa3c002bee0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 -- 172.21.15.47:0/1995839997 shutdown_connections 2024-04-20T10:29:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 --2- 172.21.15.47:0/1995839997 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815c190 0x7fa3c806ec40 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 --2- 172.21.15.47:0/1995839997 >> v2:172.21.15.175:3300/0 conn(0x7fa3c815b7e0 0x7fa3c815bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 -- 172.21.15.47:0/1995839997 >> 172.21.15.47:0/1995839997 conn(0x7fa3c80d0040 msgr2=0x7fa3c80d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 -- 172.21.15.47:0/1995839997 shutdown_connections 2024-04-20T10:29:57.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.429+0000 7fa3cd000640 1 -- 172.21.15.47:0/1995839997 wait complete. 2024-04-20T10:29:57.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.431+0000 7fa3cd000640 1 Processor -- start 2024-04-20T10:29:57.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.431+0000 7fa3cd000640 1 -- start start 2024-04-20T10:29:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3cd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3c815b400 0x7fa3c815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3cd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815b7e0 0x7fa3c81f43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3cd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3c815c190 0x7fa3c80ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3cd000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c815c770 con 0x7fa3c815b7e0 2024-04-20T10:29:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3cd000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c815c5f0 con 0x7fa3c815c190 2024-04-20T10:29:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3cd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c806f180 con 0x7fa3c815b400 2024-04-20T10:29:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815b7e0 0x7fa3c81f43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815b7e0 0x7fa3c81f43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38044/0 (socket says 172.21.15.47:38044) 2024-04-20T10:29:57.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.432+0000 7fa3c6a00640 1 -- 172.21.15.47:0/916687608 learned_addr learned my addr 172.21.15.47:0/916687608 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3c815c190 0x7fa3c80ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c7400640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3c815b400 0x7fa3c815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c6a00640 1 -- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3c815b400 msgr2=0x7fa3c815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3c815b400 0x7fa3c815e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c6a00640 1 -- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3c815c190 msgr2=0x7fa3c80ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3c815c190 0x7fa3c80ce050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c6a00640 1 -- 172.21.15.47:0/916687608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c80ce950 con 0x7fa3c815b7e0 2024-04-20T10:29:57.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3c815c190 0x7fa3c80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.433+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815b7e0 0x7fa3c81f43b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400d800 tx=0x7fa3b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.434+0000 7fa3bf400640 1 -- 172.21.15.47:0/916687608 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400b6c0 con 0x7fa3c815b7e0 2024-04-20T10:29:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.434+0000 7fa3bf400640 1 -- 172.21.15.47:0/916687608 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4019070 con 0x7fa3c815b7e0 2024-04-20T10:29:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.434+0000 7fa3bf400640 1 -- 172.21.15.47:0/916687608 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400ba80 con 0x7fa3c815b7e0 2024-04-20T10:29:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.434+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c80cec30 con 0x7fa3c815b7e0 2024-04-20T10:29:57.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.434+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c80cf1f0 con 0x7fa3c815b7e0 2024-04-20T10:29:57.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.436+0000 7fa3bf400640 1 -- 172.21.15.47:0/916687608 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa3b400bc20 con 0x7fa3c815b7e0 2024-04-20T10:29:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.437+0000 7fa3bf400640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa3a806c8f0 0x7fa3a806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.438+0000 7fa3bf400640 1 -- 172.21.15.47:0/916687608 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fa3b408c260 con 0x7fa3c815b7e0 2024-04-20T10:29:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.438+0000 7fa3c7400640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa3a806c8f0 0x7fa3a806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.438+0000 7fa3d1271740 1 Processor -- start 2024-04-20T10:29:57.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3d1271740 1 -- start start 2024-04-20T10:29:57.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 0x56101be7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101be7b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3c7400640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa3a806c8f0 0x7fa3a806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa3c81f7e70 tx=0x7fa3b8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be75b90 0x56101bea5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3d1271740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56101bde19d0 con 0x56101be75b90 2024-04-20T10:29:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56101bd88270 con 0x56101be796f0 2024-04-20T10:29:57.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.439+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56101bddd990 con 0x56101bec3670 2024-04-20T10:29:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be75b90 0x56101bea5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be75b90 0x56101bea5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38054/0 (socket says 172.21.15.47:38054) 2024-04-20T10:29:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1787127773 learned_addr learned my addr 172.21.15.47:0/1787127773 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101be7b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7400640 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 0x56101be7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 msgr2=0x56101be7aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 0x56101be7aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 msgr2=0x56101be7b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101be7b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.440+0000 7fa3c7e00640 1 -- 172.21.15.47:0/1787127773 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101baf42d0 con 0x56101be75b90 2024-04-20T10:29:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.441+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be75b90 0x56101bea5980 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0036040 tx=0x7fa3c002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0d76776984168e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.441+0000 7fa3b3e00640 1 -- 172.21.15.47:0/1787127773 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c0045070 con 0x56101be75b90 2024-04-20T10:29:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.441+0000 7fa3b3e00640 1 -- 172.21.15.47:0/1787127773 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c002ca80 con 0x56101be75b90 2024-04-20T10:29:57.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.441+0000 7fa3b3e00640 1 -- 172.21.15.47:0/1787127773 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3c002cc30 con 0x56101be75b90 2024-04-20T10:29:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 -- 172.21.15.47:0/1787127773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be75b90 msgr2=0x56101bea5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be75b90 0x56101bea5980 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fa3c0036040 tx=0x7fa3c002e000 comp rx=0 tx=0).stop 2024-04-20T10:29:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 -- 172.21.15.47:0/1787127773 shutdown_connections 2024-04-20T10:29:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be75b90 0x56101bea5980 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101be7b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 --2- 172.21.15.47:0/1787127773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 0x56101be7aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 -- 172.21.15.47:0/1787127773 >> 172.21.15.47:0/1787127773 conn(0x56101be77e00 msgr2=0x56101bec0980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 -- 172.21.15.47:0/1787127773 shutdown_connections 2024-04-20T10:29:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.442+0000 7fa3d1271740 1 -- 172.21.15.47:0/1787127773 wait complete. 2024-04-20T10:29:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.443+0000 7fa3d1271740 1 -- start start 2024-04-20T10:29:57.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.443+0000 7fa3d1271740 1 Processor -- start 2024-04-20T10:29:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be796f0 0x56101bec6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 0x56101bea3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be9cbb0 0x56101bec1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3d1271740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56101be79c00 con 0x56101be796f0 2024-04-20T10:29:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56101bd9b5e0 con 0x56101be9cbb0 2024-04-20T10:29:57.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56101be79ee0 con 0x56101bec3670 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3d1271740 10 client.0 _ll_register_callbacks cb 0x56101baf3ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3c7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be796f0 0x56101bec6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3c7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be796f0 0x56101bec6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38070/0 (socket says 172.21.15.47:38070) 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.444+0000 7fa3c7400640 1 -- 172.21.15.47:0/3075454174 learned_addr learned my addr 172.21.15.47:0/3075454174 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be9cbb0 0x56101bec1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7400640 1 -- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 msgr2=0x56101bea3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7400640 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 0x56101bea3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7400640 1 -- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be9cbb0 msgr2=0x56101bec1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7400640 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be9cbb0 0x56101bec1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7400640 1 -- 172.21.15.47:0/3075454174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101bbbe6b0 con 0x56101be796f0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be9cbb0 0x56101bec1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3c7400640 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be796f0 0x56101bec6180 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800dd30 tx=0x7fa3b80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8fb6a60801fe283 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3b0c00640 1 -- 172.21.15.47:0/3075454174 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8025060 con 0x56101be796f0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3b0c00640 1 -- 172.21.15.47:0/3075454174 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fa3b8005030 con 0x56101be796f0 2024-04-20T10:29:57.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.445+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3c002ca80 con 0x56101be796f0 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3b0c00640 1 -- 172.21.15.47:0/3075454174 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3b802b050 con 0x56101be796f0 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 10 client.5808 fetch_fsmap learned FSMap version 24 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56101be89130 con 0x56101be796f0 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3b0c00640 1 -- 172.21.15.47:0/3075454174 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3c002ca80 con 0x56101be796f0 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3b0c00640 1 -- 172.21.15.47:0/3075454174 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7fa3b802e020 con 0x56101be796f0 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 10 client.5808 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 10 client.5808 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 20 client.5808 populate_metadata read hostname 'smithi047' 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be8a0e0 0x56101bec5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 4 client.5808 mds_command: new command op to 5475 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:29:57.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.446+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 --> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56101bdef3f0 con 0x56101be8a0e0 2024-04-20T10:29:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.447+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be8a0e0 0x56101bec5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.447+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be8a0e0 0x56101bec5870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48331ea687776049 server_cookie=eb3bbf1d92151a74 in_seq=0 out_seq=0 2024-04-20T10:29:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.447+0000 7fa3b0c00640 10 client.5808 ms_handle_connect on v2:172.21.15.47:6832/2735464998 2024-04-20T10:29:57.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.448+0000 7fa3b0c00640 1 -- 172.21.15.47:0/3075454174 <== mds.0 v2:172.21.15.47:6832/2735464998 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56101bdef3f0 con 0x56101be8a0e0 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.448+0000 7fa3b0c00640 10 client.5808 handle_command_reply: tid=0 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.448+0000 7fa3d1271740 1 client.5808 shutdown 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.448+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be796f0 msgr2=0x56101bec6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.448+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be796f0 0x56101bec6180 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800dd30 tx=0x7fa3b80036f0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 shutdown_connections 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be8a0e0 0x56101bec5870 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be9cbb0 0x56101bec1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bec3670 0x56101bea3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3075454174 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101be796f0 0x56101bec6180 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3b0c00640 0 client.5808 ms_handle_reset on v2:172.21.15.47:6832/2735464998 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 >> 172.21.15.47:0/3075454174 conn(0x56101be75b90 msgr2=0x56101be76840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 shutdown_connections 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 1 -- 172.21.15.47:0/3075454174 wait complete. 2024-04-20T10:29:57.450 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.449+0000 7fa3d1271740 20 client.5808 trim_cache size 0 max 16384 2024-04-20T10:29:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.469+0000 7fa3d1271740 1 Processor -- start 2024-04-20T10:29:57.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.469+0000 7fa3d1271740 1 -- start start 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101be796f0 0x56101bea93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bea98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101bee5380 0x56101be74c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3d1271740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c8200160 con 0x56101bec3670 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c80cd480 con 0x56101bee5380 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c81f3320 con 0x56101be796f0 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bea98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bea98e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38080/0 (socket says 172.21.15.47:38080) 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.470+0000 7fa3c6a00640 1 -- 172.21.15.47:0/3237756483 learned_addr learned my addr 172.21.15.47:0/3237756483 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c7400640 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101be796f0 0x56101bea93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101bee5380 0x56101be74c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c6a00640 1 -- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101be796f0 msgr2=0x56101bea93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101be796f0 0x56101bea93a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c6a00640 1 -- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101bee5380 msgr2=0x56101be74c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101bee5380 0x56101be74c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c6a00640 1 -- 172.21.15.47:0/3237756483 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56101bea3bd0 con 0x56101bec3670 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101bee5380 0x56101be74c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bea98e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4059a10 tx=0x7fa3b408c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec8e514734f8ce8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.471+0000 7fa3b3400640 1 -- 172.21.15.47:0/3237756483 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4059d10 con 0x56101bec3670 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3b3400640 1 -- 172.21.15.47:0/3237756483 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4088070 con 0x56101bec3670 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3b3400640 1 -- 172.21.15.47:0/3237756483 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b405e0c0 con 0x56101bec3670 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 -- 172.21.15.47:0/3237756483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 msgr2=0x56101bea98e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bea98e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4059a10 tx=0x7fa3b408c040 comp rx=0 tx=0).stop 2024-04-20T10:29:57.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 -- 172.21.15.47:0/3237756483 shutdown_connections 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101bee5380 0x56101be74c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bea98e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 --2- 172.21.15.47:0/3237756483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101be796f0 0x56101bea93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 -- 172.21.15.47:0/3237756483 >> 172.21.15.47:0/3237756483 conn(0x56101be75b90 msgr2=0x56101be71280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 -- 172.21.15.47:0/3237756483 shutdown_connections 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.472+0000 7fa3d1271740 1 -- 172.21.15.47:0/3237756483 wait complete. 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.473+0000 7fa3d1271740 1 -- start start 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.473+0000 7fa3d1271740 1 Processor -- start 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.474+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101bee3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.474+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bec5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.474+0000 7fa3d1271740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bee5380 0x56101bec5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.474+0000 7fa3d1271740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x56101be9d310 con 0x56101bec3670 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.474+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56101be896f0 con 0x56101be796f0 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.474+0000 7fa3d1271740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56101bec52e0 con 0x56101bee5380 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.474+0000 7fa3d1271740 10 client.0 _ll_register_callbacks cb 0x7fa3b800d4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:29:57.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bec5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bec5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38094/0 (socket says 172.21.15.47:38094) 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 -- 172.21.15.47:0/2608363883 learned_addr learned my addr 172.21.15.47:0/2608363883 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c7e00640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bee5380 0x56101bec5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c7400640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101bee3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 -- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bee5380 msgr2=0x56101bec5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bee5380 0x56101bec5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 -- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 msgr2=0x56101bee3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101bee3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 -- 172.21.15.47:0/2608363883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56101bf3b5f0 con 0x56101bec3670 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c7400640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101bee3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3c6a00640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bec5a10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4056170 tx=0x7fa3b4020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e70b7545a658c64 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.475+0000 7fa3b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3b0c00640 1 -- 172.21.15.47:0/2608363883 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40628f0 con 0x56101bec3670 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3b0c00640 1 -- 172.21.15.47:0/2608363883 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fa3b408a590 con 0x56101bec3670 2024-04-20T10:29:57.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b4088070 con 0x56101bec3670 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3b0c00640 1 -- 172.21.15.47:0/2608363883 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fa3b405e0c0 con 0x56101bec3670 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 10 client.5814 fetch_fsmap learned FSMap version 24 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) -- 0x56101bf3bbb0 con 0x56101bec3670 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3b0c00640 1 -- 172.21.15.47:0/2608363883 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4088070 con 0x56101bec3670 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3b0c00640 1 -- 172.21.15.47:0/2608363883 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 24) ==== 3495+0+0 (secure 0 0 0) 0x7fa3b4013070 con 0x56101bec3670 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 10 client.5814 fetch_fsmap finished waiting for FSMap version 24 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 10 client.5814 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 20 client.5814 populate_metadata read hostname 'smithi047' 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be9dec0 0x56101bec3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 4 client.5814 mds_command: new command op to 5475 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.476+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 --> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] -- command(tid 0: {"prefix": "status"}) -- 0x56101bde9260 con 0x56101be9dec0 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.477+0000 7fa3c7400640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be9dec0 0x56101bec3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.477+0000 7fa3c7400640 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be9dec0 0x56101bec3a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a6100948356e79b server_cookie=7dd3e5154d7c0207 in_seq=0 out_seq=0 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.477+0000 7fa3b0c00640 10 client.5814 ms_handle_connect on v2:172.21.15.47:6832/2735464998 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3b0c00640 1 -- 172.21.15.47:0/2608363883 <== mds.0 v2:172.21.15.47:6832/2735464998 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x56101bde9260 con 0x56101be9dec0 2024-04-20T10:29:57.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3b0c00640 10 client.5814 handle_command_reply: tid=0 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 client.5814 shutdown 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 msgr2=0x56101bec5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bec5a10 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4056170 tx=0x7fa3b4020040 comp rx=0 tx=0).stop 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 shutdown_connections 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56101bee5380 0x56101bec5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x56101bec3670 0x56101bec5a10 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.47:6832/2735464998,v1:172.21.15.47:6833/2735464998] conn(0x56101be9dec0 0x56101bec3a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 --2- 172.21.15.47:0/2608363883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56101be796f0 0x56101bee3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 >> 172.21.15.47:0/2608363883 conn(0x56101be75b90 msgr2=0x56101be7ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3b0c00640 0 client.5814 ms_handle_reset on v2:172.21.15.47:6832/2735464998 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 shutdown_connections 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 1 -- 172.21.15.47:0/2608363883 wait complete. 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.478+0000 7fa3d1271740 20 client.5814 trim_cache size 0 max 16384 2024-04-20T10:29:57.484 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "775946e6-93c9-4843-aaf4-db758801eec1", 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5475, 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 4.9845949989999996, 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 24, 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 43, 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 43, 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 47.501904629999999 2024-04-20T10:29:57.485 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa3a806c8f0 msgr2=0x7fa3a806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa3a806c8f0 0x7fa3a806edb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa3c81f7e70 tx=0x7fa3b8009040 comp rx=0 tx=0).stop 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815b7e0 msgr2=0x7fa3c81f43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815b7e0 0x7fa3c81f43b0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400d800 tx=0x7fa3b400dcc0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 shutdown_connections 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3c815c190 0x7fa3c80ce050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa3a806c8f0 0x7fa3a806edb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa3c815b7e0 0x7fa3c81f43b0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 --2- 172.21.15.47:0/916687608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa3c815b400 0x7fa3c815e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 >> 172.21.15.47:0/916687608 conn(0x7fa3c80d0040 msgr2=0x7fa3c80722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 shutdown_connections 2024-04-20T10:29:57.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.479+0000 7fa3cd000640 1 -- 172.21.15.47:0/916687608 wait complete. 2024-04-20T10:29:57.497 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmxattr 10000000000.00000000 parent 2024-04-20T10:29:57.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.589+0000 7f3c2d386f40 1 Processor -- start 2024-04-20T10:29:57.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.590+0000 7f3c2d386f40 1 -- start start 2024-04-20T10:29:57.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.590+0000 7f3c2d386f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 0x561e42c80820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.590+0000 7f3c2d386f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c83ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.590+0000 7f3c2d386f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 0x561e42c843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.590+0000 7f3c2d386f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561e42b51cc0 con 0x561e42ba9610 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.590+0000 7f3c2d386f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561e42b55d00 con 0x561e42bdb1e0 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.590+0000 7f3c2d386f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561e42bf4940 con 0x561e42ba9230 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c83ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c83ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38114/0 (socket says 172.21.15.47:38114) 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 -- 172.21.15.47:0/396977743 learned_addr learned my addr 172.21.15.47:0/396977743 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 -- 172.21.15.47:0/396977743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 msgr2=0x561e42c80820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 --2- 172.21.15.47:0/396977743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 0x561e42c80820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 -- 172.21.15.47:0/396977743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 msgr2=0x561e42c843e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 --2- 172.21.15.47:0/396977743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 0x561e42c843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 -- 172.21.15.47:0/396977743 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e42c8c3c0 con 0x561e42ba9610 2024-04-20T10:29:57.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.591+0000 7f3c29c00640 1 --2- 172.21.15.47:0/396977743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c83ea0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3c180447f0 tx=0x7f3c1804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4421ab42ec5fd534 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c23e00640 1 -- 172.21.15.47:0/396977743 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18057040 con 0x561e42ba9610 2024-04-20T10:29:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c23e00640 1 -- 172.21.15.47:0/396977743 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c18062070 con 0x561e42ba9610 2024-04-20T10:29:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c23e00640 1 -- 172.21.15.47:0/396977743 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18052c50 con 0x561e42ba9610 2024-04-20T10:29:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 -- 172.21.15.47:0/396977743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 msgr2=0x561e42c83ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/396977743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c83ea0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3c180447f0 tx=0x7f3c1804c000 comp rx=0 tx=0).stop 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 -- 172.21.15.47:0/396977743 shutdown_connections 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/396977743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 0x561e42c843e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/396977743 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c83ea0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/396977743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 0x561e42c80820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 -- 172.21.15.47:0/396977743 >> 172.21.15.47:0/396977743 conn(0x561e42bd99a0 msgr2=0x561e42c621b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.594+0000 7f3c2d386f40 1 -- 172.21.15.47:0/396977743 shutdown_connections 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.595+0000 7f3c2d386f40 1 -- 172.21.15.47:0/396977743 wait complete. 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.595+0000 7f3c2d386f40 1 Processor -- start 2024-04-20T10:29:57.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.596+0000 7f3c2d386f40 1 -- start start 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.596+0000 7f3c2d386f40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 0x561e42c60fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.596+0000 7f3c2d386f40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c7fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.596+0000 7f3c2d386f40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 0x561e42c80a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.596+0000 7f3c2d386f40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x561e42c8c660 con 0x561e42ba9610 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.596+0000 7f3c2d386f40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x561e42c651e0 con 0x561e42bdb1e0 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.596+0000 7f3c2d386f40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x561e42c81990 con 0x561e42ba9230 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c7fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c7fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38118/0 (socket says 172.21.15.47:38118) 2024-04-20T10:29:57.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 -- 172.21.15.47:0/3491261860 learned_addr learned my addr 172.21.15.47:0/3491261860 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 -- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 msgr2=0x561e42c60fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c2a600640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 0x561e42c60fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c2b000640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 0x561e42c80a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x561e42ba9230 0x561e42c60fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 -- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 msgr2=0x561e42c80a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 0x561e42c80a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c29c00640 1 -- 172.21.15.47:0/3491261860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e42c64790 con 0x561e42ba9610 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.597+0000 7f3c2b000640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x561e42bdb1e0 0x561e42c80a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.598+0000 7f3c29c00640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c7fc80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3c1805f040 tx=0x7f3c18002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.598+0000 7f3c22a00640 1 -- 172.21.15.47:0/3491261860 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c18057040 con 0x561e42ba9610 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.598+0000 7f3c22a00640 1 -- 172.21.15.47:0/3491261860 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3c18052d00 con 0x561e42ba9610 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.598+0000 7f3c22a00640 1 -- 172.21.15.47:0/3491261860 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1805d6b0 con 0x561e42ba9610 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.598+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561e42bf3f90 con 0x561e42ba9610 2024-04-20T10:29:57.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.598+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e42c80220 con 0x561e42ba9610 2024-04-20T10:29:57.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.600+0000 7f3c22a00640 1 -- 172.21.15.47:0/3491261860 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3c1806f020 con 0x561e42ba9610 2024-04-20T10:29:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.601+0000 7f3c22a00640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3c0c06b2d0 0x7f3c0c06d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.601+0000 7f3c22a00640 1 -- 172.21.15.47:0/3491261860 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f3c180ce370 con 0x561e42ba9610 2024-04-20T10:29:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.602+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x561e42c80f50 0x561e42c60510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.602+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 --> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43) -- 0x561e42c68cf0 con 0x561e42c80f50 2024-04-20T10:29:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.602+0000 7f3c2b000640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x561e42c80f50 0x561e42c60510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.602+0000 7f3c2a600640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3c0c06b2d0 0x7f3c0c06d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.603+0000 7f3c2b000640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x561e42c80f50 0x561e42c60510 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.603+0000 7f3c2a600640 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3c0c06b2d0 0x7f3c0c06d790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c006fa0 tx=0x7f3c1c0064d0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.604+0000 7f3c2b000640 1 -- 172.21.15.47:0/3491261860 <== osd.2 v2:172.21.15.47:6822/2832687130 1 ==== osd_op_reply(1 10000000000.00000000 [rmxattr] v43'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f3c24001650 con 0x561e42c80f50 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.604+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x561e42c80f50 msgr2=0x561e42c60510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.604+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x561e42c80f50 0x561e42c60510 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3c0c06b2d0 msgr2=0x7f3c0c06d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3c0c06b2d0 0x7f3c0c06d790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c006fa0 tx=0x7f3c1c0064d0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 msgr2=0x561e42c7fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 --2- 172.21.15.47:0/3491261860 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x561e42ba9610 0x561e42c7fc80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f3c1805f040 tx=0x7f3c18002fb0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2b000640 1 -- 172.21.15.47:0/3491261860 reap_dead start 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 shutdown_connections 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 >> 172.21.15.47:0/3491261860 conn(0x561e42bd99a0 msgr2=0x561e42c84160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 shutdown_connections 2024-04-20T10:29:57.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.605+0000 7f3c2d386f40 1 -- 172.21.15.47:0/3491261860 wait complete. 2024-04-20T10:29:57.609 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:29:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.807+0000 7f52eca00640 1 Processor -- start 2024-04-20T10:29:57.808 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.808+0000 7f52eca00640 1 -- start start 2024-04-20T10:29:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.809+0000 7f52eca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 0x7f52e80d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.809 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.809+0000 7f52eca00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e806c190 con 0x7f52e806cde0 2024-04-20T10:29:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.810+0000 7f52eca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f52e806c470 con 0x7f52e80b3330 2024-04-20T10:29:57.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.810+0000 7f52eca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52e8162130 con 0x7f52e8161d50 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.810+0000 7f52e7e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f52e806cde0 0x7f52e80b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43848/0 (socket says 172.21.15.47:43848) 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.810+0000 7f52e7e00640 1 -- 172.21.15.47:0/1850019167 learned_addr learned my addr 172.21.15.47:0/1850019167 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.810+0000 7f52e7400640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 0x7f52e80d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.810+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 388378070 0 0) 0x7f52e806c190 con 0x7f52e806cde0 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.810+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f52cc003520 con 0x7f52e806cde0 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3249854516 0 0) 0x7f52cc003520 con 0x7f52e806cde0 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 msgr2=0x7f52e80d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 0x7f52e80d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 >> v1:172.21.15.175:6789/0 conn(0x7f52e80b3330 legacy=0x7f52e806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e80d2e00 con 0x7f52e806cde0 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f52e0001a10 con 0x7f52e806cde0 2024-04-20T10:29:57.812 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 >> v1:172.21.15.47:6789/0 conn(0x7f52e806cde0 legacy=0x7f52e80b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52cc0039e0 0x7f52cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 0x7f52cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc00ba70 con 0x7f52cc004d40 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc00bbf0 con 0x7f52cc0039e0 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc00bdf0 con 0x7f52e8161d50 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.811+0000 7f52e7e00640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.812+0000 7f52e7400640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52cc0039e0 0x7f52cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.812+0000 7f52e6a00640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 0x7f52cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.812+0000 7f52e7e00640 1 -- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 msgr2=0x7f52cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.812+0000 7f52e7e00640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e8161d50 0x7f52cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.812+0000 7f52e7e00640 1 -- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52cc0039e0 msgr2=0x7f52cc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.812+0000 7f52e7e00640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52cc0039e0 0x7f52cc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.813 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.812+0000 7f52e7e00640 1 -- 172.21.15.47:0/1850019167 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc003520 con 0x7f52cc004d40 2024-04-20T10:29:57.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.813+0000 7f52e7e00640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52cc0051c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f52e80d5550 tx=0x7f52cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec15738fb8599a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.813+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e000a480 con 0x7f52cc004d40 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.813+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52e0034650 con 0x7f52cc004d40 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.813+0000 7f52e6000640 1 -- 172.21.15.47:0/1850019167 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e002ce60 con 0x7f52cc004d40 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.814+0000 7f52eca00640 1 -- 172.21.15.47:0/1850019167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 msgr2=0x7f52cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.814+0000 7f52eca00640 1 --2- 172.21.15.47:0/1850019167 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52cc0051c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f52e80d5550 tx=0x7f52cc005830 comp rx=0 tx=0).stop 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.814+0000 7f52e7e00640 1 -- 172.21.15.47:0/1850019167 reap_dead start 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.814+0000 7f52eca00640 1 -- 172.21.15.47:0/1850019167 shutdown_connections 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.814+0000 7f52eca00640 1 -- 172.21.15.47:0/1850019167 >> 172.21.15.47:0/1850019167 conn(0x7f52e80d4450 msgr2=0x7f52e80d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.814+0000 7f52eca00640 1 -- 172.21.15.47:0/1850019167 shutdown_connections 2024-04-20T10:29:57.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.814+0000 7f52eca00640 1 -- 172.21.15.47:0/1850019167 wait complete. 2024-04-20T10:29:57.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.816+0000 7f52eca00640 1 Processor -- start 2024-04-20T10:29:57.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.816+0000 7f52eca00640 1 -- start start 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52eca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52e8157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52eca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e814a720 0x7f52e80c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52eca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e8159ea0 0x7f52e815a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52eca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80d38e0 con 0x7f52cc004d40 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52eca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52e80d33c0 con 0x7f52e8159ea0 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52eca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52e80d3a60 con 0x7f52e814a720 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52e7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52e8157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52e7400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52e8157350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38142/0 (socket says 172.21.15.47:38142) 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.817+0000 7f52e7400640 1 -- 172.21.15.47:0/2559733428 learned_addr learned my addr 172.21.15.47:0/2559733428 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52e7400640 1 -- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e814a720 msgr2=0x7f52e80c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52e7400640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e814a720 0x7f52e80c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52e7400640 1 -- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e8159ea0 msgr2=0x7f52e815a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:57.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52e7400640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e8159ea0 0x7f52e815a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52e7400640 1 -- 172.21.15.47:0/2559733428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e81590e0 con 0x7f52cc004d40 2024-04-20T10:29:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52e7400640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52e8157350 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f52d8011630 tx=0x7f52d8011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52df400640 1 -- 172.21.15.47:0/2559733428 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8015870 con 0x7f52cc004d40 2024-04-20T10:29:57.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52df400640 1 -- 172.21.15.47:0/2559733428 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52d8015a10 con 0x7f52cc004d40 2024-04-20T10:29:57.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e81593c0 con 0x7f52cc004d40 2024-04-20T10:29:57.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52df400640 1 -- 172.21.15.47:0/2559733428 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d8015dc0 con 0x7f52cc004d40 2024-04-20T10:29:57.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.818+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e8158dd0 con 0x7f52cc004d40 2024-04-20T10:29:57.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.820+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e8075b60 con 0x7f52cc004d40 2024-04-20T10:29:57.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.821+0000 7f52df400640 1 -- 172.21.15.47:0/2559733428 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f52d8013430 con 0x7f52cc004d40 2024-04-20T10:29:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.821+0000 7f52df400640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f52c406c920 0x7f52c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:29:57.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.822+0000 7f52df400640 1 -- 172.21.15.47:0/2559733428 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f52d808c6a0 con 0x7f52cc004d40 2024-04-20T10:29:57.822 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.822+0000 7f52e6a00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f52c406c920 0x7f52c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:29:57.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.823+0000 7f52e6a00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f52c406c920 0x7f52c406ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52d0007fd0 tx=0x7f52d0009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:29:57.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:57.826+0000 7f52df400640 1 -- 172.21.15.47:0/2559733428 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f52d8019030 con 0x7f52cc004d40 2024-04-20T10:29:58.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.078+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52e80b9d40 con 0x7f52cc004d40 2024-04-20T10:29:58.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.079+0000 7f52df400640 1 -- 172.21.15.47:0/2559733428 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f52d805a7c0 con 0x7f52cc004d40 2024-04-20T10:29:58.079 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T10:29:58.081 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.081+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f52c406c920 msgr2=0x7f52c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.081+0000 7f52eca00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f52c406c920 0x7f52c406ede0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f52d0007fd0 tx=0x7f52d0009040 comp rx=0 tx=0).stop 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.081+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 msgr2=0x7f52e8157350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.081+0000 7f52eca00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52e8157350 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f52d8011630 tx=0x7f52d8011af0 comp rx=0 tx=0).stop 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 shutdown_connections 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e8159ea0 0x7f52e815a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e814a720 0x7f52e80c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f52c406c920 0x7f52c406ede0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 --2- 172.21.15.47:0/2559733428 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f52cc004d40 0x7f52e8157350 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 >> 172.21.15.47:0/2559733428 conn(0x7f52e80d4450 msgr2=0x7f52e8160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:29:58.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 shutdown_connections 2024-04-20T10:29:58.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58.082+0000 7f52eca00640 1 -- 172.21.15.47:0/2559733428 wait complete. 2024-04-20T10:29:58.099 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:29:58.099 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:29:58.099 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-20T10:29:58.099 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:29:58.099 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:58.099 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:29:58.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:29:58.100 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:29:58.100 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:29:58.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:58.100 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:58.152 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42684sec preferred_lft 42684sec 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:29:58.153 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::50dc:baff:fe1b:7c56/64 scope link 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::8014:63ff:fed5:b599/64 scope link 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether aa:c8:12:38:db:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::a8c8:12ff:fe38:db8e/64 scope link 2024-04-20T10:29:58.154 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:29:58.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:58.155 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:29:58.233 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-20T10:29:58.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:58.237 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-20T10:29:58.289 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-20T10:29:58.289 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:29:58.289 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:29:58.289 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:29:58.289 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:29:58.290 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:29:58.290 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:29:58.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:29:58.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:29:58.290 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:29:58.291 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:29:58.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:29:58.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:29:58.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:58.512 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:29:58.560 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:29:58.560 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T10:29:58.561 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:29:58.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:58.562 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-20T10:29:58.619 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:58.624 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:29:58.648 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T10:29:58.648 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:29:58.648 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:58.738 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:29:58.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:29:58.738 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:58.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:29:58.931 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:29:58.991 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:29:58.991 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:29:58.991 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:29:58.991 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:29:58.991 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:29:58.992 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:58.993 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:58.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:58.994 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:29:59.049 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:29:59.049 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T10:29:59.054 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:29:59.123 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:29:59.165 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:29:59.166 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:59.167 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:29:59.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:59.168 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:29:59.226 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-04-20T10:29:59.226+0000 7f1f10843480 -1 init, newargv = 0x558f2a96cee0 newargc=15 2024-04-20T10:29:59.227 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[38989]: starting ceph client 2024-04-20T10:29:59.237 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[38989]: starting fuse 2024-04-20T10:29:59.255 INFO:teuthology.orchestra.run.smithi047.stdout:42 2024-04-20T10:29:59.255 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T10:29:59.255 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:29:59.255 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-04-20T10:29:59.255 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-04-20T10:29:59.255 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:29:59.256 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:29:59.257 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-04-20T10:29:59.257 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-04-20T10:29:59.257 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-04-20T10:29:59.257 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:29:59.257 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:29:59.257 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-04-20T10:29:59.257 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:29:59.455 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.38989.asok 2024-04-20T10:29:59.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:29:59 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:29:59.473 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.38989.asok 2024-04-20T10:29:59.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:59.473 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.38989.asok status 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "38989", 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-04-20T10:29:59.714 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 5832, 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 5832 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 957679016 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:29:59.715 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 957679016 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.5832 192.168.0.1:0/957679016", 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/957679016", 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 24, 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 43, 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-04-20T10:29:59.716 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:29:59.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:59.729 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-04-20T10:29:59.729 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:59.788 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-04-20T10:29:59.788 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:59.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:59.788 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:29:59.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:29:59.875 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls testdir1) 2024-04-20T10:29:59.942 INFO:tasks.ceph.mds.c.smithi047.stderr:2024-04-20T10:29:59.942+0000 7f9ab4c00640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-04-20T10:29:59.944 INFO:teuthology.orchestra.run.smithi047.stdout:testfile 2024-04-20T10:29:59.945 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:00.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.163+0000 7f3b47800640 1 Processor -- start 2024-04-20T10:30:00.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.163+0000 7f3b47800640 1 -- start start 2024-04-20T10:30:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.165+0000 7f3b47800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 0x7f3b401696e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.165+0000 7f3b47800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b400cd550 con 0x7f3b4015ed40 2024-04-20T10:30:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.165+0000 7f3b47800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b400d0b30 con 0x7f3b4015f120 2024-04-20T10:30:00.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.165+0000 7f3b47800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400ccf90 con 0x7f3b401672f0 2024-04-20T10:30:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.165+0000 7f3b44e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3b4015ed40 0x7f3b400b6f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43850/0 (socket says 172.21.15.47:43850) 2024-04-20T10:30:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.165+0000 7f3b44e00640 1 -- 172.21.15.47:0/1488214392 learned_addr learned my addr 172.21.15.47:0/1488214392 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b45800640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 0x7f3b401696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 491436944 0 0) 0x7f3b400cd550 con 0x7f3b4015ed40 2024-04-20T10:30:00.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b24003520 con 0x7f3b4015ed40 2024-04-20T10:30:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1495634285 0 0) 0x7f3b24003520 con 0x7f3b4015ed40 2024-04-20T10:30:00.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 msgr2=0x7f3b401696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 0x7f3b401696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 >> v1:172.21.15.175:6789/0 conn(0x7f3b4015f120 legacy=0x7f3b4015f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b400cd270 con 0x7f3b4015ed40 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f3b300019f0 con 0x7f3b4015ed40 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 >> v1:172.21.15.47:6789/0 conn(0x7f3b4015ed40 legacy=0x7f3b400b6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 0x7f3b240039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b24003f60 0x7f3b24004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b24004d40 0x7f3b240051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400ccf90 con 0x7f3b24003f60 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2400bb50 con 0x7f3b24004d40 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.166+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b2400bd50 con 0x7f3b401672f0 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b24003f60 0x7f3b24004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b44e00640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 0x7f3b240039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b3fe00640 1 -- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 msgr2=0x7f3b240039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b401672f0 0x7f3b240039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b45800640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b24004d40 0x7f3b240051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b3fe00640 1 -- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b24004d40 msgr2=0x7f3b240051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b24004d40 0x7f3b240051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.167+0000 7f3b3fe00640 1 -- 172.21.15.47:0/1488214392 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b24003520 con 0x7f3b24003f60 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.168+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b24003f60 0x7f3b24004800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3b28001750 tx=0x7f3b24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f304614bfd11cf61 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28001370 con 0x7f3b24003f60 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b280052d0 con 0x7f3b24003f60 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b3f400640 1 -- 172.21.15.47:0/1488214392 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28003830 con 0x7f3b24003f60 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b47800640 1 -- 172.21.15.47:0/1488214392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b24003f60 msgr2=0x7f3b24004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b47800640 1 --2- 172.21.15.47:0/1488214392 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b24003f60 0x7f3b24004800 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f3b28001750 tx=0x7f3b24005830 comp rx=0 tx=0).stop 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b45800640 1 -- 172.21.15.47:0/1488214392 reap_dead start 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b47800640 1 -- 172.21.15.47:0/1488214392 shutdown_connections 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.169+0000 7f3b47800640 1 -- 172.21.15.47:0/1488214392 >> 172.21.15.47:0/1488214392 conn(0x7f3b400704e0 msgr2=0x7f3b400708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.170+0000 7f3b47800640 1 -- 172.21.15.47:0/1488214392 shutdown_connections 2024-04-20T10:30:00.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.170+0000 7f3b47800640 1 -- 172.21.15.47:0/1488214392 wait complete. 2024-04-20T10:30:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.171+0000 7f3b47800640 1 Processor -- start 2024-04-20T10:30:00.171 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.172+0000 7f3b47800640 1 -- start start 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b47800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b24003f60 0x7f3b40162560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b47800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b400b5aa0 0x7f3b40162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b47800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40166a50 0x7f3b40144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b47800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400ccc60 con 0x7f3b400b5aa0 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b47800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b400cf600 con 0x7f3b40166a50 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b47800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b400b4f00 con 0x7f3b24003f60 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b400b5aa0 0x7f3b40162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b400b5aa0 0x7f3b40162aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38154/0 (socket says 172.21.15.47:38154) 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 -- 172.21.15.47:0/3960238487 learned_addr learned my addr 172.21.15.47:0/3960238487 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b44e00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b24003f60 0x7f3b40162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 -- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b24003f60 msgr2=0x7f3b40162560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b24003f60 0x7f3b40162560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 -- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40166a50 msgr2=0x7f3b40144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b45800640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40166a50 0x7f3b40144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40166a50 0x7f3b40144500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b3fe00640 1 -- 172.21.15.47:0/3960238487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b401659b0 con 0x7f3b400b5aa0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.173+0000 7f3b44e00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b24003f60 0x7f3b40162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.174+0000 7f3b3fe00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b400b5aa0 0x7f3b40162aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3b2802bd00 tx=0x7f3b28001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.174+0000 7f3b3cc00640 1 -- 172.21.15.47:0/3960238487 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28041070 con 0x7f3b400b5aa0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.174+0000 7f3b3cc00640 1 -- 172.21.15.47:0/3960238487 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b28005480 con 0x7f3b400b5aa0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.174+0000 7f3b3cc00640 1 -- 172.21.15.47:0/3960238487 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b28001370 con 0x7f3b400b5aa0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.174+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b40165c90 con 0x7f3b400b5aa0 2024-04-20T10:30:00.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.174+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b40166250 con 0x7f3b400b5aa0 2024-04-20T10:30:00.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.175+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b40069090 con 0x7f3b400b5aa0 2024-04-20T10:30:00.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.176+0000 7f3b3cc00640 1 -- 172.21.15.47:0/3960238487 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b28044070 con 0x7f3b400b5aa0 2024-04-20T10:30:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.178+0000 7f3b3cc00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b1c06c920 0x7f3b1c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.178+0000 7f3b3cc00640 1 -- 172.21.15.47:0/3960238487 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f3b280ad820 con 0x7f3b400b5aa0 2024-04-20T10:30:00.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.179+0000 7f3b44e00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b1c06c920 0x7f3b1c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.180+0000 7f3b44e00640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b1c06c920 0x7f3b1c06ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b30002230 tx=0x7f3b30002100 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:00.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.182+0000 7f3b3cc00640 1 -- 172.21.15.47:0/3960238487 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3b2807b9d0 con 0x7f3b400b5aa0 2024-04-20T10:30:00.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.432+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3b40145970 con 0x7f3b400b5aa0 2024-04-20T10:30:00.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.433+0000 7f3b3cc00640 1 -- 172.21.15.47:0/3960238487 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5207 (secure 0 0 0) 0x7f3b2807b2d0 con 0x7f3b400b5aa0 2024-04-20T10:30:00.434 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:00.434 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:55.499328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5475,"qdb_cluster":[5475]},"id":2}]} 2024-04-20T10:30:00.434 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-04-20T10:30:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b1c06c920 msgr2=0x7f3b1c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b1c06c920 0x7f3b1c06ede0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3b30002230 tx=0x7f3b30002100 comp rx=0 tx=0).stop 2024-04-20T10:30:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b400b5aa0 msgr2=0x7f3b40162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b400b5aa0 0x7f3b40162aa0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f3b2802bd00 tx=0x7f3b28001120 comp rx=0 tx=0).stop 2024-04-20T10:30:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 shutdown_connections 2024-04-20T10:30:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b40166a50 0x7f3b40144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b1c06c920 0x7f3b1c06ede0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b400b5aa0 0x7f3b40162aa0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 --2- 172.21.15.47:0/3960238487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b24003f60 0x7f3b40162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.436+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 >> 172.21.15.47:0/3960238487 conn(0x7f3b400704e0 msgr2=0x7f3b40145e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.437+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 shutdown_connections 2024-04-20T10:30:00.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.437+0000 7f3b47800640 1 -- 172.21.15.47:0/3960238487 wait complete. 2024-04-20T10:30:00.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-20T10:30:00.710 INFO:teuthology.orchestra.run.smithi047.stdout:{"message":"","return_code":0} 2024-04-20T10:30:00.711 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-20T10:30:00.711 DEBUG:teuthology.orchestra.run.smithi047:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 parent 2024-04-20T10:30:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.816+0000 7fedab20bf40 1 Processor -- start 2024-04-20T10:30:00.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.816+0000 7fedab20bf40 1 -- start start 2024-04-20T10:30:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.817+0000 7fedab20bf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c39a230 0x555e1c475100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.817+0000 7fedab20bf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 0x555e1c455600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.817+0000 7fedab20bf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c3cc1e0 0x555e1c4546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.817+0000 7fedab20bf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555e1c342cc0 con 0x555e1c39a230 2024-04-20T10:30:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.817+0000 7fedab20bf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555e1c346d00 con 0x555e1c39a610 2024-04-20T10:30:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.817+0000 7fedab20bf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x555e1c3e5940 con 0x555e1c3cc1e0 2024-04-20T10:30:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.817+0000 7feda7800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c39a230 0x555e1c475100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c39a230 0x555e1c475100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38168/0 (socket says 172.21.15.47:38168) 2024-04-20T10:30:00.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 -- 172.21.15.47:0/1623947080 learned_addr learned my addr 172.21.15.47:0/1623947080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda6e00640 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 0x555e1c455600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda8200640 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c3cc1e0 0x555e1c4546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 -- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c3cc1e0 msgr2=0x555e1c4546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c3cc1e0 0x555e1c4546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 -- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 msgr2=0x555e1c455600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 0x555e1c455600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 -- 172.21.15.47:0/1623947080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1c457690 con 0x555e1c39a230 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda8200640 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c3cc1e0 0x555e1c4546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.818+0000 7feda7800640 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c39a230 0x555e1c475100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed98001660 tx=0x7fed98001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e828b76b71a4c7d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7feda5a00640 1 -- 172.21.15.47:0/1623947080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed98016070 con 0x555e1c39a230 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7feda5a00640 1 -- 172.21.15.47:0/1623947080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9800f710 con 0x555e1c39a230 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7feda5a00640 1 -- 172.21.15.47:0/1623947080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9800fae0 con 0x555e1c39a230 2024-04-20T10:30:00.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 -- 172.21.15.47:0/1623947080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c39a230 msgr2=0x555e1c475100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c39a230 0x555e1c475100 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fed98001660 tx=0x7fed98001b20 comp rx=0 tx=0).stop 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 -- 172.21.15.47:0/1623947080 shutdown_connections 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c3cc1e0 0x555e1c4546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 0x555e1c455600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 --2- 172.21.15.47:0/1623947080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c39a230 0x555e1c475100 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 -- 172.21.15.47:0/1623947080 >> 172.21.15.47:0/1623947080 conn(0x555e1c3ca9a0 msgr2=0x555e1c458370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 -- 172.21.15.47:0/1623947080 shutdown_connections 2024-04-20T10:30:00.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.819+0000 7fedab20bf40 1 -- 172.21.15.47:0/1623947080 wait complete. 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.820+0000 7fedab20bf40 1 Processor -- start 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.820+0000 7fedab20bf40 1 -- start start 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.821+0000 7fedab20bf40 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c39a230 0x555e1c454f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.821+0000 7fedab20bf40 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 0x555e1c458e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.821+0000 7fedab20bf40 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c3cc1e0 0x555e1c4751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.821+0000 7fedab20bf40 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x555e1c4581a0 con 0x555e1c3cc1e0 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.821+0000 7fedab20bf40 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x555e1c3ce390 con 0x555e1c39a610 2024-04-20T10:30:00.823 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.821+0000 7fedab20bf40 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x555e1c3ce0f0 con 0x555e1c39a230 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c3cc1e0 0x555e1c4751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c3cc1e0 0x555e1c4751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38172/0 (socket says 172.21.15.47:38172) 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 -- 172.21.15.47:0/3834951188 learned_addr learned my addr 172.21.15.47:0/3834951188 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda7800640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c39a230 0x555e1c454f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda6e00640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 0x555e1c458e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 -- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c39a230 msgr2=0x555e1c454f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c39a230 0x555e1c454f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 -- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 msgr2=0x555e1c458e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x555e1c39a610 0x555e1c458e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda8200640 1 -- 172.21.15.47:0/3834951188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e1c3e4c20 con 0x555e1c3cc1e0 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.822+0000 7feda7800640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x555e1c39a230 0x555e1c454f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.823+0000 7feda8200640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c3cc1e0 0x555e1c4751a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feda00015f0 tx=0x7feda000a6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.823+0000 7fed9fe00640 1 -- 172.21.15.47:0/3834951188 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda000e640 con 0x555e1c3cc1e0 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.823+0000 7fed9fe00640 1 -- 172.21.15.47:0/3834951188 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feda000e7e0 con 0x555e1c3cc1e0 2024-04-20T10:30:00.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.823+0000 7fed9fe00640 1 -- 172.21.15.47:0/3834951188 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda000eb90 con 0x555e1c3cc1e0 2024-04-20T10:30:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.823+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555e1c4725f0 con 0x555e1c3cc1e0 2024-04-20T10:30:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.823+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e1c46ce70 con 0x555e1c3cc1e0 2024-04-20T10:30:00.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.826+0000 7fed9fe00640 1 -- 172.21.15.47:0/3834951188 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feda0009030 con 0x555e1c3cc1e0 2024-04-20T10:30:00.826 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.826+0000 7fed9fe00640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8806b300 0x7fed8806d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.827+0000 7fed9fe00640 1 -- 172.21.15.47:0/3834951188 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7feda008af00 con 0x555e1c3cc1e0 2024-04-20T10:30:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.827+0000 7fedab20bf40 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x555e1c4756e0 0x555e1c46ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.827+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 --> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] -- osd_op(unknown.0.0:1 5.10 5:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43) -- 0x555e1c453ca0 con 0x555e1c4756e0 2024-04-20T10:30:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.827+0000 7feda7800640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8806b300 0x7fed8806d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.827 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.827+0000 7feda6e00640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x555e1c4756e0 0x555e1c46ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:00.828 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.828+0000 7feda6e00640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x555e1c4756e0 0x555e1c46ded0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.828+0000 7feda6e00640 1 -- 172.21.15.47:0/3834951188 <== osd.2 v2:172.21.15.47:6822/2832687130 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (64) out=64b] v0'0 uv17 ondisk = 0) ==== 164+0+64 (crc 0 0 0) 0x7fed94052440 con 0x555e1c4756e0 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.828+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x555e1c4756e0 msgr2=0x555e1c46ded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.828+0000 7feda7800640 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8806b300 0x7fed8806d7c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed98001b60 tx=0x7fed98008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.828+0000 7fedab20bf40 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x555e1c4756e0 0x555e1c46ded0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8806b300 msgr2=0x7fed8806d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8806b300 0x7fed8806d7c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed98001b60 tx=0x7fed98008030 comp rx=0 tx=0).stop 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c3cc1e0 msgr2=0x555e1c4751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 --2- 172.21.15.47:0/3834951188 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x555e1c3cc1e0 0x555e1c4751a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7feda00015f0 tx=0x7feda000a6b0 comp rx=0 tx=0).stop 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7feda8200640 1 -- 172.21.15.47:0/3834951188 reap_dead start 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 shutdown_connections 2024-04-20T10:30:00.829 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 >> 172.21.15.47:0/3834951188 conn(0x555e1c3ca9a0 msgr2=0x555e1c458320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 shutdown_connections 2024-04-20T10:30:00.830 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:00.829+0000 7fedab20bf40 1 -- 172.21.15.47:0/3834951188 wait complete. 2024-04-20T10:30:00.833 INFO:teuthology.orchestra.run.smithi047.stdout::testdir1 2024-04-20T10:30:00.834 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T10:30:00.834 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:30:00.920 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:30:01.051 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T10:30:01.051 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:30:01.052 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:30:01.052 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:01.113 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:01.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.326+0000 7f687f400640 1 Processor -- start 2024-04-20T10:30:01.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.326+0000 7f687f400640 1 -- start start 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.327+0000 7f687f400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68780d4270 0x7f68780d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.327+0000 7f687f400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f68780d4650 0x7f68780b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.327+0000 7f687f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780b17d0 0x7f68780d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.327+0000 7f687f400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f68780d40c0 con 0x7f68780d4270 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.327+0000 7f687f400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f68780d3820 con 0x7f68780d4650 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.327+0000 7f687f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68780d3b00 con 0x7f68780b17d0 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68780d4270 0x7f68780d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f68780d4270 0x7f68780d1a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38176/0 (socket says 172.21.15.47:38176) 2024-04-20T10:30:01.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 -- 172.21.15.47:0/2065257365 learned_addr learned my addr 172.21.15.47:0/2065257365 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:01.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687d400640 1 --2- 172.21.15.47:0/2065257365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780b17d0 0x7f68780d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 -- 172.21.15.47:0/2065257365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780b17d0 msgr2=0x7f68780d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:01.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 --2- 172.21.15.47:0/2065257365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780b17d0 0x7f68780d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 -- 172.21.15.47:0/2065257365 >> v2:172.21.15.175:3300/0 conn(0x7f68780d4650 msgr2=0x7f68780b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:01.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 --2- 172.21.15.47:0/2065257365 >> v2:172.21.15.175:3300/0 conn(0x7f68780d4650 0x7f68780b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.328+0000 7f687ca00640 1 -- 172.21.15.47:0/2065257365 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68780d3de0 con 0x7f68780d4270 2024-04-20T10:30:01.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.329+0000 7f687ca00640 1 --2- 172.21.15.47:0/2065257365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4270 0x7f68780d1a00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f68780b8060 tx=0x7f686802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de772f4ecbac8e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:01.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.330+0000 7f6877400640 1 -- 172.21.15.47:0/2065257365 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686802ca80 con 0x7f68780d4270 2024-04-20T10:30:01.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.330+0000 7f6877400640 1 -- 172.21.15.47:0/2065257365 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f686802cc20 con 0x7f68780d4270 2024-04-20T10:30:01.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.330+0000 7f6877400640 1 -- 172.21.15.47:0/2065257365 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68680343f0 con 0x7f68780d4270 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.330+0000 7f687f400640 1 -- 172.21.15.47:0/2065257365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4270 msgr2=0x7f68780d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.330+0000 7f687f400640 1 --2- 172.21.15.47:0/2065257365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4270 0x7f68780d1a00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f68780b8060 tx=0x7f686802bee0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.331+0000 7f687f400640 1 -- 172.21.15.47:0/2065257365 shutdown_connections 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.331+0000 7f687f400640 1 --2- 172.21.15.47:0/2065257365 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780b17d0 0x7f68780d0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.331+0000 7f687f400640 1 --2- 172.21.15.47:0/2065257365 >> v2:172.21.15.175:3300/0 conn(0x7f68780d4650 0x7f68780b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.331+0000 7f687f400640 1 --2- 172.21.15.47:0/2065257365 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4270 0x7f68780d1a00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.331+0000 7f687f400640 1 -- 172.21.15.47:0/2065257365 >> 172.21.15.47:0/2065257365 conn(0x7f68780b99c0 msgr2=0x7f68780b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.331+0000 7f687f400640 1 -- 172.21.15.47:0/2065257365 shutdown_connections 2024-04-20T10:30:01.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.331+0000 7f687f400640 1 -- 172.21.15.47:0/2065257365 wait complete. 2024-04-20T10:30:01.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.333+0000 7f687f400640 1 Processor -- start 2024-04-20T10:30:01.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.333+0000 7f687f400640 1 -- start start 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68780b17d0 0x7f687806d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780d4270 0x7f687815daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4650 0x7f68780c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687f400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f68780d4ae0 con 0x7f68780d4650 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68780d53b0 con 0x7f68780b17d0 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68780d4c60 con 0x7f68780d4270 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4650 0x7f68780c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4650 0x7f68780c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38182/0 (socket says 172.21.15.47:38182) 2024-04-20T10:30:01.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687d400640 1 -- 172.21.15.47:0/76764845 learned_addr learned my addr 172.21.15.47:0/76764845 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f6877e00640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780d4270 0x7f687815daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.334+0000 7f687ca00640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68780b17d0 0x7f687806d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687d400640 1 -- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780d4270 msgr2=0x7f687815daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687d400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780d4270 0x7f687815daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687d400640 1 -- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68780b17d0 msgr2=0x7f687806d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687d400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68780b17d0 0x7f687806d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687d400640 1 -- 172.21.15.47:0/76764845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687815dfe0 con 0x7f68780d4650 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687d400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4650 0x7f68780c9bc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f687000c510 tx=0x7f687000c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:01.336 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f6874c00640 1 -- 172.21.15.47:0/76764845 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870015070 con 0x7f68780d4650 2024-04-20T10:30:01.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f6874c00640 1 -- 172.21.15.47:0/76764845 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6870004240 con 0x7f68780d4650 2024-04-20T10:30:01.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f6874c00640 1 -- 172.21.15.47:0/76764845 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870003610 con 0x7f68780d4650 2024-04-20T10:30:01.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687806c790 con 0x7f68780d4650 2024-04-20T10:30:01.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.335+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687806cd50 con 0x7f68780d4650 2024-04-20T10:30:01.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.337+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6878068980 con 0x7f68780d4650 2024-04-20T10:30:01.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.338+0000 7f6874c00640 1 -- 172.21.15.47:0/76764845 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68700037b0 con 0x7f68780d4650 2024-04-20T10:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.338+0000 7f6874c00640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f685406c920 0x7f685406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.339+0000 7f6874c00640 1 -- 172.21.15.47:0/76764845 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f687008bb60 con 0x7f68780d4650 2024-04-20T10:30:01.339 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.339+0000 7f687ca00640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f685406c920 0x7f685406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.340+0000 7f687ca00640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f685406c920 0x7f685406ede0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6868035000 tx=0x7f6868030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:01.343 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.343+0000 7f6874c00640 1 -- 172.21.15.47:0/76764845 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6870059c80 con 0x7f68780d4650 2024-04-20T10:30:01.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.547+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f68780b22d0 con 0x7f68780d4650 2024-04-20T10:30:01.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.549+0000 7f6874c00640 1 -- 172.21.15.47:0/76764845 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17083 (secure 0 0 0) 0x7f6870059580 con 0x7f68780d4650 2024-04-20T10:30:01.550 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:01.550 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:52.475429+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1013167846":"2024-04-21T10:29:52.475379+0000","172.21.15.47:6834/1013167846":"2024-04-21T10:29:52.475379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f685406c920 msgr2=0x7f685406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f685406c920 0x7f685406ede0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6868035000 tx=0x7f6868030000 comp rx=0 tx=0).stop 2024-04-20T10:30:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4650 msgr2=0x7f68780c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4650 0x7f68780c9bc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f687000c510 tx=0x7f687000c9d0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 shutdown_connections 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f685406c920 0x7f685406ede0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68780d4650 0x7f68780c9bc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68780d4270 0x7f687815daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 --2- 172.21.15.47:0/76764845 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68780b17d0 0x7f687806d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.552+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 >> 172.21.15.47:0/76764845 conn(0x7f68780b99c0 msgr2=0x7f687815f4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.553+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 shutdown_connections 2024-04-20T10:30:01.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.553+0000 7f687f400640 1 -- 172.21.15.47:0/76764845 wait complete. 2024-04-20T10:30:01.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:29:23.363565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:29:24.330114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:30:01.573 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:30:01.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.782+0000 7fcd5e000640 1 Processor -- start 2024-04-20T10:30:01.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.782+0000 7fcd5e000640 1 -- start start 2024-04-20T10:30:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.784+0000 7fcd5e000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcd5815b3b0 0x7fcd5806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.784+0000 7fcd5e000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd580ade40 con 0x7fcd580d0360 2024-04-20T10:30:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.784+0000 7fcd5e000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcd580ae0e0 con 0x7fcd5806dcb0 2024-04-20T10:30:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.784+0000 7fcd5e000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fcd5815b790 con 0x7fcd5815b3b0 2024-04-20T10:30:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.784+0000 7fcd57400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcd5815b3b0 0x7fcd5806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.784+0000 7fcd57e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcd580d0360 0x7fcd580d0770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43870/0 (socket says 172.21.15.47:43870) 2024-04-20T10:30:01.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.784+0000 7fcd57e00640 1 -- 172.21.15.47:0/1502363651 learned_addr learned my addr 172.21.15.47:0/1502363651 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.785+0000 7fcd57400640 1 -- 172.21.15.47:0/1502363651 >> v1:172.21.15.175:6789/0 conn(0x7fcd5806dcb0 legacy=0x7fcd58071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.785+0000 7fcd57400640 1 -- 172.21.15.47:0/1502363651 >> v1:172.21.15.47:6789/0 conn(0x7fcd580d0360 legacy=0x7fcd580d0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.785+0000 7fcd57400640 1 -- 172.21.15.47:0/1502363651 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd58072260 con 0x7fcd5815b3b0 2024-04-20T10:30:01.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.785+0000 7fcd56a00640 1 -- 172.21.15.47:0/1502363651 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4071862508 0 0) 0x7fcd580ade40 con 0x7fcd580d0360 2024-04-20T10:30:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.786+0000 7fcd57400640 1 --2- 172.21.15.47:0/1502363651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd5815b3b0 0x7fcd5806eaa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcd5806cfa0 tx=0x7fcd4802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad42852f1f6d039 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.786+0000 7fcd56a00640 1 -- 172.21.15.47:0/1502363651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48035020 con 0x7fcd5815b3b0 2024-04-20T10:30:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.786+0000 7fcd56a00640 1 -- 172.21.15.47:0/1502363651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd48031830 con 0x7fcd5815b3b0 2024-04-20T10:30:01.786 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.786+0000 7fcd56a00640 1 -- 172.21.15.47:0/1502363651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd48031c00 con 0x7fcd5815b3b0 2024-04-20T10:30:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.787+0000 7fcd5e000640 1 -- 172.21.15.47:0/1502363651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd5815b3b0 msgr2=0x7fcd5806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.787+0000 7fcd5e000640 1 --2- 172.21.15.47:0/1502363651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd5815b3b0 0x7fcd5806eaa0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcd5806cfa0 tx=0x7fcd4802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.787+0000 7fcd5e000640 1 -- 172.21.15.47:0/1502363651 shutdown_connections 2024-04-20T10:30:01.787 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.787+0000 7fcd5e000640 1 --2- 172.21.15.47:0/1502363651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd5815b3b0 0x7fcd5806eaa0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.787+0000 7fcd5e000640 1 -- 172.21.15.47:0/1502363651 >> 172.21.15.47:0/1502363651 conn(0x7fcd580b3350 msgr2=0x7fcd580b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.787+0000 7fcd5e000640 1 -- 172.21.15.47:0/1502363651 shutdown_connections 2024-04-20T10:30:01.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.787+0000 7fcd5e000640 1 -- 172.21.15.47:0/1502363651 wait complete. 2024-04-20T10:30:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.789+0000 7fcd5e000640 1 Processor -- start 2024-04-20T10:30:01.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.790+0000 7fcd5e000640 1 -- start start 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd5806dcb0 0x7fcd580cf290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd580d0360 0x7fcd580b1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd5e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5815b3b0 0x7fcd581562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd5e000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcd580ae940 con 0x7fcd580d0360 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd5e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd580ae6a0 con 0x7fcd5815b3b0 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd5e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd580aec20 con 0x7fcd5806dcb0 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd4ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd580d0360 0x7fcd580b1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd4ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd580d0360 0x7fcd580b1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38188/0 (socket says 172.21.15.47:38188) 2024-04-20T10:30:01.792 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.791+0000 7fcd4ea00640 1 -- 172.21.15.47:0/121871440 learned_addr learned my addr 172.21.15.47:0/121871440 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd57e00640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5815b3b0 0x7fcd581562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd57400640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd5806dcb0 0x7fcd580cf290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd4ea00640 1 -- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd5806dcb0 msgr2=0x7fcd580cf290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd4ea00640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd5806dcb0 0x7fcd580cf290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd4ea00640 1 -- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5815b3b0 msgr2=0x7fcd581562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd4ea00640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5815b3b0 0x7fcd581562b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd4ea00640 1 -- 172.21.15.47:0/121871440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd58151300 con 0x7fcd580d0360 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd4ea00640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd580d0360 0x7fcd580b1fc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c410 tx=0x7fcd4000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.792+0000 7fcd4fe00640 1 -- 172.21.15.47:0/121871440 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40017070 con 0x7fcd580d0360 2024-04-20T10:30:01.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.793+0000 7fcd4fe00640 1 -- 172.21.15.47:0/121871440 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcd400035f0 con 0x7fcd580d0360 2024-04-20T10:30:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.793+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd58169260 con 0x7fcd580d0360 2024-04-20T10:30:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.793+0000 7fcd4fe00640 1 -- 172.21.15.47:0/121871440 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd40003a20 con 0x7fcd580d0360 2024-04-20T10:30:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.793+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd58169820 con 0x7fcd580d0360 2024-04-20T10:30:01.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.794+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd58169a20 con 0x7fcd580d0360 2024-04-20T10:30:01.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.795+0000 7fcd4fe00640 1 -- 172.21.15.47:0/121871440 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcd4001a070 con 0x7fcd580d0360 2024-04-20T10:30:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.796+0000 7fcd4fe00640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd3406c8f0 0x7fcd3406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:01.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.796+0000 7fcd4fe00640 1 -- 172.21.15.47:0/121871440 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fcd4008b280 con 0x7fcd580d0360 2024-04-20T10:30:01.797 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.797+0000 7fcd57400640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd3406c8f0 0x7fcd3406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:01.798 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.798+0000 7fcd57400640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd3406c8f0 0x7fcd3406edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcd48036000 tx=0x7fcd48008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:01.800 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:01.801+0000 7fcd4fe00640 1 -- 172.21.15.47:0/121871440 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcd58169a20 con 0x7fcd580d0360 2024-04-20T10:30:02.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.004+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fcd58068980 con 0x7fcd580d0360 2024-04-20T10:30:02.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.005+0000 7fcd4fe00640 1 -- 172.21.15.47:0/121871440 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7fcd40059430 con 0x7fcd580d0360 2024-04-20T10:30:02.005 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd3406c8f0 msgr2=0x7fcd3406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd3406c8f0 0x7fcd3406edb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fcd48036000 tx=0x7fcd48008030 comp rx=0 tx=0).stop 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd580d0360 msgr2=0x7fcd580b1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd580d0360 0x7fcd580b1fc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000c410 tx=0x7fcd4000c8d0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 shutdown_connections 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd5815b3b0 0x7fcd581562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd3406c8f0 0x7fcd3406edb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcd580d0360 0x7fcd580b1fc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 --2- 172.21.15.47:0/121871440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd5806dcb0 0x7fcd580cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 >> 172.21.15.47:0/121871440 conn(0x7fcd580b3350 msgr2=0x7fcd58158180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:02.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 shutdown_connections 2024-04-20T10:30:02.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.008+0000 7fcd5e000640 1 -- 172.21.15.47:0/121871440 wait complete. 2024-04-20T10:30:02.028 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:30:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.246+0000 7f7162a00640 1 Processor -- start 2024-04-20T10:30:02.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.247+0000 7f7162a00640 1 -- start start 2024-04-20T10:30:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.248+0000 7f7162a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c161a50 0x7f715c0b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.248+0000 7f7162a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f715c06c010 0x7f715c0ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.248+0000 7f7162a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f715c0d0980 con 0x7f715c0d3b20 2024-04-20T10:30:02.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.248+0000 7f7162a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f715c0d0c60 con 0x7f715c06c010 2024-04-20T10:30:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.248+0000 7f7162a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f715c162ae0 con 0x7f715c161a50 2024-04-20T10:30:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.248+0000 7f7160c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f715c0d3b20 0x7f715c06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43882/0 (socket says 172.21.15.47:43882) 2024-04-20T10:30:02.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f7160c00640 1 -- 172.21.15.47:0/2953749826 learned_addr learned my addr 172.21.15.47:0/2953749826 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f7161600640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c161a50 0x7f715c0b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f7162000640 1 --2- 172.21.15.47:0/2953749826 >> v2:172.21.15.175:3300/0 conn(0x7f715c06c010 0x7f715c0ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 651775930 0 0) 0x7f715c0d0980 con 0x7f715c0d3b20 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7144003520 con 0x7f715c0d3b20 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 699049682 0 0) 0x7f7144003520 con 0x7f715c0d3b20 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c161a50 msgr2=0x7f715c0b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.249+0000 7f715be00640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c161a50 0x7f715c0b3180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 >> v2:172.21.15.175:3300/0 conn(0x7f715c06c010 msgr2=0x7f715c0ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 --2- 172.21.15.47:0/2953749826 >> v2:172.21.15.175:3300/0 conn(0x7f715c06c010 0x7f715c0ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c162490 con 0x7f715c0d3b20 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f7148001670 con 0x7f715c0d3b20 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 >> v1:172.21.15.47:6789/0 conn(0x7f715c0d3b20 legacy=0x7f715c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71440039b0 0x7f7144003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7144004800 0x7f7144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71440051c0 0x7f714400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f715c0d0c60 con 0x7f71440051c0 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f715c162ae0 con 0x7f7144004800 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f714400bf00 con 0x7f71440039b0 2024-04-20T10:30:02.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7162000640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71440051c0 0x7f714400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7161600640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71440039b0 0x7f7144003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7160c00640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7144004800 0x7f7144004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7162000640 1 -- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71440039b0 msgr2=0x7f7144003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7162000640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71440039b0 0x7f7144003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7162000640 1 -- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7144004800 msgr2=0x7f7144004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7162000640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7144004800 0x7f7144004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.250+0000 7f7162000640 1 -- 172.21.15.47:0/2953749826 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7144003520 con 0x7f71440051c0 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.251+0000 7f7161600640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71440039b0 0x7f7144003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.251+0000 7f7160c00640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7144004800 0x7f7144004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.252+0000 7f7162000640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71440051c0 0x7f714400b9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f715400e170 tx=0x7f7144005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=118d8ee2830988a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.252+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154039070 con 0x7f71440051c0 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.252+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f715402cc20 con 0x7f71440051c0 2024-04-20T10:30:02.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.252+0000 7f715be00640 1 -- 172.21.15.47:0/2953749826 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154009c10 con 0x7f71440051c0 2024-04-20T10:30:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.253+0000 7f7162a00640 1 -- 172.21.15.47:0/2953749826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71440051c0 msgr2=0x7f714400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.253+0000 7f7162a00640 1 --2- 172.21.15.47:0/2953749826 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71440051c0 0x7f714400b9c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f715400e170 tx=0x7f7144005830 comp rx=0 tx=0).stop 2024-04-20T10:30:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.253+0000 7f7162000640 1 -- 172.21.15.47:0/2953749826 reap_dead start 2024-04-20T10:30:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.253+0000 7f7162a00640 1 -- 172.21.15.47:0/2953749826 shutdown_connections 2024-04-20T10:30:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.253+0000 7f7162a00640 1 -- 172.21.15.47:0/2953749826 >> 172.21.15.47:0/2953749826 conn(0x7f715c0d0ed0 msgr2=0x7f715c0d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:02.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.253+0000 7f7162a00640 1 -- 172.21.15.47:0/2953749826 shutdown_connections 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.253+0000 7f7162a00640 1 -- 172.21.15.47:0/2953749826 wait complete. 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.255+0000 7f7162a00640 1 Processor -- start 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.255+0000 7f7162a00640 1 -- start start 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71440051c0 0x7f715c0c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c1581c0 0x7f715c158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f715c159b60 0x7f715c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f715c0d0600 con 0x7f715c159b60 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f715c0d0480 con 0x7f71440051c0 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f715c161e30 con 0x7f715c1581c0 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f715c159b60 0x7f715c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f715c159b60 0x7f715c159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38206/0 (socket says 172.21.15.47:38206) 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162000640 1 -- 172.21.15.47:0/3487886046 learned_addr learned my addr 172.21.15.47:0/3487886046 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:02.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7161600640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71440051c0 0x7f715c0c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162000640 1 -- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c1581c0 msgr2=0x7f715c158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162000640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c1581c0 0x7f715c158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162000640 1 -- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71440051c0 msgr2=0x7f715c0c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.256+0000 7f7162000640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71440051c0 0x7f715c0c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7162000640 1 -- 172.21.15.47:0/3487886046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c15b170 con 0x7f715c159b60 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7161600640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71440051c0 0x7f715c0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7162000640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f715c159b60 0x7f715c159fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f715400e170 tx=0x7f7154009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7159600640 1 -- 172.21.15.47:0/3487886046 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154039030 con 0x7f715c159b60 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7159600640 1 -- 172.21.15.47:0/3487886046 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7154043070 con 0x7f715c159b60 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7159600640 1 -- 172.21.15.47:0/3487886046 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154009c10 con 0x7f715c159b60 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c15a8d0 con 0x7f715c159b60 2024-04-20T10:30:02.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.257+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c15ae90 con 0x7f715c159b60 2024-04-20T10:30:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.258+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c075b30 con 0x7f715c159b60 2024-04-20T10:30:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.260+0000 7f7159600640 1 -- 172.21.15.47:0/3487886046 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f715403d7d0 con 0x7f715c159b60 2024-04-20T10:30:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.260+0000 7f7159600640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f713c06c8f0 0x7f713c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.261+0000 7f7159600640 1 -- 172.21.15.47:0/3487886046 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f71540ae1d0 con 0x7f715c159b60 2024-04-20T10:30:02.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.261+0000 7f7161600640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f713c06c8f0 0x7f713c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.262+0000 7f7161600640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f713c06c8f0 0x7f713c06edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f715c06c5e0 tx=0x7f7150008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:02.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.265+0000 7f7159600640 1 -- 172.21.15.47:0/3487886046 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f715407c4b0 con 0x7f715c159b60 2024-04-20T10:30:02.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.468+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f715c0d3f10 con 0x7f715c159b60 2024-04-20T10:30:02.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.469+0000 7f7159600640 1 -- 172.21.15.47:0/3487886046 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) ==== 104+0+11 (secure 0 0 0) 0x7f715407bdb0 con 0x7f715c159b60 2024-04-20T10:30:02.469 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:30:02.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f713c06c8f0 msgr2=0x7f713c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f713c06c8f0 0x7f713c06edb0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f715c06c5e0 tx=0x7f7150008040 comp rx=0 tx=0).stop 2024-04-20T10:30:02.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f715c159b60 msgr2=0x7f715c159fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f715c159b60 0x7f715c159fe0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f715400e170 tx=0x7f7154009750 comp rx=0 tx=0).stop 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 shutdown_connections 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f713c06c8f0 0x7f713c06edb0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f715c159b60 0x7f715c159fe0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f715c1581c0 0x7f715c158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 --2- 172.21.15.47:0/3487886046 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71440051c0 0x7f715c0c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 >> 172.21.15.47:0/3487886046 conn(0x7f715c0d0ed0 msgr2=0x7f715c06b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 shutdown_connections 2024-04-20T10:30:02.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.472+0000 7f7162a00640 1 -- 172.21.15.47:0/3487886046 wait complete. 2024-04-20T10:30:02.493 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:30:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.707+0000 7f2d65e00640 1 Processor -- start 2024-04-20T10:30:02.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.707+0000 7f2d65e00640 1 -- start start 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60162210 0x7f2d60162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d600b6400 0x7f2d600b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2d600b3ba0 con 0x7f2d6015e770 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2d6015eb50 con 0x7f2d600b6400 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d600d3590 con 0x7f2d60162210 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2d600b6400 0x7f2d600b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d64a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2d6015e770 0x7f2d600b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:43892/0 (socket says 172.21.15.47:43892) 2024-04-20T10:30:02.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d64a00640 1 -- 172.21.15.47:0/3083697092 learned_addr learned my addr 172.21.15.47:0/3083697092 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d5fe00640 1 --2- 172.21.15.47:0/3083697092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60162210 0x7f2d60162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65400640 1 -- 172.21.15.47:0/3083697092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60162210 msgr2=0x7f2d60162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65400640 1 --2- 172.21.15.47:0/3083697092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60162210 0x7f2d60162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.709+0000 7f2d65400640 1 -- 172.21.15.47:0/3083697092 >> v1:172.21.15.47:6789/0 conn(0x7f2d6015e770 legacy=0x7f2d600b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.710+0000 7f2d65400640 1 -- 172.21.15.47:0/3083697092 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d600d3870 con 0x7f2d600b6400 2024-04-20T10:30:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.710+0000 7f2d5f400640 1 -- 172.21.15.47:0/3083697092 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 10867876 0 0) 0x7f2d600b3ba0 con 0x7f2d6015e770 2024-04-20T10:30:02.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.710+0000 7f2d5fe00640 1 --2- 172.21.15.47:0/3083697092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60162210 0x7f2d60162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.711+0000 7f2d65400640 1 --2- 172.21.15.47:0/3083697092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d600b6400 0x7f2d600b6890 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009d90 tx=0x7f2d5802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37489eb0246eaca server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.711+0000 7f2d5f400640 1 -- 172.21.15.47:0/3083697092 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58039070 con 0x7f2d600b6400 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.711+0000 7f2d5f400640 1 -- 172.21.15.47:0/3083697092 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5802cbc0 con 0x7f2d600b6400 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.711+0000 7f2d5f400640 1 -- 172.21.15.47:0/3083697092 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58034440 con 0x7f2d600b6400 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 -- 172.21.15.47:0/3083697092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d600b6400 msgr2=0x7f2d600b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 --2- 172.21.15.47:0/3083697092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d600b6400 0x7f2d600b6890 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009d90 tx=0x7f2d5802be10 comp rx=0 tx=0).stop 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 -- 172.21.15.47:0/3083697092 shutdown_connections 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 --2- 172.21.15.47:0/3083697092 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d600b6400 0x7f2d600b6890 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 --2- 172.21.15.47:0/3083697092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d60162210 0x7f2d60162670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 -- 172.21.15.47:0/3083697092 >> 172.21.15.47:0/3083697092 conn(0x7f2d6009d6d0 msgr2=0x7f2d600b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 -- 172.21.15.47:0/3083697092 shutdown_connections 2024-04-20T10:30:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.712+0000 7f2d65e00640 1 -- 172.21.15.47:0/3083697092 wait complete. 2024-04-20T10:30:02.713 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.714+0000 7f2d65e00640 1 Processor -- start 2024-04-20T10:30:02.714 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.714+0000 7f2d65e00640 1 -- start start 2024-04-20T10:30:02.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d600b6400 0x7f2d601616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d6015e770 0x7f2d600cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60162210 0x7f2d600ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2d600d3e30 con 0x7f2d60162210 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d600b5ee0 con 0x7f2d6015e770 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d600d3120 con 0x7f2d600b6400 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60162210 0x7f2d600ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60162210 0x7f2d600ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38214/0 (socket says 172.21.15.47:38214) 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65400640 1 -- 172.21.15.47:0/946800806 learned_addr learned my addr 172.21.15.47:0/946800806 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d64a00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d600b6400 0x7f2d601616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d5fe00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d6015e770 0x7f2d600cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65400640 1 -- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d600b6400 msgr2=0x7f2d601616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65400640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d600b6400 0x7f2d601616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.716 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65400640 1 -- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d6015e770 msgr2=0x7f2d600cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.715+0000 7f2d65400640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d6015e770 0x7f2d600cc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d65400640 1 -- 172.21.15.47:0/946800806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d600cd4d0 con 0x7f2d60162210 2024-04-20T10:30:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d64a00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d600b6400 0x7f2d601616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:02.717 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d65400640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60162210 0x7f2d600ccea0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009d90 tx=0x7f2d58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d5cc00640 1 -- 172.21.15.47:0/946800806 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58044030 con 0x7f2d60162210 2024-04-20T10:30:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d5cc00640 1 -- 172.21.15.47:0/946800806 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d5802c9c0 con 0x7f2d60162210 2024-04-20T10:30:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d5cc00640 1 -- 172.21.15.47:0/946800806 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d58039040 con 0x7f2d60162210 2024-04-20T10:30:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d600cd7b0 con 0x7f2d60162210 2024-04-20T10:30:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.716+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d600cdd70 con 0x7f2d60162210 2024-04-20T10:30:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.718+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d60075b30 con 0x7f2d60162210 2024-04-20T10:30:02.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.719+0000 7f2d5cc00640 1 -- 172.21.15.47:0/946800806 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2d58047070 con 0x7f2d60162210 2024-04-20T10:30:02.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.720+0000 7f2d5cc00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4006c8f0 0x7f2d4006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:02.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.720+0000 7f2d5cc00640 1 -- 172.21.15.47:0/946800806 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f2d580ae6b0 con 0x7f2d60162210 2024-04-20T10:30:02.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.720+0000 7f2d64a00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4006c8f0 0x7f2d4006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:02.721 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.721+0000 7f2d64a00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4006c8f0 0x7f2d4006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d54000f60 tx=0x7f2d54009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:02.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.724+0000 7f2d5cc00640 1 -- 172.21.15.47:0/946800806 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2d5804e070 con 0x7f2d60162210 2024-04-20T10:30:02.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.939+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2d600cb170 con 0x7f2d60162210 2024-04-20T10:30:02.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.940+0000 7f2d5cc00640 1 -- 172.21.15.47:0/946800806 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) ==== 100+0+11 (secure 0 0 0) 0x7f2d5807c860 con 0x7f2d60162210 2024-04-20T10:30:02.940 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:30:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.942+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4006c8f0 msgr2=0x7f2d4006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.942+0000 7f2d65e00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4006c8f0 0x7f2d4006edb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d54000f60 tx=0x7f2d54009040 comp rx=0 tx=0).stop 2024-04-20T10:30:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60162210 msgr2=0x7f2d600ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60162210 0x7f2d600ccea0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009d90 tx=0x7f2d58004a40 comp rx=0 tx=0).stop 2024-04-20T10:30:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 shutdown_connections 2024-04-20T10:30:02.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2d4006c8f0 0x7f2d4006edb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2d60162210 0x7f2d600ccea0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d6015e770 0x7f2d600cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 --2- 172.21.15.47:0/946800806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d600b6400 0x7f2d601616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 >> 172.21.15.47:0/946800806 conn(0x7f2d6009d6d0 msgr2=0x7f2d600b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 shutdown_connections 2024-04-20T10:30:02.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:02.943+0000 7f2d65e00640 1 -- 172.21.15.47:0/946800806 wait complete. 2024-04-20T10:30:02.964 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:30:03.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.174+0000 7fde81000640 1 Processor -- start 2024-04-20T10:30:03.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.174+0000 7fde81000640 1 -- start start 2024-04-20T10:30:03.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.175+0000 7fde81000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:03.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.175+0000 7fde81000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fde7c15ca40 0x7fde7c0ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:03.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.176+0000 7fde81000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fde7c141c60 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.176+0000 7fde81000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fde7c15cf90 con 0x7fde7c15ca40 2024-04-20T10:30:03.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.176+0000 7fde81000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c15d270 con 0x7fde7c156c20 2024-04-20T10:30:03.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.176+0000 7fde79e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:03.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.176+0000 7fde79400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fde7c15ca40 0x7fde7c0ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:03.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.176+0000 7fde79e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:50394/0 (socket says 172.21.15.47:50394) 2024-04-20T10:30:03.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.176+0000 7fde79e00640 1 -- 172.21.15.47:0/3623507515 learned_addr learned my addr 172.21.15.47:0/3623507515 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.177+0000 7fde79400640 1 -- 172.21.15.47:0/3623507515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 msgr2=0x7fde7c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.177+0000 7fde79400640 1 --2- 172.21.15.47:0/3623507515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.177+0000 7fde79400640 1 -- 172.21.15.47:0/3623507515 >> v1:172.21.15.175:6789/0 conn(0x7fde7c0ad8e0 legacy=0x7fde7c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.177+0000 7fde79400640 1 -- 172.21.15.47:0/3623507515 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c15d550 con 0x7fde7c15ca40 2024-04-20T10:30:03.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.179+0000 7fde79400640 1 --2- 172.21.15.47:0/3623507515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0ad3a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde680097a0 tx=0x7fde6802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a55da91c3a9eece8 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.179+0000 7fde78a00640 1 -- 172.21.15.47:0/3623507515 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68035020 con 0x7fde7c15ca40 2024-04-20T10:30:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.179+0000 7fde78a00640 1 -- 172.21.15.47:0/3623507515 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde68031830 con 0x7fde7c15ca40 2024-04-20T10:30:03.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.179+0000 7fde78a00640 1 -- 172.21.15.47:0/3623507515 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde68031c00 con 0x7fde7c15ca40 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 -- 172.21.15.47:0/3623507515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 msgr2=0x7fde7c0ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 --2- 172.21.15.47:0/3623507515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0ad3a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fde680097a0 tx=0x7fde6802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 -- 172.21.15.47:0/3623507515 shutdown_connections 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 --2- 172.21.15.47:0/3623507515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0ad3a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 --2- 172.21.15.47:0/3623507515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 -- 172.21.15.47:0/3623507515 >> 172.21.15.47:0/3623507515 conn(0x7fde7c0b3350 msgr2=0x7fde7c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 -- 172.21.15.47:0/3623507515 shutdown_connections 2024-04-20T10:30:03.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.180+0000 7fde81000640 1 -- 172.21.15.47:0/3623507515 wait complete. 2024-04-20T10:30:03.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.182+0000 7fde81000640 1 Processor -- start 2024-04-20T10:30:03.182 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.182+0000 7fde81000640 1 -- start start 2024-04-20T10:30:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde81000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde7c0ad8e0 0x7fde7c1f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde81000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:03.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde81000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde81000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c06bfc0 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde81000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde7c06be40 con 0x7fde7c15ca40 2024-04-20T10:30:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde81000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde7c0d0a70 con 0x7fde7c156c20 2024-04-20T10:30:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde7c0ad8e0 0x7fde7c1f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde79e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde7c0ad8e0 0x7fde7c1f43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38220/0 (socket says 172.21.15.47:38220) 2024-04-20T10:30:03.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.183+0000 7fde79e00640 1 -- 172.21.15.47:0/4102730176 learned_addr learned my addr 172.21.15.47:0/4102730176 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79400640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde7a800640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79e00640 1 -- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 msgr2=0x7fde7c06f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79e00640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79e00640 1 -- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 msgr2=0x7fde7c0b0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79e00640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0b0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79e00640 1 -- 172.21.15.47:0/4102730176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde7c06e9a0 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79400640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde7a800640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde79e00640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde7c0ad8e0 0x7fde7c1f43d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009fd0 tx=0x7fde6c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde72000640 1 -- 172.21.15.47:0/4102730176 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c017070 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde72000640 1 -- 172.21.15.47:0/4102730176 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fde6c00c940 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde72000640 1 -- 172.21.15.47:0/4102730176 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde6c00cd10 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.184+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde7c06ec80 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.185+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde7c0ccd80 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.186+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde7c068980 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.187+0000 7fde72000640 1 -- 172.21.15.47:0/4102730176 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fde6c01a070 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.188+0000 7fde72000640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fde5c06c8f0 0x7fde5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.188+0000 7fde72000640 1 -- 172.21.15.47:0/4102730176 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fde6c08c0c0 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.188+0000 7fde79400640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fde5c06c8f0 0x7fde5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:03.189 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.189+0000 7fde79400640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fde5c06c8f0 0x7fde5c06edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde68036000 tx=0x7fde68008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:03.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.192+0000 7fde72000640 1 -- 172.21.15.47:0/4102730176 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fde6c05a270 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.409+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fde7c0cfd90 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.410 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.410+0000 7fde72000640 1 -- 172.21.15.47:0/4102730176 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v43) ==== 80+0+122 (secure 0 0 0) 0x7fde6c059b70 con 0x7fde7c0ad8e0 2024-04-20T10:30:03.411 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/1013167846 2024-04-21T10:29:52.475379+0000 2024-04-20T10:30:03.411 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/1013167846 2024-04-21T10:29:52.475379+0000 2024-04-20T10:30:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:listed 2 entries 2024-04-20T10:30:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fde5c06c8f0 msgr2=0x7fde5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fde5c06c8f0 0x7fde5c06edb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fde68036000 tx=0x7fde68008030 comp rx=0 tx=0).stop 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde7c0ad8e0 msgr2=0x7fde7c1f43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde7c0ad8e0 0x7fde7c1f43d0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fde6c009fd0 tx=0x7fde6c00ed40 comp rx=0 tx=0).stop 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 shutdown_connections 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde7c15ca40 0x7fde7c0b0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde7c156c20 0x7fde7c06f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fde5c06c8f0 0x7fde5c06edb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 --2- 172.21.15.47:0/4102730176 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fde7c0ad8e0 0x7fde7c1f43d0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 >> 172.21.15.47:0/4102730176 conn(0x7fde7c0b3350 msgr2=0x7fde7c099850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 shutdown_connections 2024-04-20T10:30:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:03.413+0000 7fde81000640 1 -- 172.21.15.47:0/4102730176 wait complete. 2024-04-20T10:30:03.433 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-04-20T10:30:03.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:03.433 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T10:30:03.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[38989]: fuse finished with error 0 and tester_r 0 2024-04-20T10:30:03.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:30:03.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:03.525 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T10:30:03.626 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:30:03.626 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:30:03.626 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:03.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:30:03.683 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:30:03.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:03.683 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T10:30:03.731 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T10:30:03.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:30:03.732 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:30:03.733 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:30:03.733 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:03.795 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T10:30:03.795 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:30:03.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:03.795 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T10:30:03.852 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T10:30:03.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:30:03.853 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:30:03.854 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:30:03.854 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:03.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:30:03.911 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:30:03.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:03.912 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T10:30:03.961 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T10:30:03.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:30:03.962 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.174+0000 7fb582800640 1 Processor -- start 2024-04-20T10:30:04.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.174+0000 7fb582800640 1 -- start start 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.175+0000 7fb582800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.175+0000 7fb582800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb57c15a1f0 0x7fb57c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.175+0000 7fb582800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb57c15cc90 con 0x7fb57c06da90 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.175+0000 7fb582800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb57c072900 con 0x7fb57c15a1f0 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.175+0000 7fb582800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c15ca40 con 0x7fb57c159e10 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.176+0000 7fb581e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb57c06da90 0x7fb57c160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56606/0 (socket says 172.21.15.47:56606) 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.176+0000 7fb581e00640 1 -- 172.21.15.47:0/3152653327 learned_addr learned my addr 172.21.15.47:0/3152653327 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:04.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.176+0000 7fb580a00640 1 --2- 172.21.15.47:0/3152653327 >> v2:172.21.15.47:3300/0 conn(0x7fb57c15a1f0 0x7fb57c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.176+0000 7fb581400640 1 --2- 172.21.15.47:0/3152653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.176+0000 7fb57be00640 1 -- 172.21.15.47:0/3152653327 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2860704086 0 0) 0x7fb57c15cc90 con 0x7fb57c06da90 2024-04-20T10:30:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.176+0000 7fb57be00640 1 -- 172.21.15.47:0/3152653327 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb564003520 con 0x7fb57c06da90 2024-04-20T10:30:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.177+0000 7fb580a00640 1 -- 172.21.15.47:0/3152653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 msgr2=0x7fb57c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.177+0000 7fb580a00640 1 --2- 172.21.15.47:0/3152653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.177+0000 7fb580a00640 1 -- 172.21.15.47:0/3152653327 >> v1:172.21.15.47:6789/0 conn(0x7fb57c06da90 legacy=0x7fb57c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.177+0000 7fb580a00640 1 -- 172.21.15.47:0/3152653327 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c15cf70 con 0x7fb57c15a1f0 2024-04-20T10:30:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.178+0000 7fb580a00640 1 --2- 172.21.15.47:0/3152653327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c15a1f0 0x7fb57c071fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb57c15cbc0 tx=0x7fb56802b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d787b168bd06779 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.178+0000 7fb57be00640 1 -- 172.21.15.47:0/3152653327 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568035020 con 0x7fb57c15a1f0 2024-04-20T10:30:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.178+0000 7fb57be00640 1 -- 172.21.15.47:0/3152653327 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb568031800 con 0x7fb57c15a1f0 2024-04-20T10:30:04.178 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.178+0000 7fb57be00640 1 -- 172.21.15.47:0/3152653327 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568031bd0 con 0x7fb57c15a1f0 2024-04-20T10:30:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 -- 172.21.15.47:0/3152653327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c15a1f0 msgr2=0x7fb57c071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 --2- 172.21.15.47:0/3152653327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c15a1f0 0x7fb57c071fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb57c15cbc0 tx=0x7fb56802b8a0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 -- 172.21.15.47:0/3152653327 shutdown_connections 2024-04-20T10:30:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 --2- 172.21.15.47:0/3152653327 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c15a1f0 0x7fb57c071fc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 --2- 172.21.15.47:0/3152653327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 -- 172.21.15.47:0/3152653327 >> 172.21.15.47:0/3152653327 conn(0x7fb57c0d0040 msgr2=0x7fb57c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 -- 172.21.15.47:0/3152653327 shutdown_connections 2024-04-20T10:30:04.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.179+0000 7fb582800640 1 -- 172.21.15.47:0/3152653327 wait complete. 2024-04-20T10:30:04.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.181+0000 7fb582800640 1 Processor -- start 2024-04-20T10:30:04.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.181+0000 7fb582800640 1 -- start start 2024-04-20T10:30:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.182+0000 7fb582800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c06da90 0x7fb57c14faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.182+0000 7fb582800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.182+0000 7fb582800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb57c15a1f0 0x7fb57c09c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.182+0000 7fb582800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15d810 con 0x7fb57c06da90 2024-04-20T10:30:04.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.182+0000 7fb582800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb57c15d530 con 0x7fb57c15a1f0 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.182+0000 7fb582800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb57c06b900 con 0x7fb57c159e10 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c06da90 0x7fb57c14faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c06da90 0x7fb57c14faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37554/0 (socket says 172.21.15.47:37554) 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 -- 172.21.15.47:0/1027656170 learned_addr learned my addr 172.21.15.47:0/1027656170 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581e00640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb57c15a1f0 0x7fb57c09c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb580a00640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 -- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 msgr2=0x7fb57c15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c15db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 -- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb57c15a1f0 msgr2=0x7fb57c09c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb57c15a1f0 0x7fb57c09c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 -- 172.21.15.47:0/1027656170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb57c15e450 con 0x7fb57c06da90 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581e00640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb57c15a1f0 0x7fb57c09c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:04.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.183+0000 7fb581400640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c06da90 0x7fb57c14faa0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb57000d800 tx=0x7fb57000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.184+0000 7fb579600640 1 -- 172.21.15.47:0/1027656170 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000b6c0 con 0x7fb57c06da90 2024-04-20T10:30:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.184+0000 7fb579600640 1 -- 172.21.15.47:0/1027656170 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb570019070 con 0x7fb57c06da90 2024-04-20T10:30:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.184+0000 7fb579600640 1 -- 172.21.15.47:0/1027656170 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57000ba80 con 0x7fb57c06da90 2024-04-20T10:30:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.184+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb57c15e730 con 0x7fb57c06da90 2024-04-20T10:30:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.184+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb57c0c9540 con 0x7fb57c06da90 2024-04-20T10:30:04.185 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.185+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb57c075b60 con 0x7fb57c06da90 2024-04-20T10:30:04.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.186+0000 7fb579600640 1 -- 172.21.15.47:0/1027656170 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb57000bc20 con 0x7fb57c06da90 2024-04-20T10:30:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.187+0000 7fb579600640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb55c06c8f0 0x7fb55c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.187+0000 7fb579600640 1 -- 172.21.15.47:0/1027656170 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fb57008c2c0 con 0x7fb57c06da90 2024-04-20T10:30:04.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.188+0000 7fb580a00640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb55c06c8f0 0x7fb55c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.189+0000 7fb580a00640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb55c06c8f0 0x7fb55c06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb568036000 tx=0x7fb568008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:04.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.191+0000 7fb579600640 1 -- 172.21.15.47:0/1027656170 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb57001c360 con 0x7fb57c06da90 2024-04-20T10:30:04.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.434+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb57c073d20 con 0x7fb57c06da90 2024-04-20T10:30:04.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.435+0000 7fb579600640 1 -- 172.21.15.47:0/1027656170 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5207 (secure 0 0 0) 0x7fb57005a470 con 0x7fb57c06da90 2024-04-20T10:30:04.436 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:04.436 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:55.499328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5475,"qdb_cluster":[5475]},"id":2}]} 2024-04-20T10:30:04.436 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-04-20T10:30:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb55c06c8f0 msgr2=0x7fb55c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb55c06c8f0 0x7fb55c06edb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb568036000 tx=0x7fb568008030 comp rx=0 tx=0).stop 2024-04-20T10:30:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c06da90 msgr2=0x7fb57c14faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c06da90 0x7fb57c14faa0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fb57000d800 tx=0x7fb57000dcc0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 shutdown_connections 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb57c15a1f0 0x7fb57c09c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb57c159e10 0x7fb57c15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb55c06c8f0 0x7fb55c06edb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 --2- 172.21.15.47:0/1027656170 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb57c06da90 0x7fb57c14faa0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 >> 172.21.15.47:0/1027656170 conn(0x7fb57c0d0040 msgr2=0x7fb57c072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 shutdown_connections 2024-04-20T10:30:04.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.438+0000 7fb582800640 1 -- 172.21.15.47:0/1027656170 wait complete. 2024-04-20T10:30:04.459 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:04.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.664+0000 7fd725600640 1 Processor -- start 2024-04-20T10:30:04.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.665+0000 7fd725600640 1 -- start start 2024-04-20T10:30:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.666+0000 7fd725600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fd720161d50 0x7fd7200d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.666+0000 7fd725600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72006c190 con 0x7fd72006cde0 2024-04-20T10:30:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.666+0000 7fd725600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd72006c470 con 0x7fd7200b3330 2024-04-20T10:30:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.666+0000 7fd725600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fd720162130 con 0x7fd720161d50 2024-04-20T10:30:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.667+0000 7fd724c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fd72006cde0 0x7fd7200b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56612/0 (socket says 172.21.15.47:56612) 2024-04-20T10:30:04.667 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.667+0000 7fd724c00640 1 -- 172.21.15.47:0/3715040381 learned_addr learned my addr 172.21.15.47:0/3715040381 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:04.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.667+0000 7fd71fe00640 1 --2- 172.21.15.47:0/3715040381 >> v2:172.21.15.175:3300/0 conn(0x7fd720161d50 0x7fd7200d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.667+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3431853112 0 0) 0x7fd72006c190 con 0x7fd72006cde0 2024-04-20T10:30:04.669 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.667+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fd704003520 con 0x7fd72006cde0 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 335342861 0 0) 0x7fd704003520 con 0x7fd72006cde0 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 >> v2:172.21.15.175:3300/0 conn(0x7fd720161d50 msgr2=0x7fd7200d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 --2- 172.21.15.47:0/3715040381 >> v2:172.21.15.175:3300/0 conn(0x7fd720161d50 0x7fd7200d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 >> v1:172.21.15.175:6789/0 conn(0x7fd7200b3330 legacy=0x7fd72006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7200d2e00 con 0x7fd72006cde0 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fd718001670 con 0x7fd72006cde0 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 >> v1:172.21.15.47:6789/0 conn(0x7fd72006cde0 legacy=0x7fd7200b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd704003a00 0x7fd704003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704004800 0x7fd704004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040051c0 0x7fd70400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd70400c090 con 0x7fd704003a00 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd70400c240 con 0x7fd704004800 2024-04-20T10:30:04.670 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd70400c440 con 0x7fd7040051c0 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd71fe00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd704003a00 0x7fd704003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.668+0000 7fd724c00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040051c0 0x7fd70400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd71fe00640 1 -- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040051c0 msgr2=0x7fd70400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd71f400640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704004800 0x7fd704004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd71fe00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040051c0 0x7fd70400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd71fe00640 1 -- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704004800 msgr2=0x7fd704004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd71fe00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704004800 0x7fd704004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd71fe00640 1 -- 172.21.15.47:0/3715040381 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd704003520 con 0x7fd704003a00 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd724c00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7040051c0 0x7fd70400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd71f400640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd704004800 0x7fd704004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:04.671 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.669+0000 7fd724c00640 1 -- 172.21.15.47:0/3715040381 reap_dead start 2024-04-20T10:30:04.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.670+0000 7fd71fe00640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd704003a00 0x7fd704003e20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd710009fc0 tx=0x7fd704005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b139b5cdc2ce3e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:04.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.670+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710039070 con 0x7fd704003a00 2024-04-20T10:30:04.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.670+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd71002cc20 con 0x7fd704003a00 2024-04-20T10:30:04.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.670+0000 7fd71ea00640 1 -- 172.21.15.47:0/3715040381 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710009c10 con 0x7fd704003a00 2024-04-20T10:30:04.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.671+0000 7fd725600640 1 -- 172.21.15.47:0/3715040381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd704003a00 msgr2=0x7fd704003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.671+0000 7fd725600640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd704003a00 0x7fd704003e20 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd710009fc0 tx=0x7fd704005830 comp rx=0 tx=0).stop 2024-04-20T10:30:04.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.671+0000 7fd725600640 1 -- 172.21.15.47:0/3715040381 shutdown_connections 2024-04-20T10:30:04.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.671+0000 7fd725600640 1 --2- 172.21.15.47:0/3715040381 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd704003a00 0x7fd704003e20 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.671+0000 7fd725600640 1 -- 172.21.15.47:0/3715040381 >> 172.21.15.47:0/3715040381 conn(0x7fd7200d4450 msgr2=0x7fd7200d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:04.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.671+0000 7fd725600640 1 -- 172.21.15.47:0/3715040381 shutdown_connections 2024-04-20T10:30:04.673 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.671+0000 7fd725600640 1 -- 172.21.15.47:0/3715040381 wait complete. 2024-04-20T10:30:04.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.673+0000 7fd725600640 1 Processor -- start 2024-04-20T10:30:04.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.673+0000 7fd725600640 1 -- start start 2024-04-20T10:30:04.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd725600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704003a00 0x7fd720152840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd725600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7200c9d50 0x7fd7200ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd725600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7200b8fd0 0x7fd7200b9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd725600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd7200d38e0 con 0x7fd7200b8fd0 2024-04-20T10:30:04.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd725600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7200d33c0 con 0x7fd7200c9d50 2024-04-20T10:30:04.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd725600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd7200d3a60 con 0x7fd704003a00 2024-04-20T10:30:04.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd724c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7200b8fd0 0x7fd7200b9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd724c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7200b8fd0 0x7fd7200b9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37574/0 (socket says 172.21.15.47:37574) 2024-04-20T10:30:04.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.674+0000 7fd724c00640 1 -- 172.21.15.47:0/1544006745 learned_addr learned my addr 172.21.15.47:0/1544006745 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:04.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd71f400640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7200c9d50 0x7fd7200ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd71fe00640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704003a00 0x7fd720152840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd724c00640 1 -- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704003a00 msgr2=0x7fd720152840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd724c00640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704003a00 0x7fd720152840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd724c00640 1 -- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7200c9d50 msgr2=0x7fd7200ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd724c00640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7200c9d50 0x7fd7200ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd724c00640 1 -- 172.21.15.47:0/1544006745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7200c81b0 con 0x7fd7200b8fd0 2024-04-20T10:30:04.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd724c00640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7200b8fd0 0x7fd7200b9480 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd7200c7f30 tx=0x7fd71800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:04.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd717e00640 1 -- 172.21.15.47:0/1544006745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718007440 con 0x7fd7200b8fd0 2024-04-20T10:30:04.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd717e00640 1 -- 172.21.15.47:0/1544006745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd718007600 con 0x7fd7200b8fd0 2024-04-20T10:30:04.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd717e00640 1 -- 172.21.15.47:0/1544006745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd718004a20 con 0x7fd7200b8fd0 2024-04-20T10:30:04.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.675+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7200c8490 con 0x7fd7200b8fd0 2024-04-20T10:30:04.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.676+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7200c8a50 con 0x7fd7200b8fd0 2024-04-20T10:30:04.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.677+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd720075b60 con 0x7fd7200b8fd0 2024-04-20T10:30:04.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.678+0000 7fd717e00640 1 -- 172.21.15.47:0/1544006745 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd71801b020 con 0x7fd7200b8fd0 2024-04-20T10:30:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.679+0000 7fd717e00640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd6fc06c920 0x7fd6fc06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:04.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.679+0000 7fd717e00640 1 -- 172.21.15.47:0/1544006745 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fd71808b8a0 con 0x7fd7200b8fd0 2024-04-20T10:30:04.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.680+0000 7fd71fe00640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd6fc06c920 0x7fd6fc06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:04.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.682+0000 7fd71fe00640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd6fc06c920 0x7fd6fc06ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd710036000 tx=0x7fd71002e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:04.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.684+0000 7fd717e00640 1 -- 172.21.15.47:0/1544006745 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd718059a50 con 0x7fd7200b8fd0 2024-04-20T10:30:04.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.894+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7200cb2e0 con 0x7fd7200b8fd0 2024-04-20T10:30:04.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.896+0000 7fd717e00640 1 -- 172.21.15.47:0/1544006745 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17083 (secure 0 0 0) 0x7fd718059350 con 0x7fd7200b8fd0 2024-04-20T10:30:04.897 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:04.897 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:52.475429+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1013167846":"2024-04-21T10:29:52.475379+0000","172.21.15.47:6834/1013167846":"2024-04-21T10:29:52.475379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd6fc06c920 msgr2=0x7fd6fc06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd6fc06c920 0x7fd6fc06ede0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fd710036000 tx=0x7fd71002e000 comp rx=0 tx=0).stop 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7200b8fd0 msgr2=0x7fd7200b9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7200b8fd0 0x7fd7200b9480 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd7200c7f30 tx=0x7fd71800d950 comp rx=0 tx=0).stop 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 shutdown_connections 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd6fc06c920 0x7fd6fc06ede0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd7200b8fd0 0x7fd7200b9480 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd7200c9d50 0x7fd7200ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.899+0000 7fd725600640 1 --2- 172.21.15.47:0/1544006745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd704003a00 0x7fd720152840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:04.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.900+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 >> 172.21.15.47:0/1544006745 conn(0x7fd7200d4450 msgr2=0x7fd72015faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.900+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 shutdown_connections 2024-04-20T10:30:04.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:04.900+0000 7fd725600640 1 -- 172.21.15.47:0/1544006745 wait complete. 2024-04-20T10:30:04.919 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:29:23.363565+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:29:24.330114+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:30:04.919 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:30:05.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.135+0000 7f2a7fe00640 1 Processor -- start 2024-04-20T10:30:05.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.135+0000 7f2a7fe00640 1 -- start start 2024-04-20T10:30:05.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.136+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80162210 0x7f2a80162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.136+0000 7f2a7fe00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a800b3ba0 con 0x7f2a8015e770 2024-04-20T10:30:05.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.136+0000 7f2a7fe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2a800d3590 con 0x7f2a800b6400 2024-04-20T10:30:05.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.136+0000 7f2a7fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a8015eb50 con 0x7f2a80162210 2024-04-20T10:30:05.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a7ea00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f2a8015e770 0x7f2a800b3490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56628/0 (socket says 172.21.15.47:56628) 2024-04-20T10:30:05.137 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a7ea00640 1 -- 172.21.15.47:0/1841805296 learned_addr learned my addr 172.21.15.47:0/1841805296 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a76000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80162210 0x7f2a80162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 528994605 0 0) 0x7f2a800b3ba0 con 0x7f2a8015e770 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f2a64003520 con 0x7f2a8015e770 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 547682593 0 0) 0x7f2a64003520 con 0x7f2a8015e770 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80162210 msgr2=0x7f2a80162670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.137+0000 7f2a7e000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80162210 0x7f2a80162670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 >> v1:172.21.15.175:6789/0 conn(0x7f2a800b6400 legacy=0x7f2a800d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a800d3870 con 0x7f2a8015e770 2024-04-20T10:30:05.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f2a70001670 con 0x7f2a8015e770 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 >> v1:172.21.15.47:6789/0 conn(0x7f2a8015e770 legacy=0x7f2a800b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a640039e0 0x7f2a64003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80162210 0x7f2a6400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2a8015eb50 con 0x7f2a64004800 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a640054f0 con 0x7f2a640039e0 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6400bf00 con 0x7f2a80162210 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a76000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a7ea00640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a640039e0 0x7f2a64003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a76000640 1 -- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80162210 msgr2=0x7f2a6400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a76000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a80162210 0x7f2a6400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a76000640 1 -- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a640039e0 msgr2=0x7f2a64003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.138+0000 7f2a76000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a640039e0 0x7f2a64003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.140 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.139+0000 7f2a76000640 1 -- 172.21.15.47:0/1841805296 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a64003520 con 0x7f2a64004800 2024-04-20T10:30:05.141 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.140+0000 7f2a76000640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ba10 tx=0x7f2a64005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f8c4f0f0b2a711 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.140+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a68036020 con 0x7f2a64004800 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.140+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a680326a0 con 0x7f2a64004800 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.140+0000 7f2a7e000640 1 -- 172.21.15.47:0/1841805296 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800b5e0 con 0x7f2a64004800 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.141+0000 7f2a7fe00640 1 -- 172.21.15.47:0/1841805296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 msgr2=0x7f2a64004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.141+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/1841805296 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a64004c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800ba10 tx=0x7f2a64005830 comp rx=0 tx=0).stop 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.141+0000 7f2a7f400640 1 -- 172.21.15.47:0/1841805296 reap_dead start 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.141+0000 7f2a7fe00640 1 -- 172.21.15.47:0/1841805296 shutdown_connections 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.141+0000 7f2a7fe00640 1 -- 172.21.15.47:0/1841805296 >> 172.21.15.47:0/1841805296 conn(0x7f2a8009d6d0 msgr2=0x7f2a800b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.141+0000 7f2a7fe00640 1 -- 172.21.15.47:0/1841805296 shutdown_connections 2024-04-20T10:30:05.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.141+0000 7f2a7fe00640 1 -- 172.21.15.47:0/1841805296 wait complete. 2024-04-20T10:30:05.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.143+0000 7f2a7fe00640 1 Processor -- start 2024-04-20T10:30:05.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.143+0000 7f2a7fe00640 1 -- start start 2024-04-20T10:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a80150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a8006e3b0 0x7f2a800cd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.144 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8006e830 0x7f2a800cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7fe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800d3e30 con 0x7f2a64004800 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a800b5ee0 con 0x7f2a8006e830 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a800d3120 con 0x7f2a8006e3b0 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a80150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a80150890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37578/0 (socket says 172.21.15.47:37578) 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7ea00640 1 -- 172.21.15.47:0/3373625112 learned_addr learned my addr 172.21.15.47:0/3373625112 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7ea00640 1 -- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a8006e3b0 msgr2=0x7f2a800cd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7ea00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a8006e3b0 0x7f2a800cd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.144+0000 7f2a7ea00640 1 -- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8006e830 msgr2=0x7f2a800cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a7ea00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8006e830 0x7f2a800cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.145 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a7ea00640 1 -- 172.21.15.47:0/3373625112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a800cc410 con 0x7f2a64004800 2024-04-20T10:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a7ea00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a80150890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d810 tx=0x7f2a7000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a77400640 1 -- 172.21.15.47:0/3373625112 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a70019070 con 0x7f2a64004800 2024-04-20T10:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a77400640 1 -- 172.21.15.47:0/3373625112 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a700074e0 con 0x7f2a64004800 2024-04-20T10:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a77400640 1 -- 172.21.15.47:0/3373625112 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a700049e0 con 0x7f2a64004800 2024-04-20T10:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a800cc6b0 con 0x7f2a64004800 2024-04-20T10:30:05.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.145+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a801599a0 con 0x7f2a64004800 2024-04-20T10:30:05.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.147+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a800ccbd0 con 0x7f2a64004800 2024-04-20T10:30:05.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.148+0000 7f2a77400640 1 -- 172.21.15.47:0/3373625112 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2a7001c070 con 0x7f2a64004800 2024-04-20T10:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.149+0000 7f2a77400640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.149+0000 7f2a77400640 1 -- 172.21.15.47:0/3373625112 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f2a7008b840 con 0x7f2a64004800 2024-04-20T10:30:05.149 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.150+0000 7f2a76000640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.151+0000 7f2a76000640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800b9e0 tx=0x7f2a68032000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:05.153 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.154+0000 7f2a77400640 1 -- 172.21.15.47:0/3373625112 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2a800ccbd0 con 0x7f2a64004800 2024-04-20T10:30:05.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.355+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f2a80075b30 con 0x7f2a64004800 2024-04-20T10:30:05.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.356+0000 7f2a77400640 1 -- 172.21.15.47:0/3373625112 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) ==== 93+0+10 (secure 0 0 0) 0x7f2a70059a80 con 0x7f2a64004800 2024-04-20T10:30:05.357 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:30:05.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2a5c06c8f0 msgr2=0x7f2a5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2a6800b9e0 tx=0x7f2a68032000 comp rx=0 tx=0).stop 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 msgr2=0x7f2a80150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a80150890 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000d810 tx=0x7f2a7000dcd0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 shutdown_connections 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a8006e830 0x7f2a800cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a8006e3b0 0x7f2a800cd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2a5c06c8f0 0x7f2a5c06edb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 --2- 172.21.15.47:0/3373625112 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2a64004800 0x7f2a80150890 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 >> 172.21.15.47:0/3373625112 conn(0x7f2a8009d6d0 msgr2=0x7f2a800b30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.359+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 shutdown_connections 2024-04-20T10:30:05.360 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.360+0000 7f2a7fe00640 1 -- 172.21.15.47:0/3373625112 wait complete. 2024-04-20T10:30:05.378 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:30:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.591+0000 7f207e600640 1 Processor -- start 2024-04-20T10:30:05.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.591+0000 7f207e600640 1 -- start start 2024-04-20T10:30:05.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.593+0000 7f207e600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2078162c00 0x7f20780d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.593+0000 7f207e600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207806df70 con 0x7f207806f090 2024-04-20T10:30:05.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.593+0000 7f207e600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f207806c720 con 0x7f207806fc30 2024-04-20T10:30:05.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.593+0000 7f207e600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2078162fe0 con 0x7f2078162c00 2024-04-20T10:30:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.594+0000 7f2077400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2078162c00 0x7f20780d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.594+0000 7f2077400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2078162c00 0x7f20780d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37588/0 (socket says 172.21.15.47:37588) 2024-04-20T10:30:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.594+0000 7f2077400640 1 -- 172.21.15.47:0/3621879279 learned_addr learned my addr 172.21.15.47:0/3621879279 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.594+0000 7f2077400640 1 -- 172.21.15.47:0/3621879279 >> v1:172.21.15.175:6789/0 conn(0x7f207806fc30 legacy=0x7f207806c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:05.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.594+0000 7f2077400640 1 -- 172.21.15.47:0/3621879279 >> v1:172.21.15.47:6789/0 conn(0x7f207806f090 legacy=0x7f207806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.594+0000 7f2077400640 1 -- 172.21.15.47:0/3621879279 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207806e310 con 0x7f2078162c00 2024-04-20T10:30:05.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.595+0000 7f2077400640 1 --2- 172.21.15.47:0/3621879279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f20780d0360 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f206802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be6a0d954fb92b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.596+0000 7f2076000640 1 -- 172.21.15.47:0/3621879279 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068035020 con 0x7f2078162c00 2024-04-20T10:30:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.596+0000 7f2076000640 1 -- 172.21.15.47:0/3621879279 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2068031870 con 0x7f2078162c00 2024-04-20T10:30:05.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.596+0000 7f2076000640 1 -- 172.21.15.47:0/3621879279 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2068031c40 con 0x7f2078162c00 2024-04-20T10:30:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.596+0000 7f207e600640 1 -- 172.21.15.47:0/3621879279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 msgr2=0x7f20780d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.596+0000 7f207e600640 1 --2- 172.21.15.47:0/3621879279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f20780d0360 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f2068009870 tx=0x7f206802b910 comp rx=0 tx=0).stop 2024-04-20T10:30:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.597+0000 7f207e600640 1 -- 172.21.15.47:0/3621879279 shutdown_connections 2024-04-20T10:30:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.597+0000 7f207e600640 1 --2- 172.21.15.47:0/3621879279 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f20780d0360 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.597+0000 7f207e600640 1 -- 172.21.15.47:0/3621879279 >> 172.21.15.47:0/3621879279 conn(0x7f20780b3350 msgr2=0x7f20780b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.597+0000 7f207e600640 1 -- 172.21.15.47:0/3621879279 shutdown_connections 2024-04-20T10:30:05.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.597+0000 7f207e600640 1 -- 172.21.15.47:0/3621879279 wait complete. 2024-04-20T10:30:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.598+0000 7f207e600640 1 Processor -- start 2024-04-20T10:30:05.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.599+0000 7f207e600640 1 -- start start 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.599+0000 7f207e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f207806f090 0x7f207815c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.599+0000 7f207e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f207806fc30 0x7f20781644f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f207e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f2078155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f207e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f207806e890 con 0x7f2078162c00 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f207e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f207806ec00 con 0x7f207806fc30 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f207e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f207806dbb0 con 0x7f207806f090 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f2078155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f2078155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37604/0 (socket says 172.21.15.47:37604) 2024-04-20T10:30:05.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 -- 172.21.15.47:0/582123520 learned_addr learned my addr 172.21.15.47:0/582123520 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:05.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2076a00640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f207806fc30 0x7f20781644f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077400640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f207806f090 0x7f207815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 -- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f207806f090 msgr2=0x7f207815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f207806f090 0x7f207815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 -- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f207806fc30 msgr2=0x7f20781644f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f207806fc30 0x7f20781644f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.600+0000 7f2077e00640 1 -- 172.21.15.47:0/582123520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2078163170 con 0x7f2078162c00 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.601+0000 7f2077e00640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f2078155420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f207000c440 tx=0x7f207000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.601+0000 7f206f400640 1 -- 172.21.15.47:0/582123520 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070011790 con 0x7f2078162c00 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.601+0000 7f206f400640 1 -- 172.21.15.47:0/582123520 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2070017070 con 0x7f2078162c00 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.601+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2078163450 con 0x7f2078162c00 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.601+0000 7f206f400640 1 -- 172.21.15.47:0/582123520 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2070011b70 con 0x7f2078162c00 2024-04-20T10:30:05.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.601+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2078163a10 con 0x7f2078162c00 2024-04-20T10:30:05.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.602+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20781641e0 con 0x7f2078162c00 2024-04-20T10:30:05.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.603+0000 7f206f400640 1 -- 172.21.15.47:0/582123520 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2070011d10 con 0x7f2078162c00 2024-04-20T10:30:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.604+0000 7f206f400640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f205406c8f0 0x7f205406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.604+0000 7f206f400640 1 -- 172.21.15.47:0/582123520 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f207008ba90 con 0x7f2078162c00 2024-04-20T10:30:05.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.605+0000 7f2077400640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f205406c8f0 0x7f205406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:05.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.606+0000 7f2077400640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f205406c8f0 0x7f205406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2068036000 tx=0x7f206803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:05.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.608+0000 7f206f400640 1 -- 172.21.15.47:0/582123520 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2070059cd0 con 0x7f2078162c00 2024-04-20T10:30:05.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.814+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2078068980 con 0x7f2078162c00 2024-04-20T10:30:05.814 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.814+0000 7f206f400640 1 -- 172.21.15.47:0/582123520 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) ==== 104+0+11 (secure 0 0 0) 0x7f20700595d0 con 0x7f2078162c00 2024-04-20T10:30:05.815 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:30:05.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f205406c8f0 msgr2=0x7f205406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f205406c8f0 0x7f205406edb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2068036000 tx=0x7f206803c040 comp rx=0 tx=0).stop 2024-04-20T10:30:05.817 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 msgr2=0x7f2078155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f2078155420 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f207000c440 tx=0x7f207000c900 comp rx=0 tx=0).stop 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 shutdown_connections 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f205406c8f0 0x7f205406edb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2078162c00 0x7f2078155420 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f207806fc30 0x7f20781644f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 --2- 172.21.15.47:0/582123520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f207806f090 0x7f207815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 >> 172.21.15.47:0/582123520 conn(0x7f20780b3350 msgr2=0x7f2078072560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 shutdown_connections 2024-04-20T10:30:05.818 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:05.817+0000 7f207e600640 1 -- 172.21.15.47:0/582123520 wait complete. 2024-04-20T10:30:05.838 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:30:06.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.056+0000 7feb73200640 1 Processor -- start 2024-04-20T10:30:06.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.056+0000 7feb73200640 1 -- start start 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb73200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feb7415e770 0x7feb740b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb73200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb74162210 0x7feb74162670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb73200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feb740d3590 con 0x7feb740b6400 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb73200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7feb7415eb50 con 0x7feb74162210 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb73200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7feb740d3870 con 0x7feb7415e770 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb71400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feb74162210 0x7feb74162670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb72800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7feb740b6400 0x7feb740d42c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56648/0 (socket says 172.21.15.47:56648) 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.058+0000 7feb72800640 1 -- 172.21.15.47:0/746617039 learned_addr learned my addr 172.21.15.47:0/746617039 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.059+0000 7feb71e00640 1 --2- 172.21.15.47:0/746617039 >> v2:172.21.15.175:3300/0 conn(0x7feb7415e770 0x7feb740b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.059+0000 7feb71400640 1 -- 172.21.15.47:0/746617039 >> v2:172.21.15.175:3300/0 conn(0x7feb7415e770 msgr2=0x7feb740b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:06.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.059+0000 7feb71400640 1 --2- 172.21.15.47:0/746617039 >> v2:172.21.15.175:3300/0 conn(0x7feb7415e770 0x7feb740b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.059+0000 7feb71400640 1 -- 172.21.15.47:0/746617039 >> v1:172.21.15.47:6789/0 conn(0x7feb740b6400 legacy=0x7feb740d42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.059+0000 7feb71400640 1 -- 172.21.15.47:0/746617039 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb740d3b50 con 0x7feb74162210 2024-04-20T10:30:06.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.060+0000 7feb71400640 1 --2- 172.21.15.47:0/746617039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb74162210 0x7feb74162670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7feb7408e630 tx=0x7feb6002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2a9f649fe487d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:06.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.060+0000 7feb70a00640 1 -- 172.21.15.47:0/746617039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb600315a0 con 0x7feb74162210 2024-04-20T10:30:06.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.060+0000 7feb70a00640 1 -- 172.21.15.47:0/746617039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb60037070 con 0x7feb74162210 2024-04-20T10:30:06.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.060+0000 7feb70a00640 1 -- 172.21.15.47:0/746617039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60031960 con 0x7feb74162210 2024-04-20T10:30:06.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 -- 172.21.15.47:0/746617039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb74162210 msgr2=0x7feb74162670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 --2- 172.21.15.47:0/746617039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb74162210 0x7feb74162670 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7feb7408e630 tx=0x7feb6002b840 comp rx=0 tx=0).stop 2024-04-20T10:30:06.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 -- 172.21.15.47:0/746617039 shutdown_connections 2024-04-20T10:30:06.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 --2- 172.21.15.47:0/746617039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb74162210 0x7feb74162670 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 --2- 172.21.15.47:0/746617039 >> v2:172.21.15.175:3300/0 conn(0x7feb7415e770 0x7feb740b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 -- 172.21.15.47:0/746617039 >> 172.21.15.47:0/746617039 conn(0x7feb7409d6d0 msgr2=0x7feb740b92f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:06.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 -- 172.21.15.47:0/746617039 shutdown_connections 2024-04-20T10:30:06.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.061+0000 7feb73200640 1 -- 172.21.15.47:0/746617039 wait complete. 2024-04-20T10:30:06.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.063+0000 7feb73200640 1 Processor -- start 2024-04-20T10:30:06.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.063+0000 7feb73200640 1 -- start start 2024-04-20T10:30:06.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb73200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb740b6400 0x7feb741f5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb73200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7415e770 0x7feb740cfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb73200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb74162210 0x7feb740a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb73200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feb740b39d0 con 0x7feb7415e770 2024-04-20T10:30:06.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb73200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb740d3e30 con 0x7feb740b6400 2024-04-20T10:30:06.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb73200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb740b3b50 con 0x7feb74162210 2024-04-20T10:30:06.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb71400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7415e770 0x7feb740cfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb71400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7415e770 0x7feb740cfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37636/0 (socket says 172.21.15.47:37636) 2024-04-20T10:30:06.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.064+0000 7feb71400640 1 -- 172.21.15.47:0/2191849449 learned_addr learned my addr 172.21.15.47:0/2191849449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:06.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb71e00640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb740b6400 0x7feb741f5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb72800640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb74162210 0x7feb740a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb71400640 1 -- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb74162210 msgr2=0x7feb740a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb71400640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb74162210 0x7feb740a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb71400640 1 -- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb740b6400 msgr2=0x7feb741f5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb71400640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb740b6400 0x7feb741f5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb71400640 1 -- 172.21.15.47:0/2191849449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb741f5750 con 0x7feb7415e770 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.065+0000 7feb71400640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7415e770 0x7feb740cfc30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feb60009890 tx=0x7feb600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.066+0000 7feb6a000640 1 -- 172.21.15.47:0/2191849449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60032840 con 0x7feb7415e770 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.066+0000 7feb6a000640 1 -- 172.21.15.47:0/2191849449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7feb600329e0 con 0x7feb7415e770 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.066+0000 7feb6a000640 1 -- 172.21.15.47:0/2191849449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb60032dd0 con 0x7feb7415e770 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.066+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb741602d0 con 0x7feb7415e770 2024-04-20T10:30:06.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.066+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb74160890 con 0x7feb7415e770 2024-04-20T10:30:06.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.067+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb74075b30 con 0x7feb7415e770 2024-04-20T10:30:06.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.068+0000 7feb6a000640 1 -- 172.21.15.47:0/2191849449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7feb6003b3f0 con 0x7feb7415e770 2024-04-20T10:30:06.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.069+0000 7feb6a000640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb5406c920 0x7feb5406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.069+0000 7feb6a000640 1 -- 172.21.15.47:0/2191849449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7feb6002f030 con 0x7feb7415e770 2024-04-20T10:30:06.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.070+0000 7feb71e00640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb5406c920 0x7feb5406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.071+0000 7feb71e00640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb5406c920 0x7feb5406ede0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb64006fd0 tx=0x7feb64008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:06.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.073+0000 7feb6a000640 1 -- 172.21.15.47:0/2191849449 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7feb741f5750 con 0x7feb7415e770 2024-04-20T10:30:06.276 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.276+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7feb740cb170 con 0x7feb7415e770 2024-04-20T10:30:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.277+0000 7feb6a000640 1 -- 172.21.15.47:0/2191849449 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) ==== 100+0+11 (secure 0 0 0) 0x7feb60040070 con 0x7feb7415e770 2024-04-20T10:30:06.278 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:30:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb5406c920 msgr2=0x7feb5406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb5406c920 0x7feb5406ede0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7feb64006fd0 tx=0x7feb64008040 comp rx=0 tx=0).stop 2024-04-20T10:30:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7415e770 msgr2=0x7feb740cfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7415e770 0x7feb740cfc30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7feb60009890 tx=0x7feb600046c0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 shutdown_connections 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb74162210 0x7feb740a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7feb5406c920 0x7feb5406ede0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feb7415e770 0x7feb740cfc30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 --2- 172.21.15.47:0/2191849449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb740b6400 0x7feb741f5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 >> 172.21.15.47:0/2191849449 conn(0x7feb7409d6d0 msgr2=0x7feb7415da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 shutdown_connections 2024-04-20T10:30:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.280+0000 7feb73200640 1 -- 172.21.15.47:0/2191849449 wait complete. 2024-04-20T10:30:06.300 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.512+0000 7f0e1a200640 1 Processor -- start 2024-04-20T10:30:06.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.512+0000 7f0e1a200640 1 -- start start 2024-04-20T10:30:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.514+0000 7f0e1a200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f0e1415a1f0 0x7f0e14072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.514+0000 7f0e1a200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1415cc90 con 0x7f0e14072190 2024-04-20T10:30:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.514+0000 7f0e1a200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0e1415cf70 con 0x7f0e14159e10 2024-04-20T10:30:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.514+0000 7f0e1a200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f0e14072900 con 0x7f0e1415a1f0 2024-04-20T10:30:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.514+0000 7f0e19800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f0e14072190 0x7f0e14073420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56650/0 (socket says 172.21.15.47:56650) 2024-04-20T10:30:06.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.514+0000 7f0e19800640 1 -- 172.21.15.47:0/155071227 learned_addr learned my addr 172.21.15.47:0/155071227 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.514+0000 7f0e13e00640 1 --2- 172.21.15.47:0/155071227 >> v2:172.21.15.175:3300/0 conn(0x7f0e1415a1f0 0x7f0e14072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 44494678 0 0) 0x7f0e1415cc90 con 0x7f0e14072190 2024-04-20T10:30:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0dfc003520 con 0x7f0e14072190 2024-04-20T10:30:06.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2417739147 0 0) 0x7f0dfc003520 con 0x7f0e14072190 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 >> v2:172.21.15.175:3300/0 conn(0x7f0e1415a1f0 msgr2=0x7f0e14072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 --2- 172.21.15.47:0/155071227 >> v2:172.21.15.175:3300/0 conn(0x7f0e1415a1f0 0x7f0e14072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 >> v1:172.21.15.175:6789/0 conn(0x7f0e14159e10 legacy=0x7f0e14071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e1415d250 con 0x7f0e14072190 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f0e0c0019f0 con 0x7f0e14072190 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 >> v1:172.21.15.47:6789/0 conn(0x7f0e14072190 legacy=0x7f0e14073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dfc003a00 0x7f0dfc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dfc0051c0 0x7f0dfc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc00c090 con 0x7f0dfc004800 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0dfc00c240 con 0x7f0dfc0051c0 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.515+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0dfc00c440 con 0x7f0dfc003a00 2024-04-20T10:30:06.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e13e00640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e19800640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dfc0051c0 0x7f0dfc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e18e00640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dfc003a00 0x7f0dfc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e13e00640 1 -- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dfc003a00 msgr2=0x7f0dfc003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e13e00640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dfc003a00 0x7f0dfc003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e13e00640 1 -- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dfc0051c0 msgr2=0x7f0dfc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e13e00640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dfc0051c0 0x7f0dfc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e13e00640 1 -- 172.21.15.47:0/155071227 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dfc003520 con 0x7f0dfc004800 2024-04-20T10:30:06.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.516+0000 7f0e19800640 1 -- 172.21.15.47:0/155071227 reap_dead start 2024-04-20T10:30:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.517+0000 7f0e13e00640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009a90 tx=0x7f0dfc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c7179964a3747b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00039070 con 0x7f0dfc004800 2024-04-20T10:30:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e0002c680 con 0x7f0dfc004800 2024-04-20T10:30:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e13400640 1 -- 172.21.15.47:0/155071227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00009690 con 0x7f0dfc004800 2024-04-20T10:30:06.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e1a200640 1 -- 172.21.15.47:0/155071227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 msgr2=0x7f0dfc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e1a200640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009a90 tx=0x7f0dfc005830 comp rx=0 tx=0).stop 2024-04-20T10:30:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e1a200640 1 -- 172.21.15.47:0/155071227 shutdown_connections 2024-04-20T10:30:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e1a200640 1 --2- 172.21.15.47:0/155071227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0dfc004c80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e1a200640 1 -- 172.21.15.47:0/155071227 >> 172.21.15.47:0/155071227 conn(0x7f0e140d0040 msgr2=0x7f0e140d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.518+0000 7f0e1a200640 1 -- 172.21.15.47:0/155071227 shutdown_connections 2024-04-20T10:30:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.519+0000 7f0e1a200640 1 -- 172.21.15.47:0/155071227 wait complete. 2024-04-20T10:30:06.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.520+0000 7f0e1a200640 1 Processor -- start 2024-04-20T10:30:06.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.521+0000 7f0e1a200640 1 -- start start 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.521+0000 7f0e1a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0e1409da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.521+0000 7f0e1a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e141499a0 0x7f0e1415c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e1a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1415c580 0x7f0e140cd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e1a200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1415d810 con 0x7f0dfc004800 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e1a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e1415ca40 con 0x7f0e141499a0 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e1a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e1406da90 con 0x7f0e1415c580 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0e1409da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0e1409da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37650/0 (socket says 172.21.15.47:37650) 2024-04-20T10:30:06.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 -- 172.21.15.47:0/3603774242 learned_addr learned my addr 172.21.15.47:0/3603774242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e13e00640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e141499a0 0x7f0e1415c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e19800640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1415c580 0x7f0e140cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 -- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1415c580 msgr2=0x7f0e140cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1415c580 0x7f0e140cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 -- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e141499a0 msgr2=0x7f0e1415c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e141499a0 0x7f0e1415c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.522+0000 7f0e18e00640 1 -- 172.21.15.47:0/3603774242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e140cfa40 con 0x7f0dfc004800 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.523+0000 7f0e18e00640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0e1409da10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d4b0 tx=0x7f0e0800d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.523+0000 7f0e10c00640 1 -- 172.21.15.47:0/3603774242 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08005480 con 0x7f0dfc004800 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.523+0000 7f0e10c00640 1 -- 172.21.15.47:0/3603774242 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e08017070 con 0x7f0dfc004800 2024-04-20T10:30:06.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.523+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e1415afa0 con 0x7f0dfc004800 2024-04-20T10:30:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.523+0000 7f0e10c00640 1 -- 172.21.15.47:0/3603774242 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e08003810 con 0x7f0dfc004800 2024-04-20T10:30:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.524+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e1415b560 con 0x7f0dfc004800 2024-04-20T10:30:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.525+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e14075b60 con 0x7f0dfc004800 2024-04-20T10:30:06.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.526+0000 7f0e10c00640 1 -- 172.21.15.47:0/3603774242 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0e080039b0 con 0x7f0dfc004800 2024-04-20T10:30:06.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.527+0000 7f0e10c00640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0df406c920 0x7f0df406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:06.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.527+0000 7f0e10c00640 1 -- 172.21.15.47:0/3603774242 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f0e0808bc10 con 0x7f0dfc004800 2024-04-20T10:30:06.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.528+0000 7f0e13e00640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0df406c920 0x7f0df406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:06.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.529+0000 7f0e13e00640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0df406c920 0x7f0df406ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0e00036000 tx=0x7f0e00030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:06.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.532+0000 7f0e10c00640 1 -- 172.21.15.47:0/3603774242 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f0e08059df0 con 0x7f0dfc004800 2024-04-20T10:30:06.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.778+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e14158040 con 0x7f0dfc004800 2024-04-20T10:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.779+0000 7f0e10c00640 1 -- 172.21.15.47:0/3603774242 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5207 (secure 0 0 0) 0x7f0e080596f0 con 0x7f0dfc004800 2024-04-20T10:30:06.779 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:06.779 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:55.499328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5475,"qdb_cluster":[5475]},"id":2}]} 2024-04-20T10:30:06.779 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0df406c920 msgr2=0x7f0df406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0df406c920 0x7f0df406ede0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0e00036000 tx=0x7f0e00030000 comp rx=0 tx=0).stop 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 msgr2=0x7f0e1409da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0e1409da10 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0e0800d4b0 tx=0x7f0e0800d970 comp rx=0 tx=0).stop 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 shutdown_connections 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1415c580 0x7f0e140cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e141499a0 0x7f0e1415c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f0df406c920 0x7f0df406ede0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 --2- 172.21.15.47:0/3603774242 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f0dfc004800 0x7f0e1409da10 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.781+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 >> 172.21.15.47:0/3603774242 conn(0x7f0e140d0040 msgr2=0x7f0e14154c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.782+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 shutdown_connections 2024-04-20T10:30:06.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:06.782+0000 7f0e1a200640 1 -- 172.21.15.47:0/3603774242 wait complete. 2024-04-20T10:30:06.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:07.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.007+0000 7fa865e00640 1 Processor -- start 2024-04-20T10:30:07.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.007+0000 7fa865e00640 1 -- start start 2024-04-20T10:30:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.008+0000 7fa865e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600b36c0 0x7fa8600d2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.008+0000 7fa865e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8600d26a0 con 0x7fa8600b1090 2024-04-20T10:30:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.008+0000 7fa865e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa8600d2980 con 0x7fa86015e5c0 2024-04-20T10:30:07.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.008+0000 7fa865e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8601624d0 con 0x7fa8600b36c0 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.009+0000 7fa865400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa8600b1090 0x7fa8600d4730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56660/0 (socket says 172.21.15.47:56660) 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.009+0000 7fa865400640 1 -- 172.21.15.47:0/1783286956 learned_addr learned my addr 172.21.15.47:0/1783286956 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.009+0000 7fa85fe00640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600b36c0 0x7fa8600d2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.009+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3731585601 0 0) 0x7fa8600d26a0 con 0x7fa8600b1090 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.009+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa844003520 con 0x7fa8600b1090 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 881908547 0 0) 0x7fa844003520 con 0x7fa8600b1090 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600b36c0 msgr2=0x7fa8600d2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600b36c0 0x7fa8600d2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 >> v1:172.21.15.175:6789/0 conn(0x7fa86015e5c0 legacy=0x7fa860162960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8600d2c60 con 0x7fa8600b1090 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fa8580019f0 con 0x7fa8600b1090 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 >> v1:172.21.15.47:6789/0 conn(0x7fa8600b1090 legacy=0x7fa8600d4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600b36c0 0x7fa8440039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa844003f60 0x7fa844004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844004d40 0x7fa8440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601624d0 con 0x7fa844003f60 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa84400bb50 con 0x7fa844004d40 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa84400bd50 con 0x7fa8600b36c0 2024-04-20T10:30:07.011 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85fe00640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa844003f60 0x7fa844004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85fe00640 1 -- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600b36c0 msgr2=0x7fa8440039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:07.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85fe00640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600b36c0 0x7fa8440039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85fe00640 1 -- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844004d40 msgr2=0x7fa8440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85fe00640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844004d40 0x7fa8440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.010+0000 7fa85fe00640 1 -- 172.21.15.47:0/1783286956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa844003520 con 0x7fa844003f60 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.011+0000 7fa85fe00640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa844003f60 0x7fa844004800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa848009ac0 tx=0x7fa844005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e2e2ebcbb46555f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848039070 con 0x7fa844003f60 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84802c710 con 0x7fa844003f60 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa85f400640 1 -- 172.21.15.47:0/1783286956 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848009690 con 0x7fa844003f60 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa865e00640 1 -- 172.21.15.47:0/1783286956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa844003f60 msgr2=0x7fa844004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa865e00640 1 --2- 172.21.15.47:0/1783286956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa844003f60 0x7fa844004800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fa848009ac0 tx=0x7fa844005830 comp rx=0 tx=0).stop 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa865400640 1 -- 172.21.15.47:0/1783286956 reap_dead start 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa865e00640 1 -- 172.21.15.47:0/1783286956 shutdown_connections 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.012+0000 7fa865e00640 1 -- 172.21.15.47:0/1783286956 >> 172.21.15.47:0/1783286956 conn(0x7fa86009d730 msgr2=0x7fa8600b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:07.013 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.013+0000 7fa865e00640 1 -- 172.21.15.47:0/1783286956 shutdown_connections 2024-04-20T10:30:07.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.013+0000 7fa865e00640 1 -- 172.21.15.47:0/1783286956 wait complete. 2024-04-20T10:30:07.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.014+0000 7fa865e00640 1 Processor -- start 2024-04-20T10:30:07.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.015+0000 7fa865e00640 1 -- start start 2024-04-20T10:30:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.015+0000 7fa865e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844003f60 0x7fa8600b5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.015+0000 7fa865e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa86006d7e0 0x7fa8600b6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.015+0000 7fa865e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600c7f70 0x7fa8600b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa865e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa8601631f0 con 0x7fa86006d7e0 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa865e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa860163070 con 0x7fa844003f60 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa865e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa86015e9a0 con 0x7fa8600c7f70 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa86006d7e0 0x7fa8600b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa86006d7e0 0x7fa8600b6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37664/0 (socket says 172.21.15.47:37664) 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 -- 172.21.15.47:0/855829834 learned_addr learned my addr 172.21.15.47:0/855829834 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa865400640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600c7f70 0x7fa8600b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa864a00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844003f60 0x7fa8600b5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 -- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600c7f70 msgr2=0x7fa8600b65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600c7f70 0x7fa8600b65c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 -- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844003f60 msgr2=0x7fa8600b5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844003f60 0x7fa8600b5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa85fe00640 1 -- 172.21.15.47:0/855829834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86006dca0 con 0x7fa86006d7e0 2024-04-20T10:30:07.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.016+0000 7fa864a00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844003f60 0x7fa8600b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.017+0000 7fa85fe00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa86006d7e0 0x7fa8600b6080 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa848000c00 tx=0x7fa848004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.017+0000 7fa85cc00640 1 -- 172.21.15.47:0/855829834 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848042020 con 0x7fa86006d7e0 2024-04-20T10:30:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.017+0000 7fa85cc00640 1 -- 172.21.15.47:0/855829834 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa84803a040 con 0x7fa86006d7e0 2024-04-20T10:30:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.017+0000 7fa85cc00640 1 -- 172.21.15.47:0/855829834 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa848039070 con 0x7fa86006d7e0 2024-04-20T10:30:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.017+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86006df80 con 0x7fa86006d7e0 2024-04-20T10:30:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.017+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8600b5340 con 0x7fa86006d7e0 2024-04-20T10:30:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.018+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa860075b60 con 0x7fa86006d7e0 2024-04-20T10:30:07.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.020+0000 7fa85cc00640 1 -- 172.21.15.47:0/855829834 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa848045020 con 0x7fa86006d7e0 2024-04-20T10:30:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.020+0000 7fa85cc00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.021+0000 7fa85cc00640 1 -- 172.21.15.47:0/855829834 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fa8480ad2d0 con 0x7fa86006d7e0 2024-04-20T10:30:07.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.021+0000 7fa864a00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.022+0000 7fa864a00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa83c06c8f0 0x7fa83c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8600d5280 tx=0x7fa850009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.025+0000 7fa85cc00640 1 -- 172.21.15.47:0/855829834 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa84804c070 con 0x7fa86006d7e0 2024-04-20T10:30:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.236+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8600b81b0 con 0x7fa86006d7e0 2024-04-20T10:30:07.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.240+0000 7fa85cc00640 1 -- 172.21.15.47:0/855829834 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17083 (secure 0 0 0) 0x7fa84807b480 con 0x7fa86006d7e0 2024-04-20T10:30:07.240 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:07.240 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:52.475429+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1013167846":"2024-04-21T10:29:52.475379+0000","172.21.15.47:6834/1013167846":"2024-04-21T10:29:52.475379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa83c06c8f0 msgr2=0x7fa83c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa83c06c8f0 0x7fa83c06edb0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa8600d5280 tx=0x7fa850009040 comp rx=0 tx=0).stop 2024-04-20T10:30:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa86006d7e0 msgr2=0x7fa8600b6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa86006d7e0 0x7fa8600b6080 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fa848000c00 tx=0x7fa848004730 comp rx=0 tx=0).stop 2024-04-20T10:30:07.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 shutdown_connections 2024-04-20T10:30:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa8600c7f70 0x7fa8600b65c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa83c06c8f0 0x7fa83c06edb0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa86006d7e0 0x7fa8600b6080 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 --2- 172.21.15.47:0/855829834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa844003f60 0x7fa8600b5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 >> 172.21.15.47:0/855829834 conn(0x7fa86009d730 msgr2=0x7fa8600d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 shutdown_connections 2024-04-20T10:30:07.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.243+0000 7fa865e00640 1 -- 172.21.15.47:0/855829834 wait complete. 2024-04-20T10:30:07.263 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T10:30:07.263 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.468+0000 7fe98ea00640 1 Processor -- start 2024-04-20T10:30:07.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.469+0000 7fe98ea00640 1 -- start start 2024-04-20T10:30:07.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.470+0000 7fe98ea00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe98806cde0 0x7fe9880d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.470+0000 7fe98ea00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98806c190 con 0x7fe9880b3330 2024-04-20T10:30:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.470+0000 7fe98ea00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe98806c470 con 0x7fe988161d50 2024-04-20T10:30:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.470+0000 7fe98ea00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe988162130 con 0x7fe98806cde0 2024-04-20T10:30:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.471+0000 7fe98cc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe98806cde0 0x7fe9880d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.471+0000 7fe98e000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe9880b3330 0x7fe9880b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56674/0 (socket says 172.21.15.47:56674) 2024-04-20T10:30:07.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.471+0000 7fe98e000640 1 -- 172.21.15.47:0/837220764 learned_addr learned my addr 172.21.15.47:0/837220764 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.471+0000 7fe98cc00640 1 -- 172.21.15.47:0/837220764 >> v1:172.21.15.175:6789/0 conn(0x7fe988161d50 legacy=0x7fe98806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.471+0000 7fe98cc00640 1 -- 172.21.15.47:0/837220764 >> v1:172.21.15.47:6789/0 conn(0x7fe9880b3330 legacy=0x7fe9880b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.471+0000 7fe98cc00640 1 -- 172.21.15.47:0/837220764 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9880d2e00 con 0x7fe98806cde0 2024-04-20T10:30:07.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.472+0000 7fe98cc00640 1 --2- 172.21.15.47:0/837220764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe98806cde0 0x7fe9880d38e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe97402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d3118b78ffde27 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.473+0000 7fe987e00640 1 -- 172.21.15.47:0/837220764 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974039070 con 0x7fe98806cde0 2024-04-20T10:30:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.473+0000 7fe987e00640 1 -- 172.21.15.47:0/837220764 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe97402cd00 con 0x7fe98806cde0 2024-04-20T10:30:07.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.473+0000 7fe987e00640 1 -- 172.21.15.47:0/837220764 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974034660 con 0x7fe98806cde0 2024-04-20T10:30:07.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.473+0000 7fe98ea00640 1 -- 172.21.15.47:0/837220764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe98806cde0 msgr2=0x7fe9880d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.473+0000 7fe98ea00640 1 --2- 172.21.15.47:0/837220764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe98806cde0 0x7fe9880d38e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe97402bf50 comp rx=0 tx=0).stop 2024-04-20T10:30:07.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.474+0000 7fe98ea00640 1 -- 172.21.15.47:0/837220764 shutdown_connections 2024-04-20T10:30:07.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.474+0000 7fe98ea00640 1 --2- 172.21.15.47:0/837220764 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe98806cde0 0x7fe9880d38e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.474+0000 7fe98ea00640 1 -- 172.21.15.47:0/837220764 >> 172.21.15.47:0/837220764 conn(0x7fe9880d4450 msgr2=0x7fe9880d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:07.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.474+0000 7fe98ea00640 1 -- 172.21.15.47:0/837220764 shutdown_connections 2024-04-20T10:30:07.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.474+0000 7fe98ea00640 1 -- 172.21.15.47:0/837220764 wait complete. 2024-04-20T10:30:07.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.476+0000 7fe98ea00640 1 Processor -- start 2024-04-20T10:30:07.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.476+0000 7fe98ea00640 1 -- start start 2024-04-20T10:30:07.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.477+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe98806cde0 0x7fe98815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.477+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9880b3330 0x7fe9880ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.477+0000 7fe98ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe988161d50 0x7fe9880cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.477+0000 7fe98ea00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe9880b3880 con 0x7fe9880b3330 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.477+0000 7fe98ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9880d33c0 con 0x7fe988161d50 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.477+0000 7fe98ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9880b3a00 con 0x7fe98806cde0 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9880b3330 0x7fe9880ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9880b3330 0x7fe9880ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37676/0 (socket says 172.21.15.47:37676) 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 -- 172.21.15.47:0/1330111072 learned_addr learned my addr 172.21.15.47:0/1330111072 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98d600640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe98806cde0 0x7fe98815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98e000640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe988161d50 0x7fe9880cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 -- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe98806cde0 msgr2=0x7fe98815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe98806cde0 0x7fe98815b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 -- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe988161d50 msgr2=0x7fe9880cb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe988161d50 0x7fe9880cb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98cc00640 1 -- 172.21.15.47:0/1330111072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9880c7f60 con 0x7fe9880b3330 2024-04-20T10:30:07.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.478+0000 7fe98e000640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe988161d50 0x7fe9880cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:07.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.479+0000 7fe98cc00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9880b3330 0x7fe9880ca5e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.479+0000 7fe985600640 1 -- 172.21.15.47:0/1330111072 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe974039050 con 0x7fe9880b3330 2024-04-20T10:30:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.479+0000 7fe985600640 1 -- 172.21.15.47:0/1330111072 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe974045070 con 0x7fe9880b3330 2024-04-20T10:30:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.479+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9880cd4a0 con 0x7fe9880b3330 2024-04-20T10:30:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.479+0000 7fe985600640 1 -- 172.21.15.47:0/1330111072 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe97403ea50 con 0x7fe9880b3330 2024-04-20T10:30:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.479+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9880cda60 con 0x7fe9880b3330 2024-04-20T10:30:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.480+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9880c8130 con 0x7fe9880b3330 2024-04-20T10:30:07.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.482+0000 7fe985600640 1 -- 172.21.15.47:0/1330111072 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe97402cb80 con 0x7fe9880b3330 2024-04-20T10:30:07.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.483+0000 7fe985600640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe96806c8f0 0x7fe96806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.483+0000 7fe985600640 1 -- 172.21.15.47:0/1330111072 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fe9740aec50 con 0x7fe9880b3330 2024-04-20T10:30:07.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.484+0000 7fe98d600640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe96806c8f0 0x7fe96806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.485+0000 7fe98d600640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe96806c8f0 0x7fe96806edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe978007fd0 tx=0x7fe978009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.488+0000 7fe985600640 1 -- 172.21.15.47:0/1330111072 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe97407cfb0 con 0x7fe9880b3330 2024-04-20T10:30:07.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.735+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe9880d4960 con 0x7fe9880b3330 2024-04-20T10:30:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.737+0000 7fe985600640 1 -- 172.21.15.47:0/1330111072 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5207 (secure 0 0 0) 0x7fe97407c8b0 con 0x7fe9880b3330 2024-04-20T10:30:07.738 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:07.738 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:55.499328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5475,"qdb_cluster":[5475]},"id":2}]} 2024-04-20T10:30:07.738 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.739+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe96806c8f0 msgr2=0x7fe96806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.739+0000 7fe98ea00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe96806c8f0 0x7fe96806edb0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe978007fd0 tx=0x7fe978009040 comp rx=0 tx=0).stop 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.739+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9880b3330 msgr2=0x7fe9880ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.739+0000 7fe98ea00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9880b3330 0x7fe9880ca5e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe974009d90 tx=0x7fe974004a40 comp rx=0 tx=0).stop 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 shutdown_connections 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe988161d50 0x7fe9880cb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe96806c8f0 0x7fe96806edb0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9880b3330 0x7fe9880ca5e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 --2- 172.21.15.47:0/1330111072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe98806cde0 0x7fe98815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 >> 172.21.15.47:0/1330111072 conn(0x7fe9880d4450 msgr2=0x7fe9880b7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 shutdown_connections 2024-04-20T10:30:07.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.740+0000 7fe98ea00640 1 -- 172.21.15.47:0/1330111072 wait complete. 2024-04-20T10:30:07.760 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:07.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.967+0000 7f5efc800640 1 Processor -- start 2024-04-20T10:30:07.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.967+0000 7f5efc800640 1 -- start start 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.968+0000 7f5efc800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5ef415ca40 0x7f5ef40ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.968+0000 7f5efc800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef4141c60 con 0x7f5ef4156c20 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.969+0000 7f5efc800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5ef415cf90 con 0x7f5ef40ad8e0 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.969+0000 7f5efc800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5ef406c010 con 0x7f5ef415ca40 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.969+0000 7f5ef9e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f5ef4156c20 0x7f5ef406b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56688/0 (socket says 172.21.15.47:56688) 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.969+0000 7f5ef9e00640 1 -- 172.21.15.47:0/1742211621 learned_addr learned my addr 172.21.15.47:0/1742211621 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.969+0000 7f5ef9400640 1 --2- 172.21.15.47:0/1742211621 >> v2:172.21.15.175:3300/0 conn(0x7f5ef415ca40 0x7f5ef40ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.969+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1458252225 0 0) 0x7f5ef4141c60 con 0x7f5ef4156c20 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.969+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5edc003520 con 0x7f5ef4156c20 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 54360126 0 0) 0x7f5edc003520 con 0x7f5ef4156c20 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 >> v2:172.21.15.175:3300/0 conn(0x7f5ef415ca40 msgr2=0x7f5ef40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 --2- 172.21.15.47:0/1742211621 >> v2:172.21.15.175:3300/0 conn(0x7f5ef415ca40 0x7f5ef40ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 >> v1:172.21.15.175:6789/0 conn(0x7f5ef40ad8e0 legacy=0x7f5ef40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef415d270 con 0x7f5ef4156c20 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f5ee8001670 con 0x7f5ef4156c20 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 >> v1:172.21.15.47:6789/0 conn(0x7f5ef4156c20 legacy=0x7f5ef406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc003a00 0x7f5edc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5edc004800 0x7f5edc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005230 0x7f5edc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef406c010 con 0x7f5edc004800 2024-04-20T10:30:07.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5edc00c190 con 0x7f5edc005230 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5edc00c390 con 0x7f5edc003a00 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef9400640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5edc004800 0x7f5edc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef9400640 1 -- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc003a00 msgr2=0x7f5edc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef9400640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5edc003a00 0x7f5edc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5efa800640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005230 0x7f5edc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.970+0000 7f5ef9400640 1 -- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005230 msgr2=0x7f5edc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.971+0000 7f5ef9400640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005230 0x7f5edc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.971+0000 7f5ef9400640 1 -- 172.21.15.47:0/1742211621 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5edc003520 con 0x7f5edc004800 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.971+0000 7f5efa800640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc005230 0x7f5edc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:07.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.971+0000 7f5efa800640 1 -- 172.21.15.47:0/1742211621 reap_dead start 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.972+0000 7f5ef9400640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5edc004800 0x7f5edc004c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400ba30 tx=0x7f5edc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e222ee19bdf399f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.972+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4036020 con 0x7f5edc004800 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.972+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee4032640 con 0x7f5edc004800 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.972+0000 7f5ef8a00640 1 -- 172.21.15.47:0/1742211621 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee400b600 con 0x7f5edc004800 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.973+0000 7f5efc800640 1 -- 172.21.15.47:0/1742211621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5edc004800 msgr2=0x7f5edc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.973+0000 7f5efc800640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5edc004800 0x7f5edc004c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5ee400ba30 tx=0x7f5edc005830 comp rx=0 tx=0).stop 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.973+0000 7f5efc800640 1 -- 172.21.15.47:0/1742211621 shutdown_connections 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.973+0000 7f5efc800640 1 --2- 172.21.15.47:0/1742211621 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5edc004800 0x7f5edc004c80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.973+0000 7f5efc800640 1 -- 172.21.15.47:0/1742211621 >> 172.21.15.47:0/1742211621 conn(0x7f5ef40b3350 msgr2=0x7f5ef40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.973+0000 7f5efc800640 1 -- 172.21.15.47:0/1742211621 shutdown_connections 2024-04-20T10:30:07.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.973+0000 7f5efc800640 1 -- 172.21.15.47:0/1742211621 wait complete. 2024-04-20T10:30:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.975+0000 7f5efc800640 1 Processor -- start 2024-04-20T10:30:07.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.975+0000 7f5efc800640 1 -- start start 2024-04-20T10:30:07.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc004800 0x7f5ef40afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef415f370 0x7f5ef40998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef4150ba0 0x7f5ef41f2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef40ad3a0 con 0x7f5ef415f370 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef415d830 con 0x7f5edc004800 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ef40ad520 con 0x7f5ef4150ba0 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef415f370 0x7f5ef40998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef415f370 0x7f5ef40998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37684/0 (socket says 172.21.15.47:37684) 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 -- 172.21.15.47:0/3616371449 learned_addr learned my addr 172.21.15.47:0/3616371449 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 -- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef4150ba0 msgr2=0x7f5ef41f2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef4150ba0 0x7f5ef41f2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 -- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc004800 msgr2=0x7f5ef40afd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc004800 0x7f5ef40afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:07.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 -- 172.21.15.47:0/3616371449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef4099df0 con 0x7f5ef415f370 2024-04-20T10:30:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5ef9400640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef415f370 0x7f5ef40998b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5ee402bf90 tx=0x7f5ee4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5eee000640 1 -- 172.21.15.47:0/3616371449 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4041070 con 0x7f5ef415f370 2024-04-20T10:30:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5eee000640 1 -- 172.21.15.47:0/3616371449 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ee4030a50 con 0x7f5ef415f370 2024-04-20T10:30:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5eee000640 1 -- 172.21.15.47:0/3616371449 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee4036040 con 0x7f5ef415f370 2024-04-20T10:30:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef409dcf0 con 0x7f5ef415f370 2024-04-20T10:30:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.977+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef41500c0 con 0x7f5ef415f370 2024-04-20T10:30:07.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.978+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef414feb0 con 0x7f5ef415f370 2024-04-20T10:30:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.979+0000 7f5eee000640 1 -- 172.21.15.47:0/3616371449 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ee404c060 con 0x7f5ef415f370 2024-04-20T10:30:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.980+0000 7f5eee000640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ed806c8f0 0x7f5ed806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:07.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.980+0000 7f5eee000640 1 -- 172.21.15.47:0/3616371449 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f5ee40adbf0 con 0x7f5ef415f370 2024-04-20T10:30:07.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.981+0000 7f5ef9e00640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ed806c8f0 0x7f5ed806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:07.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.983+0000 7f5ef9e00640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ed806c8f0 0x7f5ed806edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80078d0 tx=0x7f5ee8001540 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:07.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:07.985+0000 7f5eee000640 1 -- 172.21.15.47:0/3616371449 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5ee407bda0 con 0x7f5ef415f370 2024-04-20T10:30:08.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.228+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ef4068980 con 0x7f5ef415f370 2024-04-20T10:30:08.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.228+0000 7f5eee000640 1 -- 172.21.15.47:0/3616371449 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+5207 (secure 0 0 0) 0x7f5ee407b6a0 con 0x7f5ef415f370 2024-04-20T10:30:08.230 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:08.230 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":24,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:29:25.353181+0000","modified":"2024-04-20T10:29:55.499328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":43,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5475},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5475":{"gid":5475,"name":"c","rank":0,"incarnation":21,"state":"up:active","state_seq":14,"addr":"172.21.15.47:6833/2735464998","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":2735464998},{"type":"v1","addr":"172.21.15.47:6833","nonce":2735464998}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5475,"qdb_cluster":[5475]},"id":2}]} 2024-04-20T10:30:08.230 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 24 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ed806c8f0 msgr2=0x7f5ed806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ed806c8f0 0x7f5ed806edb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80078d0 tx=0x7f5ee8001540 comp rx=0 tx=0).stop 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef415f370 msgr2=0x7f5ef40998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef415f370 0x7f5ef40998b0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5ee402bf90 tx=0x7f5ee4004730 comp rx=0 tx=0).stop 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 shutdown_connections 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ef4150ba0 0x7f5ef41f2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ed806c8f0 0x7f5ed806edb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5ef415f370 0x7f5ef40998b0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 --2- 172.21.15.47:0/3616371449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5edc004800 0x7f5ef40afd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 >> 172.21.15.47:0/3616371449 conn(0x7f5ef40b3350 msgr2=0x7f5ef40d0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:08.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 shutdown_connections 2024-04-20T10:30:08.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.232+0000 7f5efc800640 1 -- 172.21.15.47:0/3616371449 wait complete. 2024-04-20T10:30:08.252 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:08.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.465+0000 7fa093000640 1 Processor -- start 2024-04-20T10:30:08.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.465+0000 7fa093000640 1 -- start start 2024-04-20T10:30:08.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.467+0000 7fa093000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c159e10 0x7fa08c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.467+0000 7fa093000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa08c15a1f0 0x7fa08c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.467+0000 7fa093000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa08c15cc90 con 0x7fa08c06da90 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.467+0000 7fa093000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa08c072900 con 0x7fa08c15a1f0 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.467+0000 7fa093000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c15ca40 con 0x7fa08c159e10 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.467+0000 7fa091200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa08c15a1f0 0x7fa08c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.468+0000 7fa091200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa08c15a1f0 0x7fa08c071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37698/0 (socket says 172.21.15.47:37698) 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.468+0000 7fa091200640 1 -- 172.21.15.47:0/2361468937 learned_addr learned my addr 172.21.15.47:0/2361468937 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:08.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.468+0000 7fa091c00640 1 --2- 172.21.15.47:0/2361468937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c159e10 0x7fa08c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.468+0000 7fa091200640 1 -- 172.21.15.47:0/2361468937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c159e10 msgr2=0x7fa08c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:08.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.468+0000 7fa091200640 1 --2- 172.21.15.47:0/2361468937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c159e10 0x7fa08c071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.468+0000 7fa091200640 1 -- 172.21.15.47:0/2361468937 >> v1:172.21.15.175:6789/0 conn(0x7fa08c06da90 legacy=0x7fa08c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:08.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.468+0000 7fa091200640 1 -- 172.21.15.47:0/2361468937 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c15cf70 con 0x7fa08c15a1f0 2024-04-20T10:30:08.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.469+0000 7fa091200640 1 --2- 172.21.15.47:0/2361468937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c15a1f0 0x7fa08c071fc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa08c15cbc0 tx=0x7fa07802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8a7d0d30061c4b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa08be00640 1 -- 172.21.15.47:0/2361468937 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa078035020 con 0x7fa08c15a1f0 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa08be00640 1 -- 172.21.15.47:0/2361468937 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa078031640 con 0x7fa08c15a1f0 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa08be00640 1 -- 172.21.15.47:0/2361468937 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07802d040 con 0x7fa08c15a1f0 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa093000640 1 -- 172.21.15.47:0/2361468937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c15a1f0 msgr2=0x7fa08c071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa093000640 1 --2- 172.21.15.47:0/2361468937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c15a1f0 0x7fa08c071fc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fa08c15cbc0 tx=0x7fa07802b8f0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa093000640 1 -- 172.21.15.47:0/2361468937 shutdown_connections 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa093000640 1 --2- 172.21.15.47:0/2361468937 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c15a1f0 0x7fa08c071fc0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa093000640 1 --2- 172.21.15.47:0/2361468937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c159e10 0x7fa08c071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.470+0000 7fa093000640 1 -- 172.21.15.47:0/2361468937 >> 172.21.15.47:0/2361468937 conn(0x7fa08c0d0040 msgr2=0x7fa08c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.471+0000 7fa093000640 1 -- 172.21.15.47:0/2361468937 shutdown_connections 2024-04-20T10:30:08.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.471+0000 7fa093000640 1 -- 172.21.15.47:0/2361468937 wait complete. 2024-04-20T10:30:08.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.472+0000 7fa093000640 1 Processor -- start 2024-04-20T10:30:08.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.472+0000 7fa093000640 1 -- start start 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.473+0000 7fa093000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c06da90 0x7fa08c157470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.473+0000 7fa093000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa08c159e10 0x7fa08c09d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.473+0000 7fa093000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c15a1f0 0x7fa08c1524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.473+0000 7fa093000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c15d810 con 0x7fa08c06da90 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.473+0000 7fa093000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa08c15d530 con 0x7fa08c159e10 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.473+0000 7fa093000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa08c06b900 con 0x7fa08c15a1f0 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c06da90 0x7fa08c157470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c06da90 0x7fa08c157470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37704/0 (socket says 172.21.15.47:37704) 2024-04-20T10:30:08.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 -- 172.21.15.47:0/1420276512 learned_addr learned my addr 172.21.15.47:0/1420276512 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091200640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa08c159e10 0x7fa08c09d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa092600640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c15a1f0 0x7fa08c1524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 -- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c15a1f0 msgr2=0x7fa08c1524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c15a1f0 0x7fa08c1524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 -- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa08c159e10 msgr2=0x7fa08c09d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa08c159e10 0x7fa08c09d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.474+0000 7fa091c00640 1 -- 172.21.15.47:0/1420276512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa08c15c510 con 0x7fa08c06da90 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.475+0000 7fa091c00640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c06da90 0x7fa08c157470 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa08000d800 tx=0x7fa08000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.475+0000 7fa089600640 1 -- 172.21.15.47:0/1420276512 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08000b6c0 con 0x7fa08c06da90 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.475+0000 7fa089600640 1 -- 172.21.15.47:0/1420276512 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa080019070 con 0x7fa08c06da90 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.475+0000 7fa089600640 1 -- 172.21.15.47:0/1420276512 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08000ba80 con 0x7fa08c06da90 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.475+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa08c15b110 con 0x7fa08c06da90 2024-04-20T10:30:08.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.475+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08c15b6d0 con 0x7fa08c06da90 2024-04-20T10:30:08.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.476+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa08c075b60 con 0x7fa08c06da90 2024-04-20T10:30:08.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.477+0000 7fa089600640 1 -- 172.21.15.47:0/1420276512 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa08000bc20 con 0x7fa08c06da90 2024-04-20T10:30:08.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.478+0000 7fa089600640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa06c06c8f0 0x7fa06c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.478+0000 7fa089600640 1 -- 172.21.15.47:0/1420276512 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7fa08008c230 con 0x7fa08c06da90 2024-04-20T10:30:08.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.478+0000 7fa091200640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa06c06c8f0 0x7fa06c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.480+0000 7fa091200640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa06c06c8f0 0x7fa06c06edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa078036000 tx=0x7fa07803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:08.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.482+0000 7fa089600640 1 -- 172.21.15.47:0/1420276512 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa08001c360 con 0x7fa08c06da90 2024-04-20T10:30:08.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.689+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa08c0ae300 con 0x7fa08c06da90 2024-04-20T10:30:08.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.692+0000 7fa089600640 1 -- 172.21.15.47:0/1420276512 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+17083 (secure 0 0 0) 0x7fa08005a470 con 0x7fa08c06da90 2024-04-20T10:30:08.693 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:08.693 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:29:52.475429+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:29:23.363565+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-20T10:29:24.330114+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6835/1013167846":"2024-04-21T10:29:52.475379+0000","172.21.15.47:6834/1013167846":"2024-04-21T10:29:52.475379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:08.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa06c06c8f0 msgr2=0x7fa06c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa06c06c8f0 0x7fa06c06edb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa078036000 tx=0x7fa07803c040 comp rx=0 tx=0).stop 2024-04-20T10:30:08.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c06da90 msgr2=0x7fa08c157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c06da90 0x7fa08c157470 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fa08000d800 tx=0x7fa08000dcc0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 shutdown_connections 2024-04-20T10:30:08.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa08c15a1f0 0x7fa08c1524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa08c159e10 0x7fa08c09d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa06c06c8f0 0x7fa06c06edb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 --2- 172.21.15.47:0/1420276512 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa08c06da90 0x7fa08c157470 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 >> 172.21.15.47:0/1420276512 conn(0x7fa08c0d0040 msgr2=0x7fa08c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:08.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 shutdown_connections 2024-04-20T10:30:08.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.695+0000 7fa093000640 1 -- 172.21.15.47:0/1420276512 wait complete. 2024-04-20T10:30:08.713 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T10:30:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.909+0000 7f9b92a00640 1 Processor -- start 2024-04-20T10:30:08.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.909+0000 7f9b92a00640 1 -- start start 2024-04-20T10:30:08.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.910+0000 7f9b92a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b8c159e10 0x7f9b8c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b92a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f9b8c06da90 0x7f9b8c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b92a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b8c15cc90 con 0x7f9b8c15a1f0 2024-04-20T10:30:08.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b92a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c072900 con 0x7f9b8c159e10 2024-04-20T10:30:08.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b92a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f9b8c15ca40 con 0x7f9b8c06da90 2024-04-20T10:30:08.911 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b91600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b8c159e10 0x7f9b8c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b91600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f9b8c159e10 0x7f9b8c071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37712/0 (socket says 172.21.15.47:37712) 2024-04-20T10:30:08.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b91600640 1 -- 172.21.15.47:0/67965254 learned_addr learned my addr 172.21.15.47:0/67965254 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:08.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b91600640 1 -- 172.21.15.47:0/67965254 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c06da90 msgr2=0x7f9b8c071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:08.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b91600640 1 --2- 172.21.15.47:0/67965254 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c06da90 0x7f9b8c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b91600640 1 -- 172.21.15.47:0/67965254 >> v1:172.21.15.175:6789/0 conn(0x7f9b8c15a1f0 legacy=0x7f9b8c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:08.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.911+0000 7f9b91600640 1 -- 172.21.15.47:0/67965254 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c15cf70 con 0x7f9b8c159e10 2024-04-20T10:30:08.912 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.912+0000 7f9b91600640 1 --2- 172.21.15.47:0/67965254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c071a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c050430 tx=0x7f9b8002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8329c03ac771d27e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:08.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.913+0000 7f9b8be00640 1 -- 172.21.15.47:0/67965254 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80035020 con 0x7f9b8c159e10 2024-04-20T10:30:08.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.913+0000 7f9b8be00640 1 -- 172.21.15.47:0/67965254 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b80031870 con 0x7f9b8c159e10 2024-04-20T10:30:08.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.913+0000 7f9b8be00640 1 -- 172.21.15.47:0/67965254 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80031c40 con 0x7f9b8c159e10 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.913+0000 7f9b92a00640 1 -- 172.21.15.47:0/67965254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 msgr2=0x7f9b8c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.913+0000 7f9b92a00640 1 --2- 172.21.15.47:0/67965254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c071a80 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f9b8c050430 tx=0x7f9b8002b910 comp rx=0 tx=0).stop 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.914+0000 7f9b92a00640 1 -- 172.21.15.47:0/67965254 shutdown_connections 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.914+0000 7f9b92a00640 1 --2- 172.21.15.47:0/67965254 >> v2:172.21.15.175:3300/0 conn(0x7f9b8c06da90 0x7f9b8c071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.914+0000 7f9b92a00640 1 --2- 172.21.15.47:0/67965254 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c071a80 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.914+0000 7f9b92a00640 1 -- 172.21.15.47:0/67965254 >> 172.21.15.47:0/67965254 conn(0x7f9b8c0d0040 msgr2=0x7f9b8c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.914+0000 7f9b92a00640 1 -- 172.21.15.47:0/67965254 shutdown_connections 2024-04-20T10:30:08.914 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.914+0000 7f9b92a00640 1 -- 172.21.15.47:0/67965254 wait complete. 2024-04-20T10:30:08.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.915+0000 7f9b92a00640 1 Processor -- start 2024-04-20T10:30:08.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.916+0000 7f9b92a00640 1 -- start start 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b92a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c06da90 0x7f9b8c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b92a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b92a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c15a1f0 0x7f9b8c0b0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b92a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c15d810 con 0x7f9b8c159e10 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b92a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9b8c15d530 con 0x7f9b8c06da90 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b92a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9b8c06b900 con 0x7f9b8c15a1f0 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c154f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37724/0 (socket says 172.21.15.47:37724) 2024-04-20T10:30:08.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 -- 172.21.15.47:0/3916726892 learned_addr learned my addr 172.21.15.47:0/3916726892 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:08.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 -- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c15a1f0 msgr2=0x7f9b8c0b0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c15a1f0 0x7f9b8c0b0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 -- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c06da90 msgr2=0x7f9b8c153b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:08.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c06da90 0x7f9b8c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.917+0000 7f9b90c00640 1 -- 172.21.15.47:0/3916726892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8c1557e0 con 0x7f9b8c159e10 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.918+0000 7f9b90c00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c154f90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b3c0 tx=0x7f9b7800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.918+0000 7f9b89600640 1 -- 172.21.15.47:0/3916726892 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011830 con 0x7f9b8c159e10 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.918+0000 7f9b89600640 1 -- 172.21.15.47:0/3916726892 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b78016070 con 0x7f9b8c159e10 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.918+0000 7f9b89600640 1 -- 172.21.15.47:0/3916726892 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b78011c30 con 0x7f9b8c159e10 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.918+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b8c155ac0 con 0x7f9b8c159e10 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.918+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b8c156080 con 0x7f9b8c159e10 2024-04-20T10:30:08.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.919+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b8c075b60 con 0x7f9b8c159e10 2024-04-20T10:30:08.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.920+0000 7f9b89600640 1 -- 172.21.15.47:0/3916726892 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f9b7801b460 con 0x7f9b8c159e10 2024-04-20T10:30:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.922+0000 7f9b89600640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9b6c06c8f0 0x7f9b6c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.922+0000 7f9b89600640 1 -- 172.21.15.47:0/3916726892 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 6187+0+0 (secure 0 0 0) 0x7f9b7808ba50 con 0x7f9b8c159e10 2024-04-20T10:30:08.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.922+0000 7f9b91600640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9b6c06c8f0 0x7f9b6c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:08.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.924+0000 7f9b91600640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9b6c06c8f0 0x7f9b6c06edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b80030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:08.927 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:08.927+0000 7f9b89600640 1 -- 172.21.15.47:0/3916726892 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f9b78059c90 con 0x7f9b8c159e10 2024-04-20T10:30:09.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.170+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f9b8c06c660 con 0x7f9b8c159e10 2024-04-20T10:30:09.172 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-20T10:30:09.172+0000 7f06e1000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:30:09.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.175+0000 7f9b89600640 1 -- 172.21.15.47:0/3916726892 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) ==== 161+0+0 (secure 0 0 0) 0x7f9b78059590 con 0x7f9b8c159e10 2024-04-20T10:30:09.176 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T10:30:09.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9b6c06c8f0 msgr2=0x7f9b6c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:09.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9b6c06c8f0 0x7f9b6c06edb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9b80036000 tx=0x7f9b80030000 comp rx=0 tx=0).stop 2024-04-20T10:30:09.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 msgr2=0x7f9b8c154f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:09.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c154f90 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b3c0 tx=0x7f9b7800b880 comp rx=0 tx=0).stop 2024-04-20T10:30:09.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 shutdown_connections 2024-04-20T10:30:09.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9b8c15a1f0 0x7f9b8c0b0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f9b6c06c8f0 0x7f9b6c06edb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f9b8c159e10 0x7f9b8c154f90 secure :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f9b7800b3c0 tx=0x7f9b7800b880 comp rx=0 tx=0).stop 2024-04-20T10:30:09.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 --2- 172.21.15.47:0/3916726892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c06da90 0x7f9b8c153b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 >> 172.21.15.47:0/3916726892 conn(0x7f9b8c0d0040 msgr2=0x7f9b8c06cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:09.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 shutdown_connections 2024-04-20T10:30:09.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.179+0000 7f9b92a00640 1 -- 172.21.15.47:0/3916726892 wait complete. 2024-04-20T10:30:09.186 INFO:tasks.ceph.mds.c.smithi047.stderr: -1336> 2024-04-20T10:29:56.736+0000 7f9ab9c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:30:09.187 INFO:tasks.ceph.mds.c.smithi047.stderr: -802> 2024-04-20T10:29:59.942+0000 7f9ab4c00640 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2024-04-20T10:30:09.199 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T10:30:09.245 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-20T10:30:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.372+0000 7f073b600640 1 Processor -- start 2024-04-20T10:30:09.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.373+0000 7f073b600640 1 -- start start 2024-04-20T10:30:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.374+0000 7f073b600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f07340d4270 0x7f07340d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.374+0000 7f073b600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f07340d0380 0x7f07340d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.374+0000 7f073b600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f07340d0d10 con 0x7f07340d4650 2024-04-20T10:30:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.374+0000 7f073b600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f07340d40c0 con 0x7f07340d4270 2024-04-20T10:30:09.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.374+0000 7f073b600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f07340d3820 con 0x7f07340d0380 2024-04-20T10:30:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.374+0000 7f0733e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f07340d4650 0x7f07340b1290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56694/0 (socket says 172.21.15.47:56694) 2024-04-20T10:30:09.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0733e00640 1 -- 172.21.15.47:0/503949039 learned_addr learned my addr 172.21.15.47:0/503949039 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0738c00640 1 --2- 172.21.15.47:0/503949039 >> v2:172.21.15.47:3300/0 conn(0x7f07340d4270 0x7f07340d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0739600640 1 --2- 172.21.15.47:0/503949039 >> v2:172.21.15.175:3300/0 conn(0x7f07340d0380 0x7f07340d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0733400640 1 -- 172.21.15.47:0/503949039 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1576989262 0 0) 0x7f07340d0d10 con 0x7f07340d4650 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0733400640 1 -- 172.21.15.47:0/503949039 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0718003520 con 0x7f07340d4650 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0738c00640 1 -- 172.21.15.47:0/503949039 >> v2:172.21.15.175:3300/0 conn(0x7f07340d0380 msgr2=0x7f07340d07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0738c00640 1 --2- 172.21.15.47:0/503949039 >> v2:172.21.15.175:3300/0 conn(0x7f07340d0380 0x7f07340d07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0738c00640 1 -- 172.21.15.47:0/503949039 >> v1:172.21.15.47:6789/0 conn(0x7f07340d4650 legacy=0x7f07340b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:09.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.375+0000 7f0738c00640 1 -- 172.21.15.47:0/503949039 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07340d3b00 con 0x7f07340d4270 2024-04-20T10:30:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.376+0000 7f0738c00640 1 --2- 172.21.15.47:0/503949039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4270 0x7f07340d1a00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0734068550 tx=0x7f072c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3555e5eb5f7cb295 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.376+0000 7f0733400640 1 -- 172.21.15.47:0/503949039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c039070 con 0x7f07340d4270 2024-04-20T10:30:09.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f0733400640 1 -- 172.21.15.47:0/503949039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f072c02cc70 con 0x7f07340d4270 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f0733400640 1 -- 172.21.15.47:0/503949039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c034640 con 0x7f07340d4270 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 -- 172.21.15.47:0/503949039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4270 msgr2=0x7f07340d1a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 --2- 172.21.15.47:0/503949039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4270 0x7f07340d1a00 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0734068550 tx=0x7f072c02bec0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 -- 172.21.15.47:0/503949039 shutdown_connections 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 --2- 172.21.15.47:0/503949039 >> v2:172.21.15.175:3300/0 conn(0x7f07340d0380 0x7f07340d07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 --2- 172.21.15.47:0/503949039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4270 0x7f07340d1a00 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 -- 172.21.15.47:0/503949039 >> 172.21.15.47:0/503949039 conn(0x7f07340b99c0 msgr2=0x7f07340b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 -- 172.21.15.47:0/503949039 shutdown_connections 2024-04-20T10:30:09.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.377+0000 7f073b600640 1 -- 172.21.15.47:0/503949039 wait complete. 2024-04-20T10:30:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.379+0000 7f073b600640 1 Processor -- start 2024-04-20T10:30:09.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.379+0000 7f073b600640 1 -- start start 2024-04-20T10:30:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f073b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07340d0380 0x7f073415bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f073b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07340d4270 0x7f07340b5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f073b600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4650 0x7f07340b5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:09.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f073b600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f07340b19a0 con 0x7f07340d4650 2024-04-20T10:30:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f073b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07340cfe60 con 0x7f07340d4270 2024-04-20T10:30:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f073b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07340d53b0 con 0x7f07340d0380 2024-04-20T10:30:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f0739600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4650 0x7f07340b5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f0739600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4650 0x7f07340b5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37778/0 (socket says 172.21.15.47:37778) 2024-04-20T10:30:09.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.380+0000 7f0739600640 1 -- 172.21.15.47:0/441191412 learned_addr learned my addr 172.21.15.47:0/441191412 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0733e00640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07340d4270 0x7f07340b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0739600640 1 -- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07340d0380 msgr2=0x7f073415bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0739600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07340d0380 0x7f073415bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0739600640 1 -- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07340d4270 msgr2=0x7f07340b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:09.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0739600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07340d4270 0x7f07340b5760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0739600640 1 -- 172.21.15.47:0/441191412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073415c2e0 con 0x7f07340d4650 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0739600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4650 0x7f07340b5ca0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f072400d5d0 tx=0x7f072400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0730c00640 1 -- 172.21.15.47:0/441191412 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07240125f0 con 0x7f07340d4650 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0730c00640 1 -- 172.21.15.47:0/441191412 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0724019070 con 0x7f07340d4650 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f0730c00640 1 -- 172.21.15.47:0/441191412 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07240129f0 con 0x7f07340d4650 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07341f4dc0 con 0x7f07340d4650 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.381+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07341f5380 con 0x7f07340d4650 2024-04-20T10:30:09.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.383+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0734068980 con 0x7f07340d4650 2024-04-20T10:30:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.384+0000 7f0730c00640 1 -- 172.21.15.47:0/441191412 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f0724012b90 con 0x7f07340d4650 2024-04-20T10:30:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.384+0000 7f0730c00640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f071006c8f0 0x7f071006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:09.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.384+0000 7f0730c00640 1 -- 172.21.15.47:0/441191412 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6273+0+0 (secure 0 0 0) 0x7f072408c170 con 0x7f07340d4650 2024-04-20T10:30:09.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.385+0000 7f0738c00640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f071006c8f0 0x7f071006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:09.386 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.386+0000 7f0738c00640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f071006c8f0 0x7f071006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f072c036000 tx=0x7f072c02e000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:09.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.389+0000 7f0730c00640 1 -- 172.21.15.47:0/441191412 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f072405a370 con 0x7f07340d4650 2024-04-20T10:30:09.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:09.623+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f07340b1330 con 0x7f07340d4650 2024-04-20T10:30:10.179 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.179+0000 7f0730c00640 1 -- 172.21.15.47:0/441191412 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) ==== 104+0+0 (secure 0 0 0) 0x7f0724060070 con 0x7f07340d4650 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f071006c8f0 msgr2=0x7f071006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f071006c8f0 0x7f071006edb0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f072c036000 tx=0x7f072c02e000 comp rx=0 tx=0).stop 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4650 msgr2=0x7f07340b5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4650 0x7f07340b5ca0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f072400d5d0 tx=0x7f072400da90 comp rx=0 tx=0).stop 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 shutdown_connections 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f071006c8f0 0x7f071006edb0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f07340d4650 0x7f07340b5ca0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07340d4270 0x7f07340b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 --2- 172.21.15.47:0/441191412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07340d0380 0x7f073415bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 >> 172.21.15.47:0/441191412 conn(0x7f07340b99c0 msgr2=0x7f07340b3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.183+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 shutdown_connections 2024-04-20T10:30:10.184 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.184+0000 7f073b600640 1 -- 172.21.15.47:0/441191412 wait complete. 2024-04-20T10:30:10.204 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T10:30:10.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.409+0000 7f549ca00640 1 Processor -- start 2024-04-20T10:30:10.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.409+0000 7f549ca00640 1 -- start start 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.411+0000 7f549ca00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54980ca740 0x7f54980acf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.411+0000 7f549ca00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f549806b9a0 con 0x7f54981629b0 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.411+0000 7f549ca00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f549806bc10 con 0x7f54980ad490 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.411+0000 7f549ca00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f549806c540 con 0x7f54980ca740 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.411+0000 7f5496a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f54980ca740 0x7f54980acf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.411+0000 7f5497400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f54981629b0 0x7f54980cf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56706/0 (socket says 172.21.15.47:56706) 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.411+0000 7f5497400640 1 -- 172.21.15.47:0/3334522873 learned_addr learned my addr 172.21.15.47:0/3334522873 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.412+0000 7f5496a00640 1 -- 172.21.15.47:0/3334522873 >> v1:172.21.15.175:6789/0 conn(0x7f54980ad490 legacy=0x7f54980cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.412+0000 7f5496a00640 1 -- 172.21.15.47:0/3334522873 >> v1:172.21.15.47:6789/0 conn(0x7f54981629b0 legacy=0x7f54980cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:10.412 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.412+0000 7f5496000640 1 -- 172.21.15.47:0/3334522873 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 383955765 0 0) 0x7f549806b9a0 con 0x7f54981629b0 2024-04-20T10:30:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.412+0000 7f5496a00640 1 -- 172.21.15.47:0/3334522873 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549806bef0 con 0x7f54980ca740 2024-04-20T10:30:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.413+0000 7f5496a00640 1 --2- 172.21.15.47:0/3334522873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f54980acf50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f54840097a0 tx=0x7f548402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa52d290bde6e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.413+0000 7f5496000640 1 -- 172.21.15.47:0/3334522873 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484035020 con 0x7f54980ca740 2024-04-20T10:30:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.413+0000 7f5496000640 1 -- 172.21.15.47:0/3334522873 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5484031830 con 0x7f54980ca740 2024-04-20T10:30:10.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.413+0000 7f5496000640 1 -- 172.21.15.47:0/3334522873 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5484031c00 con 0x7f54980ca740 2024-04-20T10:30:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.414+0000 7f549ca00640 1 -- 172.21.15.47:0/3334522873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 msgr2=0x7f54980acf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.414+0000 7f549ca00640 1 --2- 172.21.15.47:0/3334522873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f54980acf50 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f54840097a0 tx=0x7f548402b8d0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.414+0000 7f549ca00640 1 -- 172.21.15.47:0/3334522873 shutdown_connections 2024-04-20T10:30:10.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.414+0000 7f549ca00640 1 --2- 172.21.15.47:0/3334522873 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f54980acf50 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.414+0000 7f549ca00640 1 -- 172.21.15.47:0/3334522873 >> 172.21.15.47:0/3334522873 conn(0x7f54980cffa0 msgr2=0x7f54980d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:10.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.414+0000 7f549ca00640 1 -- 172.21.15.47:0/3334522873 shutdown_connections 2024-04-20T10:30:10.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.414+0000 7f549ca00640 1 -- 172.21.15.47:0/3334522873 wait complete. 2024-04-20T10:30:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.416+0000 7f549ca00640 1 Processor -- start 2024-04-20T10:30:10.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.416+0000 7f549ca00640 1 -- start start 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.417+0000 7f549ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980ad490 0x7f54980b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f549ca00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f5498165610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f549ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54981629b0 0x7f549806f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f549ca00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f54980cfd00 con 0x7f54980ca740 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f549ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5498162d90 con 0x7f54981629b0 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f549ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54980ce5b0 con 0x7f54980ad490 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f5498165610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f5498165610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37784/0 (socket says 172.21.15.47:37784) 2024-04-20T10:30:10.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 -- 172.21.15.47:0/3335164299 learned_addr learned my addr 172.21.15.47:0/3335164299 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:10.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5497400640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980ad490 0x7f54980b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5497e00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54981629b0 0x7f549806f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 -- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980ad490 msgr2=0x7f54980b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980ad490 0x7f54980b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 -- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54981629b0 msgr2=0x7f549806f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54981629b0 0x7f549806f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.418+0000 7f5496a00640 1 -- 172.21.15.47:0/3335164299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54980b4420 con 0x7f54980ca740 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.419+0000 7f5496a00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f5498165610 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f54840098c0 tx=0x7f54840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.419+0000 7f548f400640 1 -- 172.21.15.47:0/3335164299 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548403ba80 con 0x7f54980ca740 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.419+0000 7f548f400640 1 -- 172.21.15.47:0/3335164299 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f548403bc20 con 0x7f54980ca740 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.419+0000 7f548f400640 1 -- 172.21.15.47:0/3335164299 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54840453f0 con 0x7f54980ca740 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.419+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54980b46e0 con 0x7f54980ca740 2024-04-20T10:30:10.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.419+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54980b4ca0 con 0x7f54980ca740 2024-04-20T10:30:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.420+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5498075b30 con 0x7f54980ca740 2024-04-20T10:30:10.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.421+0000 7f548f400640 1 -- 172.21.15.47:0/3335164299 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5484038020 con 0x7f54980ca740 2024-04-20T10:30:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.422+0000 7f548f400640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f547806c8f0 0x7f547806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:10.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.422+0000 7f548f400640 1 -- 172.21.15.47:0/3335164299 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 6273+0+0 (secure 0 0 0) 0x7f54840adc50 con 0x7f54980ca740 2024-04-20T10:30:10.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.423+0000 7f5497400640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f547806c8f0 0x7f547806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:10.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.424+0000 7f5497400640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f547806c8f0 0x7f547806edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5488000f60 tx=0x7f5488007c00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:10.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.427+0000 7f548f400640 1 -- 172.21.15.47:0/3335164299 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5484040070 con 0x7f54980ca740 2024-04-20T10:30:10.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:10.645+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f54980d0590 con 0x7f54980ca740 2024-04-20T10:30:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.183+0000 7f548f400640 1 -- 172.21.15.47:0/3335164299 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) ==== 181+0+0 (secure 0 0 0) 0x7f548407bda0 con 0x7f54980ca740 2024-04-20T10:30:11.183 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-20T10:30:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.186+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f547806c8f0 msgr2=0x7f547806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.186+0000 7f549ca00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f547806c8f0 0x7f547806edb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5488000f60 tx=0x7f5488007c00 comp rx=0 tx=0).stop 2024-04-20T10:30:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.186+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 msgr2=0x7f5498165610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:11.186 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.186+0000 7f549ca00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f5498165610 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f54840098c0 tx=0x7f54840046c0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.186+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 shutdown_connections 2024-04-20T10:30:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.187+0000 7f549ca00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54981629b0 0x7f549806f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.187+0000 7f549ca00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f547806c8f0 0x7f547806edb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.187+0000 7f549ca00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f54980ca740 0x7f5498165610 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.187+0000 7f549ca00640 1 --2- 172.21.15.47:0/3335164299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54980ad490 0x7f54980b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.187+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 >> 172.21.15.47:0/3335164299 conn(0x7f54980cffa0 msgr2=0x7f549815c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:11.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.187+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 shutdown_connections 2024-04-20T10:30:11.188 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.187+0000 7f549ca00640 1 -- 172.21.15.47:0/3335164299 wait complete. 2024-04-20T10:30:11.215 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T10:30:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.417+0000 7fef23200640 1 Processor -- start 2024-04-20T10:30:11.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.418+0000 7fef23200640 1 -- start start 2024-04-20T10:30:11.419 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.419+0000 7fef23200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 0x7fef1c0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.419+0000 7fef23200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fef1c161ee0 0x7fef1c0cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.419+0000 7fef23200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef1c0d2e30 con 0x7fef1c0d4e40 2024-04-20T10:30:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.419+0000 7fef23200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fef1c0d3110 con 0x7fef1c161ee0 2024-04-20T10:30:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.419+0000 7fef23200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef1c0d33f0 con 0x7fef1c0d2460 2024-04-20T10:30:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.420+0000 7fef21400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fef1c0d4e40 0x7fef1c1617d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56722/0 (socket says 172.21.15.47:56722) 2024-04-20T10:30:11.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.420+0000 7fef21400640 1 -- 172.21.15.47:0/1050008231 learned_addr learned my addr 172.21.15.47:0/1050008231 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.420+0000 7fef22800640 1 --2- 172.21.15.47:0/1050008231 >> v2:172.21.15.47:3300/0 conn(0x7fef1c161ee0 0x7fef1c0cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.420+0000 7fef21e00640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 0x7fef1c0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.420+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375826640 0 0) 0x7fef1c0d2e30 con 0x7fef1c0d4e40 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.420+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef04003520 con 0x7fef1c0d4e40 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3342707419 0 0) 0x7fef04003520 con 0x7fef1c0d4e40 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 msgr2=0x7fef1c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 0x7fef1c0d4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 >> v2:172.21.15.47:3300/0 conn(0x7fef1c161ee0 msgr2=0x7fef1c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 --2- 172.21.15.47:0/1050008231 >> v2:172.21.15.47:3300/0 conn(0x7fef1c161ee0 0x7fef1c0cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c0d36d0 con 0x7fef1c0d4e40 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fef0c0019f0 con 0x7fef1c0d4e40 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef22800640 1 --2- 172.21.15.47:0/1050008231 >> v2:172.21.15.47:3300/0 conn(0x7fef1c161ee0 0x7fef1c0cfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 >> v1:172.21.15.47:6789/0 conn(0x7fef1c0d4e40 legacy=0x7fef1c1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 0x7fef040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003f80 0x7fef04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d40 0x7fef040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0d3110 con 0x7fef04004d40 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0d33f0 con 0x7fef04003f80 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef0400ba40 con 0x7fef1c0d2460 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef22800640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d40 0x7fef040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.421+0000 7fef21e00640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 0x7fef040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.422+0000 7fef22800640 1 -- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 msgr2=0x7fef040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.422+0000 7fef21400640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003f80 0x7fef04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.422+0000 7fef22800640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef1c0d2460 0x7fef040039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.422+0000 7fef22800640 1 -- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003f80 msgr2=0x7fef04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.422+0000 7fef22800640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003f80 0x7fef04004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.422+0000 7fef22800640 1 -- 172.21.15.47:0/1050008231 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef04003520 con 0x7fef04004d40 2024-04-20T10:30:11.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.422+0000 7fef21400640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef04003f80 0x7fef04004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.423+0000 7fef22800640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d40 0x7fef040051c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fef18009c90 tx=0x7fef04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f96d04c8e62a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.423+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18039070 con 0x7fef04004d40 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.423+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef1802c980 con 0x7fef04004d40 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.423+0000 7fef20a00640 1 -- 172.21.15.47:0/1050008231 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18009860 con 0x7fef04004d40 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.424+0000 7fef23200640 1 -- 172.21.15.47:0/1050008231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d40 msgr2=0x7fef040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.424+0000 7fef23200640 1 --2- 172.21.15.47:0/1050008231 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef04004d40 0x7fef040051c0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fef18009c90 tx=0x7fef04005830 comp rx=0 tx=0).stop 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.424+0000 7fef22800640 1 -- 172.21.15.47:0/1050008231 reap_dead start 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.424+0000 7fef23200640 1 -- 172.21.15.47:0/1050008231 shutdown_connections 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.424+0000 7fef23200640 1 -- 172.21.15.47:0/1050008231 >> 172.21.15.47:0/1050008231 conn(0x7fef1c0b7570 msgr2=0x7fef1c0b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.424+0000 7fef23200640 1 -- 172.21.15.47:0/1050008231 shutdown_connections 2024-04-20T10:30:11.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.424+0000 7fef23200640 1 -- 172.21.15.47:0/1050008231 wait complete. 2024-04-20T10:30:11.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.426+0000 7fef23200640 1 Processor -- start 2024-04-20T10:30:11.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.427+0000 7fef23200640 1 -- start start 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.428+0000 7fef23200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04004d40 0x7fef1c0c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.428+0000 7fef23200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1c0ca900 0x7fef1c0cad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.428+0000 7fef23200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef1c0b52d0 0x7fef1c0b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.428+0000 7fef23200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0d4650 con 0x7fef1c0ca900 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.428+0000 7fef23200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef1c0d44d0 con 0x7fef1c0b52d0 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.428+0000 7fef23200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef1c0d2840 con 0x7fef04004d40 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.428+0000 7fef21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1c0ca900 0x7fef1c0cad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1c0ca900 0x7fef1c0cad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37798/0 (socket says 172.21.15.47:37798) 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 -- 172.21.15.47:0/4110638620 learned_addr learned my addr 172.21.15.47:0/4110638620 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:11.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21e00640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04004d40 0x7fef1c0c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef22800640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef1c0b52d0 0x7fef1c0b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 -- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04004d40 msgr2=0x7fef1c0c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04004d40 0x7fef1c0c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 -- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef1c0b52d0 msgr2=0x7fef1c0b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef1c0b52d0 0x7fef1c0b5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:11.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 -- 172.21.15.47:0/4110638620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef1c0cb2a0 con 0x7fef1c0ca900 2024-04-20T10:30:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.429+0000 7fef21400640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1c0ca900 0x7fef1c0cad60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fef0c001b10 tx=0x7fef0c00b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.430+0000 7fef16000640 1 -- 172.21.15.47:0/4110638620 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c019070 con 0x7fef1c0ca900 2024-04-20T10:30:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.430+0000 7fef16000640 1 -- 172.21.15.47:0/4110638620 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef0c004d60 con 0x7fef1c0ca900 2024-04-20T10:30:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.430+0000 7fef16000640 1 -- 172.21.15.47:0/4110638620 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef0c0072f0 con 0x7fef1c0ca900 2024-04-20T10:30:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.430+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef1c0b6030 con 0x7fef1c0ca900 2024-04-20T10:30:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.430+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef1c0b65f0 con 0x7fef1c0ca900 2024-04-20T10:30:11.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.431+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef1c075b30 con 0x7fef1c0ca900 2024-04-20T10:30:11.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.432+0000 7fef16000640 1 -- 172.21.15.47:0/4110638620 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef0c01c070 con 0x7fef1c0ca900 2024-04-20T10:30:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.433+0000 7fef16000640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef0006c920 0x7fef0006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:11.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.434+0000 7fef16000640 1 -- 172.21.15.47:0/4110638620 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5835+0+0 (secure 0 0 0) 0x7fef0c08bee0 con 0x7fef1c0ca900 2024-04-20T10:30:11.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.434+0000 7fef21e00640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef0006c920 0x7fef0006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:11.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.435+0000 7fef21e00640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef0006c920 0x7fef0006ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef10008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:11.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.438+0000 7fef16000640 1 -- 172.21.15.47:0/4110638620 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef0c05a1f0 con 0x7fef1c0ca900 2024-04-20T10:30:11.642 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:11.642+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fef1c0d5530 con 0x7fef1c0ca900 2024-04-20T10:30:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.186+0000 7fef16000640 1 -- 172.21.15.47:0/4110638620 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) ==== 169+0+0 (secure 0 0 0) 0x7fef0c059af0 con 0x7fef1c0ca900 2024-04-20T10:30:12.187 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-20T10:30:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.191+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef0006c920 msgr2=0x7fef0006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.191+0000 7fef23200640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef0006c920 0x7fef0006ede0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fef10009990 tx=0x7fef10008040 comp rx=0 tx=0).stop 2024-04-20T10:30:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.191+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1c0ca900 msgr2=0x7fef1c0cad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.192 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.191+0000 7fef23200640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1c0ca900 0x7fef1c0cad60 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fef0c001b10 tx=0x7fef0c00b560 comp rx=0 tx=0).stop 2024-04-20T10:30:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.196+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 shutdown_connections 2024-04-20T10:30:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.196+0000 7fef23200640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef1c0b52d0 0x7fef1c0b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.196+0000 7fef23200640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef0006c920 0x7fef0006ede0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.196+0000 7fef23200640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef1c0ca900 0x7fef1c0cad60 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.196+0000 7fef23200640 1 --2- 172.21.15.47:0/4110638620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef04004d40 0x7fef1c0c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.196+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 >> 172.21.15.47:0/4110638620 conn(0x7fef1c0b7570 msgr2=0x7fef1c0b2d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.197+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 shutdown_connections 2024-04-20T10:30:12.197 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.197+0000 7fef23200640 1 -- 172.21.15.47:0/4110638620 wait complete. 2024-04-20T10:30:12.223 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T10:30:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.430+0000 7f65a7200640 1 Processor -- start 2024-04-20T10:30:12.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.430+0000 7f65a7200640 1 -- start start 2024-04-20T10:30:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.431+0000 7f65a7200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65a015d4f0 0x7f65a00ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.431+0000 7f65a7200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f65a006e060 0x7f65a006e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.431+0000 7f65a7200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f65a006b900 con 0x7f65a00aee60 2024-04-20T10:30:12.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.431+0000 7f65a7200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f65a006c510 con 0x7f65a015d4f0 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.431+0000 7f65a7200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f65a006bbe0 con 0x7f65a006e060 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a5400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f65a015d4f0 0x7f65a00ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a5e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f65a00aee60 0x7f65a00af240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:56726/0 (socket says 172.21.15.47:56726) 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a5e00640 1 -- 172.21.15.47:0/2014031706 learned_addr learned my addr 172.21.15.47:0/2014031706 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a6800640 1 --2- 172.21.15.47:0/2014031706 >> v2:172.21.15.175:3300/0 conn(0x7f65a006e060 0x7f65a006e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a5400640 1 -- 172.21.15.47:0/2014031706 >> v2:172.21.15.175:3300/0 conn(0x7f65a006e060 msgr2=0x7f65a006e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a5400640 1 --2- 172.21.15.47:0/2014031706 >> v2:172.21.15.175:3300/0 conn(0x7f65a006e060 0x7f65a006e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a5400640 1 -- 172.21.15.47:0/2014031706 >> v1:172.21.15.47:6789/0 conn(0x7f65a00aee60 legacy=0x7f65a00af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a5400640 1 -- 172.21.15.47:0/2014031706 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a006bec0 con 0x7f65a015d4f0 2024-04-20T10:30:12.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.432+0000 7f65a4a00640 1 -- 172.21.15.47:0/2014031706 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3902303094 0 0) 0x7f65a006b900 con 0x7f65a00aee60 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.434+0000 7f65a5400640 1 --2- 172.21.15.47:0/2014031706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a00ad0f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f65a00719a0 tx=0x7f659002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8bc1708264dace server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.434+0000 7f65a4a00640 1 -- 172.21.15.47:0/2014031706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590035020 con 0x7f65a015d4f0 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.434+0000 7f65a4a00640 1 -- 172.21.15.47:0/2014031706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6590031830 con 0x7f65a015d4f0 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.434+0000 7f65a4a00640 1 -- 172.21.15.47:0/2014031706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6590031c00 con 0x7f65a015d4f0 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.434+0000 7f65a7200640 1 -- 172.21.15.47:0/2014031706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 msgr2=0x7f65a00ad0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.434+0000 7f65a7200640 1 --2- 172.21.15.47:0/2014031706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a00ad0f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f65a00719a0 tx=0x7f659002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.435+0000 7f65a7200640 1 -- 172.21.15.47:0/2014031706 shutdown_connections 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.435+0000 7f65a7200640 1 --2- 172.21.15.47:0/2014031706 >> v2:172.21.15.175:3300/0 conn(0x7f65a006e060 0x7f65a006e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.435+0000 7f65a7200640 1 --2- 172.21.15.47:0/2014031706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a00ad0f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.435+0000 7f65a7200640 1 -- 172.21.15.47:0/2014031706 >> 172.21.15.47:0/2014031706 conn(0x7f65a00b5270 msgr2=0x7f65a00b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:12.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.435+0000 7f65a7200640 1 -- 172.21.15.47:0/2014031706 shutdown_connections 2024-04-20T10:30:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.435+0000 7f65a7200640 1 -- 172.21.15.47:0/2014031706 wait complete. 2024-04-20T10:30:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.436+0000 7f65a7200640 1 Processor -- start 2024-04-20T10:30:12.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.437+0000 7f65a7200640 1 -- start start 2024-04-20T10:30:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.437+0000 7f65a7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a006e060 0x7f65a015fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a00aee60 0x7f65a0098d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a0099250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f65a00ad630 con 0x7f65a015d4f0 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65a006c9a0 con 0x7f65a00aee60 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65a00ad7b0 con 0x7f65a006e060 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a0099250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a0099250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37812/0 (socket says 172.21.15.47:37812) 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 -- 172.21.15.47:0/1314113517 learned_addr learned my addr 172.21.15.47:0/1314113517 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a5400640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a00aee60 0x7f65a0098d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 -- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a006e060 msgr2=0x7f65a015fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a006e060 0x7f65a015fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 -- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a00aee60 msgr2=0x7f65a0098d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a00aee60 0x7f65a0098d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.438+0000 7f65a6800640 1 -- 172.21.15.47:0/1314113517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65a014cbb0 con 0x7f65a015d4f0 2024-04-20T10:30:12.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.439+0000 7f65a6800640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a0099250 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f659c007e60 tx=0x7f659c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.439+0000 7f659a000640 1 -- 172.21.15.47:0/1314113517 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c014420 con 0x7f65a015d4f0 2024-04-20T10:30:12.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.439+0000 7f659a000640 1 -- 172.21.15.47:0/1314113517 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f659c019070 con 0x7f65a015d4f0 2024-04-20T10:30:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.439+0000 7f659a000640 1 -- 172.21.15.47:0/1314113517 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c014820 con 0x7f65a015d4f0 2024-04-20T10:30:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.439+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65a01f28a0 con 0x7f65a015d4f0 2024-04-20T10:30:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.439+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65a01602f0 con 0x7f65a015d4f0 2024-04-20T10:30:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.440+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65a0075b30 con 0x7f65a015d4f0 2024-04-20T10:30:12.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.441+0000 7f659a000640 1 -- 172.21.15.47:0/1314113517 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f659c0149c0 con 0x7f65a015d4f0 2024-04-20T10:30:12.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.443+0000 7f659a000640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f658406c8f0 0x7f658406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.443+0000 7f659a000640 1 -- 172.21.15.47:0/1314113517 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f659c08c330 con 0x7f65a015d4f0 2024-04-20T10:30:12.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.444+0000 7f65a5e00640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f658406c8f0 0x7f658406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.445+0000 7f65a5e00640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f658406c8f0 0x7f658406edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6594007fd0 tx=0x7f6594007920 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:12.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.447+0000 7f659a000640 1 -- 172.21.15.47:0/1314113517 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f659c05a7e0 con 0x7f65a015d4f0 2024-04-20T10:30:12.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.652+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7f65a015c380 con 0x7f65a015d4f0 2024-04-20T10:30:12.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.659+0000 7f659a000640 1 -- 172.21.15.47:0/1314113517 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) ==== 104+0+0 (secure 0 0 0) 0x7f659c05a0e0 con 0x7f65a015d4f0 2024-04-20T10:30:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.661+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f658406c8f0 msgr2=0x7f658406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f658406c8f0 0x7f658406edb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6594007fd0 tx=0x7f6594007920 comp rx=0 tx=0).stop 2024-04-20T10:30:12.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 msgr2=0x7f65a0099250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a0099250 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f659c007e60 tx=0x7f659c00e4b0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 shutdown_connections 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f658406c8f0 0x7f658406edb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f65a015d4f0 0x7f65a0099250 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65a00aee60 0x7f65a0098d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 --2- 172.21.15.47:0/1314113517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65a006e060 0x7f65a015fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 >> 172.21.15.47:0/1314113517 conn(0x7f65a00b5270 msgr2=0x7f65a0152030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 shutdown_connections 2024-04-20T10:30:12.663 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.662+0000 7f65a7200640 1 -- 172.21.15.47:0/1314113517 wait complete. 2024-04-20T10:30:12.682 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair' 2024-04-20T10:30:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.889+0000 7f277a000640 1 Processor -- start 2024-04-20T10:30:12.889 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.889+0000 7f277a000640 1 -- start start 2024-04-20T10:30:12.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.890+0000 7f277a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2774156c20 0x7f277406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.890+0000 7f277a000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f277415ca40 0x7f27740ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.890+0000 7f277a000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2774141c60 con 0x7f27740ad8e0 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.890+0000 7f277a000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f277415cf90 con 0x7f277415ca40 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.890+0000 7f277a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f277415d270 con 0x7f2774156c20 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.891+0000 7f2773400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2774156c20 0x7f277406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.891+0000 7f2772a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f277415ca40 0x7f27740ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.891+0000 7f2773400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2774156c20 0x7f277406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:57528/0 (socket says 172.21.15.47:57528) 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.891+0000 7f2773400640 1 -- 172.21.15.47:0/958920271 learned_addr learned my addr 172.21.15.47:0/958920271 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:12.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.891+0000 7f2772a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f277415ca40 0x7f27740ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:45448/0 (socket says 172.21.15.47:45448) 2024-04-20T10:30:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.892+0000 7f2772a00640 1 -- 172.21.15.47:0/958920271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2774156c20 msgr2=0x7f277406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.892+0000 7f2772a00640 1 --2- 172.21.15.47:0/958920271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2774156c20 0x7f277406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.892+0000 7f2772a00640 1 -- 172.21.15.47:0/958920271 >> v1:172.21.15.175:6789/0 conn(0x7f27740ad8e0 legacy=0x7f27740d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.892+0000 7f2772a00640 1 -- 172.21.15.47:0/958920271 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f277415d550 con 0x7f277415ca40 2024-04-20T10:30:12.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.893+0000 7f2772a00640 1 --2- 172.21.15.47:0/958920271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 0x7f27740ad3a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f275c0097a0 tx=0x7f275c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b13a27fffd62f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.894+0000 7f2772000640 1 -- 172.21.15.47:0/958920271 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c035020 con 0x7f277415ca40 2024-04-20T10:30:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.894+0000 7f2772000640 1 -- 172.21.15.47:0/958920271 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c031810 con 0x7f277415ca40 2024-04-20T10:30:12.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.894+0000 7f2772000640 1 -- 172.21.15.47:0/958920271 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c031be0 con 0x7f277415ca40 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.894+0000 7f277a000640 1 -- 172.21.15.47:0/958920271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 msgr2=0x7f27740ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.895+0000 7f277a000640 1 --2- 172.21.15.47:0/958920271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 0x7f27740ad3a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f275c0097a0 tx=0x7f275c02b8b0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.895+0000 7f277a000640 1 -- 172.21.15.47:0/958920271 shutdown_connections 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.895+0000 7f277a000640 1 --2- 172.21.15.47:0/958920271 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 0x7f27740ad3a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.895+0000 7f277a000640 1 --2- 172.21.15.47:0/958920271 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2774156c20 0x7f277406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.895+0000 7f277a000640 1 -- 172.21.15.47:0/958920271 >> 172.21.15.47:0/958920271 conn(0x7f27740b3350 msgr2=0x7f27740b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.895+0000 7f277a000640 1 -- 172.21.15.47:0/958920271 shutdown_connections 2024-04-20T10:30:12.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.895+0000 7f277a000640 1 -- 172.21.15.47:0/958920271 wait complete. 2024-04-20T10:30:12.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.897+0000 7f277a000640 1 Processor -- start 2024-04-20T10:30:12.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.897+0000 7f277a000640 1 -- start start 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f277a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27740ad8e0 0x7f2774099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f277a000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2774156c20 0x7f27741e5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f277a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 0x7f27741e76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f277a000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f277406bfc0 con 0x7f2774156c20 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f277a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f277406be40 con 0x7f277415ca40 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f277a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27740d0a70 con 0x7f27740ad8e0 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f2772a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2774156c20 0x7f27741e5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f2772a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2774156c20 0x7f27741e5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37828/0 (socket says 172.21.15.47:37828) 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f2772a00640 1 -- 172.21.15.47:0/1706544366 learned_addr learned my addr 172.21.15.47:0/1706544366 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f2773400640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27740ad8e0 0x7f2774099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.898+0000 7f2773e00640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 0x7f27741e76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f2772a00640 1 -- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27740ad8e0 msgr2=0x7f2774099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f2772a00640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27740ad8e0 0x7f2774099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f2772a00640 1 -- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 msgr2=0x7f27741e76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f2772a00640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 0x7f27741e76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f2772a00640 1 -- 172.21.15.47:0/1706544366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2774099d50 con 0x7f2774156c20 2024-04-20T10:30:12.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f2772a00640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2774156c20 0x7f27741e5e50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f275c034040 tx=0x7f275c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f276b400640 1 -- 172.21.15.47:0/1706544366 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c040070 con 0x7f2774156c20 2024-04-20T10:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f276b400640 1 -- 172.21.15.47:0/1706544366 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f275c039c70 con 0x7f2774156c20 2024-04-20T10:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f276b400640 1 -- 172.21.15.47:0/1706544366 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f275c035040 con 0x7f2774156c20 2024-04-20T10:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.899+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27741e6390 con 0x7f2774156c20 2024-04-20T10:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.900+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27741e6910 con 0x7f2774156c20 2024-04-20T10:30:12.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.901+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2774068980 con 0x7f2774156c20 2024-04-20T10:30:12.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.902+0000 7f276b400640 1 -- 172.21.15.47:0/1706544366 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f275c038020 con 0x7f2774156c20 2024-04-20T10:30:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.903+0000 7f276b400640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f275006c920 0x7f275006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.903+0000 7f276b400640 1 -- 172.21.15.47:0/1706544366 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f275c0ad5d0 con 0x7f2774156c20 2024-04-20T10:30:12.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.904+0000 7f2773400640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f275006c920 0x7f275006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:12.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.905+0000 7f2773400640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f275006c920 0x7f275006ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2764009b60 tx=0x7f2764008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:12.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:12.908+0000 7f276b400640 1 -- 172.21.15.47:0/1706544366 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f275c07bb10 con 0x7f2774156c20 2024-04-20T10:30:13.196 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.195+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]} v 0) -- 0x7f27740cfd90 con 0x7f2774156c20 2024-04-20T10:30:13.202 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.202+0000 7f276b400640 1 -- 172.21.15.47:0/1706544366 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_backtrace_repair"]}]=0 v170) ==== 148+0+0 (secure 0 0 0) 0x7f275c07b410 con 0x7f2774156c20 2024-04-20T10:30:13.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f275006c920 msgr2=0x7f275006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.205 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f275006c920 0x7f275006ede0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2764009b60 tx=0x7f2764008040 comp rx=0 tx=0).stop 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2774156c20 msgr2=0x7f27741e5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2774156c20 0x7f27741e5e50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f275c034040 tx=0x7f275c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 shutdown_connections 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f277415ca40 0x7f27741e76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f275006c920 0x7f275006ede0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2774156c20 0x7f27741e5e50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 --2- 172.21.15.47:0/1706544366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27740ad8e0 0x7f2774099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.205+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 >> 172.21.15.47:0/1706544366 conn(0x7f27740b3350 msgr2=0x7f277415e270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.206+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 shutdown_connections 2024-04-20T10:30:13.206 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.206+0000 7f277a000640 1 -- 172.21.15.47:0/1706544366 wait complete. 2024-04-20T10:30:13.225 INFO:tasks.cephfs_test_runner:MDS should verify/fix backtrace on fetch dirfrag ... ok 2024-04-20T10:30:13.225 INFO:tasks.cephfs_test_runner:Starting test: test_mds_readonly (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) test if MDS behave correct when it's readonly 2024-04-20T10:30:13.225 INFO:tasks.cephfs_test_runner:test_mds_readonly (tasks.cephfs.test_auto_repair.TestMDSAutoRepair) 2024-04-20T10:30:13.226 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.430+0000 7f59f9a00640 1 Processor -- start 2024-04-20T10:30:13.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.430+0000 7f59f9a00640 1 -- start start 2024-04-20T10:30:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.431+0000 7f59f9a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59f40d4380 0x7f59f40d4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.431+0000 7f59f9a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f59f41621d0 0x7f59f41625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f9a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f59f40cf950 con 0x7f59f415e550 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f9a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f59f4162e90 con 0x7f59f40d4380 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f9a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f59f40cfbf0 con 0x7f59f41621d0 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59f40d4380 0x7f59f40d4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f59f40d4380 0x7f59f40d4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37834/0 (socket says 172.21.15.47:37834) 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f3400640 1 -- 172.21.15.47:0/1812670725 learned_addr learned my addr 172.21.15.47:0/1812670725 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f9000640 1 --2- 172.21.15.47:0/1812670725 >> v2:172.21.15.175:3300/0 conn(0x7f59f41621d0 0x7f59f41625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f3400640 1 -- 172.21.15.47:0/1812670725 >> v2:172.21.15.175:3300/0 conn(0x7f59f41621d0 msgr2=0x7f59f41625e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f3400640 1 --2- 172.21.15.47:0/1812670725 >> v2:172.21.15.175:3300/0 conn(0x7f59f41621d0 0x7f59f41625e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.432+0000 7f59f3400640 1 -- 172.21.15.47:0/1812670725 >> v1:172.21.15.175:6789/0 conn(0x7f59f415e550 legacy=0x7f59f40d06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:13.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.433+0000 7f59f3400640 1 -- 172.21.15.47:0/1812670725 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f40cfed0 con 0x7f59f40d4380 2024-04-20T10:30:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.434+0000 7f59f3400640 1 --2- 172.21.15.47:0/1812670725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f40d4380 0x7f59f40d4810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f59f40b9680 tx=0x7f59e002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0bcf2f92709531 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.434+0000 7f59f2a00640 1 -- 172.21.15.47:0/1812670725 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e00317c0 con 0x7f59f40d4380 2024-04-20T10:30:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.434+0000 7f59f2a00640 1 -- 172.21.15.47:0/1812670725 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59e0037070 con 0x7f59f40d4380 2024-04-20T10:30:13.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.434+0000 7f59f2a00640 1 -- 172.21.15.47:0/1812670725 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0031b80 con 0x7f59f40d4380 2024-04-20T10:30:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 -- 172.21.15.47:0/1812670725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f40d4380 msgr2=0x7f59f40d4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 --2- 172.21.15.47:0/1812670725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f40d4380 0x7f59f40d4810 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f59f40b9680 tx=0x7f59e002b860 comp rx=0 tx=0).stop 2024-04-20T10:30:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 -- 172.21.15.47:0/1812670725 shutdown_connections 2024-04-20T10:30:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 --2- 172.21.15.47:0/1812670725 >> v2:172.21.15.175:3300/0 conn(0x7f59f41621d0 0x7f59f41625e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 --2- 172.21.15.47:0/1812670725 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f40d4380 0x7f59f40d4810 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 -- 172.21.15.47:0/1812670725 >> 172.21.15.47:0/1812670725 conn(0x7f59f408bf80 msgr2=0x7f59f409d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 -- 172.21.15.47:0/1812670725 shutdown_connections 2024-04-20T10:30:13.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.435+0000 7f59f9a00640 1 -- 172.21.15.47:0/1812670725 wait complete. 2024-04-20T10:30:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.437+0000 7f59f9a00640 1 Processor -- start 2024-04-20T10:30:13.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.437+0000 7f59f9a00640 1 -- start start 2024-04-20T10:30:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.438+0000 7f59f9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40d4380 0x7f59f4154fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.438+0000 7f59f9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f415e550 0x7f59f40b3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.438+0000 7f59f9a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f41621d0 0x7f59f40b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.438+0000 7f59f9a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f59f415eab0 con 0x7f59f41621d0 2024-04-20T10:30:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.438+0000 7f59f9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f59f415e930 con 0x7f59f415e550 2024-04-20T10:30:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.438+0000 7f59f9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f59f40d1fc0 con 0x7f59f40d4380 2024-04-20T10:30:13.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f41621d0 0x7f59f40b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f41621d0 0x7f59f40b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:37840/0 (socket says 172.21.15.47:37840) 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 -- 172.21.15.47:0/3476035068 learned_addr learned my addr 172.21.15.47:0/3476035068 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f3400640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f415e550 0x7f59f40b3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f3e00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40d4380 0x7f59f4154fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 -- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40d4380 msgr2=0x7f59f4154fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40d4380 0x7f59f4154fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 -- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f415e550 msgr2=0x7f59f40b3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f415e550 0x7f59f40b3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.439+0000 7f59f9000640 1 -- 172.21.15.47:0/3476035068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f40cac20 con 0x7f59f41621d0 2024-04-20T10:30:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.440+0000 7f59f9000640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f41621d0 0x7f59f40b4390 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f59ec007e60 tx=0x7f59ec00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.440+0000 7f59ebe00640 1 -- 172.21.15.47:0/3476035068 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec014420 con 0x7f59f41621d0 2024-04-20T10:30:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.440+0000 7f59ebe00640 1 -- 172.21.15.47:0/3476035068 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f59ec019070 con 0x7f59f41621d0 2024-04-20T10:30:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.440+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f40c88a0 con 0x7f59f41621d0 2024-04-20T10:30:13.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.440+0000 7f59ebe00640 1 -- 172.21.15.47:0/3476035068 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec014820 con 0x7f59f41621d0 2024-04-20T10:30:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.440+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f40c8e60 con 0x7f59f41621d0 2024-04-20T10:30:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.441+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f4075b30 con 0x7f59f41621d0 2024-04-20T10:30:13.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.442+0000 7f59ebe00640 1 -- 172.21.15.47:0/3476035068 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f59ec0149c0 con 0x7f59f41621d0 2024-04-20T10:30:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.443+0000 7f59ebe00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f59d406c8f0 0x7f59d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.444+0000 7f59ebe00640 1 -- 172.21.15.47:0/3476035068 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f59ec08c330 con 0x7f59f41621d0 2024-04-20T10:30:13.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.444+0000 7f59f3e00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f59d406c8f0 0x7f59d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.445+0000 7f59f3e00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f59d406c8f0 0x7f59d406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59e4004ea0 tx=0x7f59e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:13.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.447+0000 7f59ebe00640 1 -- 172.21.15.47:0/3476035068 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f59ec05a7e0 con 0x7f59f41621d0 2024-04-20T10:30:13.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.653+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f59f40b9730 con 0x7f59f41621d0 2024-04-20T10:30:13.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.655+0000 7f59ebe00640 1 -- 172.21.15.47:0/3476035068 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+13435 (secure 0 0 0) 0x7f59ec05a0e0 con 0x7f59f41621d0 2024-04-20T10:30:13.655 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:13.655 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":46,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:12.183598+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6833/2735464998":"2024-04-21T10:30:09.172597+0000","172.21.15.47:6835/1013167846":"2024-04-21T10:29:52.475379+0000","172.21.15.47:6832/2735464998":"2024-04-21T10:30:09.172597+0000","172.21.15.47:6834/1013167846":"2024-04-21T10:29:52.475379+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f59d406c8f0 msgr2=0x7f59d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f59d406c8f0 0x7f59d406edb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f59e4004ea0 tx=0x7f59e4009040 comp rx=0 tx=0).stop 2024-04-20T10:30:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f41621d0 msgr2=0x7f59f40b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f41621d0 0x7f59f40b4390 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f59ec007e60 tx=0x7f59ec00e4b0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 shutdown_connections 2024-04-20T10:30:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f59d406c8f0 0x7f59d406edb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f59f41621d0 0x7f59f40b4390 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f59f415e550 0x7f59f40b3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 --2- 172.21.15.47:0/3476035068 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f59f40d4380 0x7f59f4154fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 >> 172.21.15.47:0/3476035068 conn(0x7f59f408bf80 msgr2=0x7f59f406eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 shutdown_connections 2024-04-20T10:30:13.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.658+0000 7f59f9a00640 1 -- 172.21.15.47:0/3476035068 wait complete. 2024-04-20T10:30:13.678 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:30:13.678 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:30:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.895+0000 7fc754e00640 1 Processor -- start 2024-04-20T10:30:13.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.896+0000 7fc754e00640 1 -- start start 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.897+0000 7fc754e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc7500d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.897+0000 7fc754e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7500d1c10 0x7fc7500d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.897+0000 7fc754e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc75015ed40 con 0x7fc7500b6120 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.897+0000 7fc754e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc7500b6e90 con 0x7fc7500d1c10 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.897+0000 7fc754e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7500d0e50 con 0x7fc75015e960 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc7500d1c10 0x7fc7500d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74e800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc7500b6120 0x7fc7500b65b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45454/0 (socket says 172.21.15.47:45454) 2024-04-20T10:30:13.898 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74e800640 1 -- 172.21.15.47:0/4106801507 learned_addr learned my addr 172.21.15.47:0/4106801507 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:13.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74de00640 1 --2- 172.21.15.47:0/4106801507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc7500d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74d400640 1 -- 172.21.15.47:0/4106801507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 msgr2=0x7fc7500d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:13.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74d400640 1 --2- 172.21.15.47:0/4106801507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc7500d4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74d400640 1 -- 172.21.15.47:0/4106801507 >> v1:172.21.15.47:6789/0 conn(0x7fc7500b6120 legacy=0x7fc7500b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.898+0000 7fc74d400640 1 -- 172.21.15.47:0/4106801507 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7500d1130 con 0x7fc7500d1c10 2024-04-20T10:30:13.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.900+0000 7fc74d400640 1 --2- 172.21.15.47:0/4106801507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500d1c10 0x7fc7500d4fe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc738009d90 tx=0x7fc73802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7dd2756511272e3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:13.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.900+0000 7fc74ca00640 1 -- 172.21.15.47:0/4106801507 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738039070 con 0x7fc7500d1c10 2024-04-20T10:30:13.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.900+0000 7fc74ca00640 1 -- 172.21.15.47:0/4106801507 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc73802cc60 con 0x7fc7500d1c10 2024-04-20T10:30:13.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.900+0000 7fc74ca00640 1 -- 172.21.15.47:0/4106801507 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc738034660 con 0x7fc7500d1c10 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 -- 172.21.15.47:0/4106801507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500d1c10 msgr2=0x7fc7500d4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 --2- 172.21.15.47:0/4106801507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500d1c10 0x7fc7500d4fe0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc738009d90 tx=0x7fc73802beb0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 -- 172.21.15.47:0/4106801507 shutdown_connections 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 --2- 172.21.15.47:0/4106801507 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500d1c10 0x7fc7500d4fe0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 --2- 172.21.15.47:0/4106801507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc7500d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 -- 172.21.15.47:0/4106801507 >> 172.21.15.47:0/4106801507 conn(0x7fc7500b96f0 msgr2=0x7fc7500b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 -- 172.21.15.47:0/4106801507 shutdown_connections 2024-04-20T10:30:13.901 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.901+0000 7fc754e00640 1 -- 172.21.15.47:0/4106801507 wait complete. 2024-04-20T10:30:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.903+0000 7fc754e00640 1 Processor -- start 2024-04-20T10:30:13.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.903+0000 7fc754e00640 1 -- start start 2024-04-20T10:30:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc754e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500b6120 0x7fc7501547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc754e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7500d1c10 0x7fc75009e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc754e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc75009e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc754e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500d3160 con 0x7fc7500b6120 2024-04-20T10:30:13.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc754e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7500d16f0 con 0x7fc7500d1c10 2024-04-20T10:30:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc754e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7500d32e0 con 0x7fc75015e960 2024-04-20T10:30:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc74de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500b6120 0x7fc7501547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc74de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500b6120 0x7fc7501547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55528/0 (socket says 172.21.15.47:55528) 2024-04-20T10:30:13.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.904+0000 7fc74de00640 1 -- 172.21.15.47:0/2343584324 learned_addr learned my addr 172.21.15.47:0/2343584324 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74de00640 1 -- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 msgr2=0x7fc75009e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74d400640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7500d1c10 0x7fc75009e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74e800640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc75009e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.906 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74de00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc75009e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74de00640 1 -- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7500d1c10 msgr2=0x7fc75009e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74de00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7500d1c10 0x7fc75009e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74de00640 1 -- 172.21.15.47:0/2343584324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7500a1e30 con 0x7fc7500b6120 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74e800640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc75009e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74d400640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7500d1c10 0x7fc75009e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc74de00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500b6120 0x7fc7501547b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc74000d5f0 tx=0x7fc74000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc746000640 1 -- 172.21.15.47:0/2343584324 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7400125f0 con 0x7fc7500b6120 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc746000640 1 -- 172.21.15.47:0/2343584324 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc740019070 con 0x7fc7500b6120 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc746000640 1 -- 172.21.15.47:0/2343584324 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7400129d0 con 0x7fc7500b6120 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.905+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7500a1b50 con 0x7fc7500b6120 2024-04-20T10:30:13.907 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.906+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7500cc490 con 0x7fc7500b6120 2024-04-20T10:30:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.907+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7500ce860 con 0x7fc7500b6120 2024-04-20T10:30:13.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.908+0000 7fc746000640 1 -- 172.21.15.47:0/2343584324 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc740012b70 con 0x7fc7500b6120 2024-04-20T10:30:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.909+0000 7fc746000640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc72c06c920 0x7fc72c06ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.909+0000 7fc746000640 1 -- 172.21.15.47:0/2343584324 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fc74008beb0 con 0x7fc7500b6120 2024-04-20T10:30:13.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.909+0000 7fc74d400640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc72c06c920 0x7fc72c06ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:13.910 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.911+0000 7fc74d400640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc72c06c920 0x7fc72c06ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc738036000 tx=0x7fc738030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:13.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:13.913+0000 7fc746000640 1 -- 172.21.15.47:0/2343584324 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc7500ce860 con 0x7fc7500b6120 2024-04-20T10:30:14.117 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.117+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fc750068980 con 0x7fc7500b6120 2024-04-20T10:30:14.118 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.118+0000 7fc746000640 1 -- 172.21.15.47:0/2343584324 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) ==== 93+0+10 (secure 0 0 0) 0x7fc74005a360 con 0x7fc7500b6120 2024-04-20T10:30:14.118 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:30:14.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc72c06c920 msgr2=0x7fc72c06ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc72c06c920 0x7fc72c06ede0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc738036000 tx=0x7fc738030000 comp rx=0 tx=0).stop 2024-04-20T10:30:14.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500b6120 msgr2=0x7fc7501547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500b6120 0x7fc7501547b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fc74000d5f0 tx=0x7fc74000dab0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 shutdown_connections 2024-04-20T10:30:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc75015e960 0x7fc75009e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7500d1c10 0x7fc75009e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc72c06c920 0x7fc72c06ede0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 --2- 172.21.15.47:0/2343584324 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc7500b6120 0x7fc7501547b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 >> 172.21.15.47:0/2343584324 conn(0x7fc7500b96f0 msgr2=0x7fc750160f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 shutdown_connections 2024-04-20T10:30:14.122 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.121+0000 7fc754e00640 1 -- 172.21.15.47:0/2343584324 wait complete. 2024-04-20T10:30:14.142 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly' 2024-04-20T10:30:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.344+0000 7fdc56c00640 1 Processor -- start 2024-04-20T10:30:14.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.344+0000 7fdc56c00640 1 -- start start 2024-04-20T10:30:14.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.346+0000 7fdc56c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdc5015ca40 0x7fdc500ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.346 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.346+0000 7fdc56c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc500ad8e0 0x7fdc500d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.346+0000 7fdc56c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fdc50141c60 con 0x7fdc50156c20 2024-04-20T10:30:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.346+0000 7fdc56c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fdc5006c010 con 0x7fdc5015ca40 2024-04-20T10:30:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.346+0000 7fdc56c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc5015cf90 con 0x7fdc500ad8e0 2024-04-20T10:30:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.347+0000 7fdc54c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc500ad8e0 0x7fdc500d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.347+0000 7fdc4f400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fdc5015ca40 0x7fdc500ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.347+0000 7fdc54c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc500ad8e0 0x7fdc500d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:38474/0 (socket says 172.21.15.47:38474) 2024-04-20T10:30:14.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.347+0000 7fdc54c00640 1 -- 172.21.15.47:0/2008802522 learned_addr learned my addr 172.21.15.47:0/2008802522 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:14.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.348+0000 7fdc4f400640 1 -- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc500ad8e0 msgr2=0x7fdc500d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:14.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.348+0000 7fdc4f400640 1 --2- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc500ad8e0 0x7fdc500d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.348+0000 7fdc4f400640 1 -- 172.21.15.47:0/2008802522 >> v1:172.21.15.175:6789/0 conn(0x7fdc50156c20 legacy=0x7fdc5006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.348+0000 7fdc4f400640 1 -- 172.21.15.47:0/2008802522 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5015d270 con 0x7fdc5015ca40 2024-04-20T10:30:14.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.348+0000 7fdc54c00640 1 --2- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc500ad8e0 0x7fdc500d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:14.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.350+0000 7fdc4f400640 1 --2- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5015ca40 0x7fdc500ad3a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0097a0 tx=0x7fdc3c02b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=741a371e79ca49a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:14.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.350+0000 7fdc4ea00640 1 -- 172.21.15.47:0/2008802522 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c035020 con 0x7fdc5015ca40 2024-04-20T10:30:14.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.350+0000 7fdc4ea00640 1 -- 172.21.15.47:0/2008802522 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc3c031810 con 0x7fdc5015ca40 2024-04-20T10:30:14.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.350+0000 7fdc4ea00640 1 -- 172.21.15.47:0/2008802522 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c031be0 con 0x7fdc5015ca40 2024-04-20T10:30:14.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.351+0000 7fdc56c00640 1 -- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5015ca40 msgr2=0x7fdc500ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.351+0000 7fdc56c00640 1 --2- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5015ca40 0x7fdc500ad3a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c0097a0 tx=0x7fdc3c02b8b0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.351+0000 7fdc56c00640 1 -- 172.21.15.47:0/2008802522 shutdown_connections 2024-04-20T10:30:14.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.351+0000 7fdc56c00640 1 --2- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc500ad8e0 0x7fdc500d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.351+0000 7fdc56c00640 1 --2- 172.21.15.47:0/2008802522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc5015ca40 0x7fdc500ad3a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.351+0000 7fdc56c00640 1 -- 172.21.15.47:0/2008802522 >> 172.21.15.47:0/2008802522 conn(0x7fdc500b3350 msgr2=0x7fdc500b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:14.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.352+0000 7fdc56c00640 1 -- 172.21.15.47:0/2008802522 shutdown_connections 2024-04-20T10:30:14.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.352+0000 7fdc56c00640 1 -- 172.21.15.47:0/2008802522 wait complete. 2024-04-20T10:30:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.354+0000 7fdc56c00640 1 Processor -- start 2024-04-20T10:30:14.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.354+0000 7fdc56c00640 1 -- start start 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.355+0000 7fdc56c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc500ad8e0 0x7fdc50153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.355+0000 7fdc56c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc50156c20 0x7fdc5009d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.355+0000 7fdc56c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc5015ca40 0x7fdc50073270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.355+0000 7fdc56c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fdc500d08a0 con 0x7fdc5015ca40 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.355+0000 7fdc56c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5015d830 con 0x7fdc500ad8e0 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.355+0000 7fdc56c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc500d0a20 con 0x7fdc50156c20 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc5015ca40 0x7fdc50073270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc5015ca40 0x7fdc50073270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55532/0 (socket says 172.21.15.47:55532) 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 -- 172.21.15.47:0/753325841 learned_addr learned my addr 172.21.15.47:0/753325841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc500ad8e0 0x7fdc50153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc4f400640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc50156c20 0x7fdc5009d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 -- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc50156c20 msgr2=0x7fdc5009d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc50156c20 0x7fdc5009d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 -- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc500ad8e0 msgr2=0x7fdc50153d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc500ad8e0 0x7fdc50153d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc54c00640 1 -- 172.21.15.47:0/753325841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc500cdcb0 con 0x7fdc5015ca40 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc4f400640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc50156c20 0x7fdc5009d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.356+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc500ad8e0 0x7fdc50153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.357+0000 7fdc54c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc5015ca40 0x7fdc50073270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e870 tx=0x7fdc4800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.357+0000 7fdc47e00640 1 -- 172.21.15.47:0/753325841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc48017070 con 0x7fdc5015ca40 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.357+0000 7fdc47e00640 1 -- 172.21.15.47:0/753325841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc48004020 con 0x7fdc5015ca40 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.357+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc50154310 con 0x7fdc5015ca40 2024-04-20T10:30:14.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.357+0000 7fdc47e00640 1 -- 172.21.15.47:0/753325841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc480043f0 con 0x7fdc5015ca40 2024-04-20T10:30:14.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.357+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc500cdf50 con 0x7fdc5015ca40 2024-04-20T10:30:14.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.358+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc50068980 con 0x7fdc5015ca40 2024-04-20T10:30:14.359 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.360+0000 7fdc47e00640 1 -- 172.21.15.47:0/753325841 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fdc4800c8e0 con 0x7fdc5015ca40 2024-04-20T10:30:14.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.361+0000 7fdc47e00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdc3006c920 0x7fdc3006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:14.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.361+0000 7fdc47e00640 1 -- 172.21.15.47:0/753325841 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fdc4808bb40 con 0x7fdc5015ca40 2024-04-20T10:30:14.361 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.362+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdc3006c920 0x7fdc3006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:14.363 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.363+0000 7fdc4fe00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdc3006c920 0x7fdc3006ede0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc40001e00 tx=0x7fdc40009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:14.365 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.366+0000 7fdc47e00640 1 -- 172.21.15.47:0/753325841 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fdc4805a140 con 0x7fdc5015ca40 2024-04-20T10:30:14.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:14.657+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]} v 0) -- 0x7fdc500cfd90 con 0x7fdc5015ca40 2024-04-20T10:30:15.215 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.215+0000 7fdc47e00640 1 -- 172.21.15.47:0/753325841 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]}]=0 v172) ==== 147+0+0 (secure 0 0 0) 0x7fdc48059a40 con 0x7fdc5015ca40 2024-04-20T10:30:15.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.218+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdc3006c920 msgr2=0x7fdc3006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.218+0000 7fdc56c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdc3006c920 0x7fdc3006ede0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc40001e00 tx=0x7fdc40009040 comp rx=0 tx=0).stop 2024-04-20T10:30:15.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.218+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc5015ca40 msgr2=0x7fdc50073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.218+0000 7fdc56c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc5015ca40 0x7fdc50073270 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800e870 tx=0x7fdc4800ed30 comp rx=0 tx=0).stop 2024-04-20T10:30:15.219 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 shutdown_connections 2024-04-20T10:30:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fdc3006c920 0x7fdc3006ede0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fdc5015ca40 0x7fdc50073270 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc50156c20 0x7fdc5009d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 --2- 172.21.15.47:0/753325841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc500ad8e0 0x7fdc50153d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 >> 172.21.15.47:0/753325841 conn(0x7fdc500b3350 msgr2=0x7fdc5015ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 shutdown_connections 2024-04-20T10:30:15.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.219+0000 7fdc56c00640 1 -- 172.21.15.47:0/753325841 wait complete. 2024-04-20T10:30:15.242 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-20T10:30:15.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.453+0000 7efd33e00640 1 Processor -- start 2024-04-20T10:30:15.454 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.454+0000 7efd33e00640 1 -- start start 2024-04-20T10:30:15.455 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.455+0000 7efd33e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd340d3b20 0x7efd340b3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.455+0000 7efd33e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efd3406c010 0x7efd340ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.455+0000 7efd33e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd340d0980 con 0x7efd34161a50 2024-04-20T10:30:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.455+0000 7efd33e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efd340d0c60 con 0x7efd340d3b20 2024-04-20T10:30:15.456 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.455+0000 7efd33e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7efd34162ae0 con 0x7efd3406c010 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd32000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd340d3b20 0x7efd340b3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd32000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efd340d3b20 0x7efd340b3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55546/0 (socket says 172.21.15.47:55546) 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd32000640 1 -- 172.21.15.47:0/2150603516 learned_addr learned my addr 172.21.15.47:0/2150603516 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd33400640 1 --2- 172.21.15.47:0/2150603516 >> v2:172.21.15.175:3300/0 conn(0x7efd3406c010 0x7efd340ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd32000640 1 -- 172.21.15.47:0/2150603516 >> v2:172.21.15.175:3300/0 conn(0x7efd3406c010 msgr2=0x7efd340ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd32000640 1 --2- 172.21.15.47:0/2150603516 >> v2:172.21.15.175:3300/0 conn(0x7efd3406c010 0x7efd340ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd32000640 1 -- 172.21.15.47:0/2150603516 >> v1:172.21.15.175:6789/0 conn(0x7efd34161a50 legacy=0x7efd3406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.457 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.456+0000 7efd32000640 1 -- 172.21.15.47:0/2150603516 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd34162490 con 0x7efd340d3b20 2024-04-20T10:30:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.458+0000 7efd32000640 1 --2- 172.21.15.47:0/2150603516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd340d3b20 0x7efd340b3180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009870 tx=0x7efd1c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6e2b9f5c2b6206e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.458+0000 7efd31600640 1 -- 172.21.15.47:0/2150603516 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c034070 con 0x7efd340d3b20 2024-04-20T10:30:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.458+0000 7efd31600640 1 -- 172.21.15.47:0/2150603516 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd1c030860 con 0x7efd340d3b20 2024-04-20T10:30:15.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.458+0000 7efd31600640 1 -- 172.21.15.47:0/2150603516 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c030c30 con 0x7efd340d3b20 2024-04-20T10:30:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 -- 172.21.15.47:0/2150603516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd340d3b20 msgr2=0x7efd340b3180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 --2- 172.21.15.47:0/2150603516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd340d3b20 0x7efd340b3180 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009870 tx=0x7efd1c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 -- 172.21.15.47:0/2150603516 shutdown_connections 2024-04-20T10:30:15.459 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 --2- 172.21.15.47:0/2150603516 >> v2:172.21.15.175:3300/0 conn(0x7efd3406c010 0x7efd340ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 --2- 172.21.15.47:0/2150603516 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd340d3b20 0x7efd340b3180 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 -- 172.21.15.47:0/2150603516 >> 172.21.15.47:0/2150603516 conn(0x7efd340d0ed0 msgr2=0x7efd340d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 -- 172.21.15.47:0/2150603516 shutdown_connections 2024-04-20T10:30:15.460 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.459+0000 7efd33e00640 1 -- 172.21.15.47:0/2150603516 wait complete. 2024-04-20T10:30:15.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.461+0000 7efd33e00640 1 Processor -- start 2024-04-20T10:30:15.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.461+0000 7efd33e00640 1 -- start start 2024-04-20T10:30:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.462+0000 7efd33e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd3406c010 0x7efd34158b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.462+0000 7efd33e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34161a50 0x7efd34159060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.462+0000 7efd33e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd340cd1c0 0x7efd340cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.462+0000 7efd33e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efd340d0600 con 0x7efd3406c010 2024-04-20T10:30:15.463 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.462+0000 7efd33e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd340d0480 con 0x7efd340cd1c0 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.462+0000 7efd33e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd34161e30 con 0x7efd34161a50 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd3406c010 0x7efd34158b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd3406c010 0x7efd34158b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55550/0 (socket says 172.21.15.47:55550) 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 -- 172.21.15.47:0/2763535434 learned_addr learned my addr 172.21.15.47:0/2763535434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd33400640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd340cd1c0 0x7efd340cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32000640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34161a50 0x7efd34159060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 -- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34161a50 msgr2=0x7efd34159060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34161a50 0x7efd34159060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 -- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd340cd1c0 msgr2=0x7efd340cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd340cd1c0 0x7efd340cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.464 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.463+0000 7efd32a00640 1 -- 172.21.15.47:0/2763535434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd341599a0 con 0x7efd3406c010 2024-04-20T10:30:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.464+0000 7efd32a00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd3406c010 0x7efd34158b20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efd2400c430 tx=0x7efd2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.464+0000 7efd2aa00640 1 -- 172.21.15.47:0/2763535434 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24017070 con 0x7efd3406c010 2024-04-20T10:30:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.464+0000 7efd2aa00640 1 -- 172.21.15.47:0/2763535434 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd240052d0 con 0x7efd3406c010 2024-04-20T10:30:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.464+0000 7efd2aa00640 1 -- 172.21.15.47:0/2763535434 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24003830 con 0x7efd3406c010 2024-04-20T10:30:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.464+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd34159c80 con 0x7efd3406c010 2024-04-20T10:30:15.465 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.464+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3415a240 con 0x7efd3406c010 2024-04-20T10:30:15.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.465+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd34075b30 con 0x7efd3406c010 2024-04-20T10:30:15.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.466+0000 7efd2aa00640 1 -- 172.21.15.47:0/2763535434 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efd2401a070 con 0x7efd3406c010 2024-04-20T10:30:15.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.467+0000 7efd2aa00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efd1006c8f0 0x7efd1006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:15.467 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.467+0000 7efd2aa00640 1 -- 172.21.15.47:0/2763535434 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7efd2408ae10 con 0x7efd3406c010 2024-04-20T10:30:15.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.468+0000 7efd32000640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efd1006c8f0 0x7efd1006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:15.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.469+0000 7efd32000640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efd1006c8f0 0x7efd1006edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efd1c036000 tx=0x7efd1c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:15.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.472+0000 7efd2aa00640 1 -- 172.21.15.47:0/2763535434 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efd2408e030 con 0x7efd3406c010 2024-04-20T10:30:15.672 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.671+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7efd340d3310 con 0x7efd3406c010 2024-04-20T10:30:15.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.678+0000 7efd2aa00640 1 -- 172.21.15.47:0/2763535434 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) ==== 122+0+0 (secure 0 0 0) 0x7efd240592c0 con 0x7efd3406c010 2024-04-20T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efd1006c8f0 msgr2=0x7efd1006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efd1006c8f0 0x7efd1006edb0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7efd1c036000 tx=0x7efd1c03b040 comp rx=0 tx=0).stop 2024-04-20T10:30:15.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd3406c010 msgr2=0x7efd34158b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd3406c010 0x7efd34158b20 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7efd2400c430 tx=0x7efd2400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 shutdown_connections 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd340cd1c0 0x7efd340cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd34161a50 0x7efd34159060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efd1006c8f0 0x7efd1006edb0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 --2- 172.21.15.47:0/2763535434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efd3406c010 0x7efd34158b20 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 >> 172.21.15.47:0/2763535434 conn(0x7efd340d0ed0 msgr2=0x7efd340b56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 shutdown_connections 2024-04-20T10:30:15.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:15.681+0000 7efd33e00640 1 -- 172.21.15.47:0/2763535434 wait complete. 2024-04-20T10:30:15.698 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T10:30:15.699 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:30:15.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:30:15.940 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T10:30:15.940 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:30:15.940 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:16.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T10:30:16.004 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:30:16.004 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:16.060 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:30:16.060 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:30:16.060 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:16.119 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T10:30:16.119 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:30:16.119 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:16.177 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:30:16.177 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-20T10:30:16.177 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:16.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.390+0000 7fef81400640 1 Processor -- start 2024-04-20T10:30:16.391 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.390+0000 7fef81400640 1 -- start start 2024-04-20T10:30:16.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.392+0000 7fef81400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fef7c06c010 0x7fef7c0adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.392+0000 7fef81400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c071a80 con 0x7fef7c0b2f70 2024-04-20T10:30:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.392+0000 7fef81400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fef7c071cf0 con 0x7fef7c0b4320 2024-04-20T10:30:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.392+0000 7fef81400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fef7c071fd0 con 0x7fef7c06c010 2024-04-20T10:30:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.392+0000 7fef7be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fef7c0b2f70 0x7fef7c15f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45464/0 (socket says 172.21.15.47:45464) 2024-04-20T10:30:16.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7be00640 1 -- 172.21.15.47:0/1667642412 learned_addr learned my addr 172.21.15.47:0/1667642412 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2422054286 0 0) 0x7fef7c071a80 con 0x7fef7c0b2f70 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fef60003520 con 0x7fef7c0b2f70 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef80a00640 1 --2- 172.21.15.47:0/1667642412 >> v2:172.21.15.175:3300/0 conn(0x7fef7c06c010 0x7fef7c0adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 170116981 0 0) 0x7fef60003520 con 0x7fef7c0b2f70 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 >> v2:172.21.15.175:3300/0 conn(0x7fef7c06c010 msgr2=0x7fef7c0adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 --2- 172.21.15.47:0/1667642412 >> v2:172.21.15.175:3300/0 conn(0x7fef7c06c010 0x7fef7c0adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 >> v1:172.21.15.175:6789/0 conn(0x7fef7c0b4320 legacy=0x7fef7c06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:16.396 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c0739c0 con 0x7fef7c0b2f70 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fef6c001670 con 0x7fef7c0b2f70 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.393+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 >> v1:172.21.15.47:6789/0 conn(0x7fef7c0b2f70 legacy=0x7fef7c15f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7b400640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef60003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7b400640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef60004800 0x7fef60004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7b400640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef60005210 0x7fef6000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c071fd0 con 0x7fef60003a00 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef6000c140 con 0x7fef60005210 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef6000c340 con 0x7fef60004800 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7be00640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef60003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef73400640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef60004800 0x7fef60004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7be00640 1 -- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef60004800 msgr2=0x7fef60004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7be00640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef60004800 0x7fef60004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef80a00640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef60005210 0x7fef6000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7be00640 1 -- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef60005210 msgr2=0x7fef6000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7be00640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef60005210 0x7fef6000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef7be00640 1 -- 172.21.15.47:0/1667642412 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef60003520 con 0x7fef60003a00 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef80a00640 1 -- 172.21.15.47:0/1667642412 reap_dead start 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.394+0000 7fef73400640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef60004800 0x7fef60004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.396+0000 7fef7be00640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef60003e00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c160 tx=0x7fef60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12282e054f2a725e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.396+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c007c60 con 0x7fef60003a00 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.396+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6c030070 con 0x7fef60003a00 2024-04-20T10:30:16.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.396+0000 7fef7b400640 1 -- 172.21.15.47:0/1667642412 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c005d50 con 0x7fef60003a00 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.396+0000 7fef81400640 1 -- 172.21.15.47:0/1667642412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 msgr2=0x7fef60003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.396+0000 7fef81400640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef60003e00 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c160 tx=0x7fef60005830 comp rx=0 tx=0).stop 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.397+0000 7fef81400640 1 -- 172.21.15.47:0/1667642412 shutdown_connections 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.397+0000 7fef81400640 1 --2- 172.21.15.47:0/1667642412 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef60003e00 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.397+0000 7fef81400640 1 -- 172.21.15.47:0/1667642412 >> 172.21.15.47:0/1667642412 conn(0x7fef7c0d0040 msgr2=0x7fef7c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.397+0000 7fef81400640 1 -- 172.21.15.47:0/1667642412 shutdown_connections 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.397+0000 7fef81400640 1 -- 172.21.15.47:0/1667642412 wait complete. 2024-04-20T10:30:16.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.399+0000 7fef81400640 1 Processor -- start 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.399+0000 7fef81400640 1 -- start start 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef81400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef7c1f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef81400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef7c157b20 0x7fef7c1e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef81400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef7c14c9d0 0x7fef7c1e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef81400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0ae0e0 con 0x7fef60003a00 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef81400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef7c0adf60 con 0x7fef7c14c9d0 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef81400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef7c0ae260 con 0x7fef7c157b20 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef7be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef7c1f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef7be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef7c1f4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55578/0 (socket says 172.21.15.47:55578) 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef7be00640 1 -- 172.21.15.47:0/1007833682 learned_addr learned my addr 172.21.15.47:0/1007833682 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.400+0000 7fef73400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef7c157b20 0x7fef7c1e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.402 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef80a00640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef7c14c9d0 0x7fef7c1e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef7be00640 1 -- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef7c157b20 msgr2=0x7fef7c1e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef7be00640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef7c157b20 0x7fef7c1e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef7be00640 1 -- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef7c14c9d0 msgr2=0x7fef7c1e57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef7be00640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef7c14c9d0 0x7fef7c1e57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef7be00640 1 -- 172.21.15.47:0/1007833682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef7c09bcb0 con 0x7fef60003a00 2024-04-20T10:30:16.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef7be00640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef7c1f4260 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c280 tx=0x7fef6c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef78c00640 1 -- 172.21.15.47:0/1007833682 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c007c60 con 0x7fef60003a00 2024-04-20T10:30:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef78c00640 1 -- 172.21.15.47:0/1007833682 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fef6c004bf0 con 0x7fef60003a00 2024-04-20T10:30:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef78c00640 1 -- 172.21.15.47:0/1007833682 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef6c007070 con 0x7fef60003a00 2024-04-20T10:30:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.401+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef7c09bf90 con 0x7fef60003a00 2024-04-20T10:30:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.402+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef7c09c550 con 0x7fef60003a00 2024-04-20T10:30:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.403+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef7c075b60 con 0x7fef60003a00 2024-04-20T10:30:16.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.404+0000 7fef78c00640 1 -- 172.21.15.47:0/1007833682 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fef6c048020 con 0x7fef60003a00 2024-04-20T10:30:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.405+0000 7fef78c00640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef5806c920 0x7fef5806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.405+0000 7fef78c00640 1 -- 172.21.15.47:0/1007833682 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7fef6c0accc0 con 0x7fef60003a00 2024-04-20T10:30:16.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.405+0000 7fef73400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef5806c920 0x7fef5806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.407+0000 7fef73400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef5806c920 0x7fef5806ede0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fef7c1584a0 tx=0x7fef64009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:16.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.409+0000 7fef78c00640 1 -- 172.21.15.47:0/1007833682 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fef6c0b3080 con 0x7fef60003a00 2024-04-20T10:30:16.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.659+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fef7c0d0630 con 0x7fef60003a00 2024-04-20T10:30:16.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.661+0000 7fef78c00640 1 -- 172.21.15.47:0/1007833682 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+3872 (secure 0 0 0) 0x7fef6c07b170 con 0x7fef60003a00 2024-04-20T10:30:16.661 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:16.661 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5092,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1291483994","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":1291483994},{"type":"v1","addr":"172.21.15.175:6835","nonce":1291483994}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5095,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1214219978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1214219978},{"type":"v1","addr":"172.21.15.175:6837","nonce":1214219978}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13},{"gid":5778,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3802401602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3802401602},{"type":"v1","addr":"172.21.15.47:6835","nonce":3802401602}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5961,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/866681622","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":866681622},{"type":"v1","addr":"172.21.15.47:6833","nonce":866681622}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":26}],"filesystems":[]} 2024-04-20T10:30:16.661 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 26 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.663+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef5806c920 msgr2=0x7fef5806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.663+0000 7fef81400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef5806c920 0x7fef5806ede0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fef7c1584a0 tx=0x7fef64009040 comp rx=0 tx=0).stop 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.663+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 msgr2=0x7fef7c1f4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.663+0000 7fef81400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef7c1f4260 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fef6c00c280 tx=0x7fef6c02e070 comp rx=0 tx=0).stop 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 shutdown_connections 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef7c14c9d0 0x7fef7c1e57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef7c157b20 0x7fef7c1e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fef5806c920 0x7fef5806ede0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 --2- 172.21.15.47:0/1007833682 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fef60003a00 0x7fef7c1f4260 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 >> 172.21.15.47:0/1007833682 conn(0x7fef7c0d0040 msgr2=0x7fef7c06efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:16.664 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 shutdown_connections 2024-04-20T10:30:16.665 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.664+0000 7fef81400640 1 -- 172.21.15.47:0/1007833682 wait complete. 2024-04-20T10:30:16.684 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T10:30:16.684 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-20T10:30:16.684 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T10:30:16.684 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:16.684 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T10:30:16.685 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-20T10:30:16.685 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T10:30:16.685 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:16.685 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:30:16.685 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-20T10:30:16.685 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T10:30:16.686 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:16.686 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T10:30:16.686 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-20T10:30:16.686 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T10:30:16.686 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:16.706 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T10:30:16.707 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T10:30:16.708 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T10:30:16.709 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T10:30:16.710 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T10:30:16.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T10:30:16.712 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T10:30:16.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:30:16.754 INFO:tasks.ceph.mds.c:Started 2024-04-20T10:30:16.754 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:16.757 INFO:tasks.ceph.mds.d:Started 2024-04-20T10:30:16.757 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:16.758 INFO:tasks.ceph.mds.a:Started 2024-04-20T10:30:16.759 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:16.761 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:30:16.761 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:16.761 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-20T10:30:16.904 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T10:30:16.904 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-20T10:30:16.906 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T10:30:16.908 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-20T10:30:16.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.942+0000 7f16c2c00640 1 Processor -- start 2024-04-20T10:30:16.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.943+0000 7f16c2c00640 1 -- start start 2024-04-20T10:30:16.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16c2c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f16bc0d0eb0 0x7f16bc0b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16c2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc06f560 0x7f16bc0b4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16c2c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f16bc15fe90 con 0x7f16bc15b320 2024-04-20T10:30:16.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16c2c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f16bc0cf1a0 con 0x7f16bc0d0eb0 2024-04-20T10:30:16.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16c2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc0cf480 con 0x7f16bc06f560 2024-04-20T10:30:16.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16b3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f16bc0d0eb0 0x7f16bc0b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16bbe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f16bc15b320 0x7f16bc06ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45490/0 (socket says 172.21.15.47:45490) 2024-04-20T10:30:16.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16bbe00640 1 -- 172.21.15.47:0/2066988314 learned_addr learned my addr 172.21.15.47:0/2066988314 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:16.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.944+0000 7f16c0c00640 1 --2- 172.21.15.47:0/2066988314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc06f560 0x7f16bc0b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.945+0000 7f16bb400640 1 -- 172.21.15.47:0/2066988314 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3459991144 0 0) 0x7f16bc15fe90 con 0x7f16bc15b320 2024-04-20T10:30:16.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.945+0000 7f16bb400640 1 -- 172.21.15.47:0/2066988314 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f16a0003520 con 0x7f16bc15b320 2024-04-20T10:30:16.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.945+0000 7f16b3400640 1 -- 172.21.15.47:0/2066988314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc06f560 msgr2=0x7f16bc0b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:16.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.945+0000 7f16b3400640 1 --2- 172.21.15.47:0/2066988314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc06f560 0x7f16bc0b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.945+0000 7f16b3400640 1 -- 172.21.15.47:0/2066988314 >> v1:172.21.15.47:6789/0 conn(0x7f16bc15b320 legacy=0x7f16bc06ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.945+0000 7f16b3400640 1 -- 172.21.15.47:0/2066988314 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc0cf760 con 0x7f16bc0d0eb0 2024-04-20T10:30:16.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.946+0000 7f16b3400640 1 --2- 172.21.15.47:0/2066988314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc0d0eb0 0x7f16bc0b4350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f16a40097a0 tx=0x7f16a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2b90780f85db4b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:16.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.946+0000 7f16bb400640 1 -- 172.21.15.47:0/2066988314 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a40315a0 con 0x7f16bc0d0eb0 2024-04-20T10:30:16.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.946+0000 7f16bb400640 1 -- 172.21.15.47:0/2066988314 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16a4037070 con 0x7f16bc0d0eb0 2024-04-20T10:30:16.947 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.946+0000 7f16bb400640 1 -- 172.21.15.47:0/2066988314 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4031960 con 0x7f16bc0d0eb0 2024-04-20T10:30:16.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 -- 172.21.15.47:0/2066988314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc0d0eb0 msgr2=0x7f16bc0b4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2066988314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc0d0eb0 0x7f16bc0b4350 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f16a40097a0 tx=0x7f16a402b840 comp rx=0 tx=0).stop 2024-04-20T10:30:16.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 -- 172.21.15.47:0/2066988314 shutdown_connections 2024-04-20T10:30:16.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2066988314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc06f560 0x7f16bc0b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2066988314 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc0d0eb0 0x7f16bc0b4350 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 -- 172.21.15.47:0/2066988314 >> 172.21.15.47:0/2066988314 conn(0x7f16bc070450 msgr2=0x7f16bc070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:16.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 -- 172.21.15.47:0/2066988314 shutdown_connections 2024-04-20T10:30:16.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.947+0000 7f16c2c00640 1 -- 172.21.15.47:0/2066988314 wait complete. 2024-04-20T10:30:16.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.949+0000 7f16c2c00640 1 Processor -- start 2024-04-20T10:30:16.949 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.949+0000 7f16c2c00640 1 -- start start 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c2c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16bc06f560 0x7f16bc0b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c2c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc15b320 0x7f16bc15a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c2c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc1466a0 0x7f16bc148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c2c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc15b880 con 0x7f16bc1466a0 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c2c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f16bc15b700 con 0x7f16bc06f560 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c2c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f16bc15f9b0 con 0x7f16bc15b320 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc1466a0 0x7f16bc148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.951 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc1466a0 0x7f16bc148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55616/0 (socket says 172.21.15.47:55616) 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16c0c00640 1 -- 172.21.15.47:0/2908285696 learned_addr learned my addr 172.21.15.47:0/2908285696 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16b3400640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc15b320 0x7f16bc15a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.950+0000 7f16bbe00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16bc06f560 0x7f16bc0b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16c0c00640 1 -- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc15b320 msgr2=0x7f16bc15a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16c0c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc15b320 0x7f16bc15a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16c0c00640 1 -- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16bc06f560 msgr2=0x7f16bc0b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16c0c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16bc06f560 0x7f16bc0b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16c0c00640 1 -- 172.21.15.47:0/2908285696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16bc0b1340 con 0x7f16bc1466a0 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16b3400640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc15b320 0x7f16bc15a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:16.952 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16c0c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc1466a0 0x7f16bc148f90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f16b400efc0 tx=0x7f16b400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16b8c00640 1 -- 172.21.15.47:0/2908285696 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4018020 con 0x7f16bc1466a0 2024-04-20T10:30:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16b8c00640 1 -- 172.21.15.47:0/2908285696 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f16b4014480 con 0x7f16bc1466a0 2024-04-20T10:30:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16bc0b1620 con 0x7f16bc1466a0 2024-04-20T10:30:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.951+0000 7f16b8c00640 1 -- 172.21.15.47:0/2908285696 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16b4009e60 con 0x7f16bc1466a0 2024-04-20T10:30:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.952+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16bc0b1be0 con 0x7f16bc1466a0 2024-04-20T10:30:16.953 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.953+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16bc068980 con 0x7f16bc1466a0 2024-04-20T10:30:16.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.954+0000 7f16b8c00640 1 -- 172.21.15.47:0/2908285696 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f16b4005d10 con 0x7f16bc1466a0 2024-04-20T10:30:16.955 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.955+0000 7f16b8c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f169806c8f0 0x7f169806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.955+0000 7f16b8c00640 1 -- 172.21.15.47:0/2908285696 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5421+0+0 (secure 0 0 0) 0x7f16b408c7d0 con 0x7f16bc1466a0 2024-04-20T10:30:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.956+0000 7f16bbe00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f169806c8f0 0x7f169806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:16.956 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.956+0000 7f16bbe00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f169806c8f0 0x7f169806edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac007cf0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:16.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:16.960+0000 7f16b8c00640 1 -- 172.21.15.47:0/2908285696 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f16b4022030 con 0x7f16bc1466a0 2024-04-20T10:30:17.181 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.181+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f16bc0b65a0 con 0x7f16bc1466a0 2024-04-20T10:30:17.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.240+0000 7f16b8c00640 1 -- 172.21.15.47:0/2908285696 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) ==== 97+0+0 (secure 0 0 0) 0x7f16b4010040 con 0x7f16bc1466a0 2024-04-20T10:30:17.241 INFO:teuthology.orchestra.run.smithi047.stderr: removed all blocklist entries 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.244+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f169806c8f0 msgr2=0x7f169806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.244+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f169806c8f0 0x7f169806edb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f16ac000c00 tx=0x7f16ac007cf0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.244+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc1466a0 msgr2=0x7f16bc148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.244+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc1466a0 0x7f16bc148f90 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f16b400efc0 tx=0x7f16b400c490 comp rx=0 tx=0).stop 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 shutdown_connections 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f169806c8f0 0x7f169806edb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f16bc1466a0 0x7f16bc148f90 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f16bc15b320 0x7f16bc15a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 --2- 172.21.15.47:0/2908285696 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f16bc06f560 0x7f16bc0b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 >> 172.21.15.47:0/2908285696 conn(0x7f16bc070450 msgr2=0x7f16bc06d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:17.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 shutdown_connections 2024-04-20T10:30:17.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.245+0000 7f16c2c00640 1 -- 172.21.15.47:0/2908285696 wait complete. 2024-04-20T10:30:17.267 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-20T10:30:17.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.466+0000 7f28cfa00640 1 Processor -- start 2024-04-20T10:30:17.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.466+0000 7f28cfa00640 1 -- start start 2024-04-20T10:30:17.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.467+0000 7f28cfa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:17.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.467+0000 7f28cfa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28c80ad520 0x7f28c80aef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.467+0000 7f28cfa00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28c806bce0 con 0x7f28c80af4d0 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.467+0000 7f28cfa00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f28c806d2c0 con 0x7f28c80ad520 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.467+0000 7f28cfa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28c806c8f0 con 0x7f28c815a050 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28c7e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28c80ad520 0x7f28c80aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28cda00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f28c80af4d0 0x7f28c8071ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45494/0 (socket says 172.21.15.47:45494) 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28cda00640 1 -- 172.21.15.47:0/1974979821 learned_addr learned my addr 172.21.15.47:0/1974979821 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28cd000640 1 --2- 172.21.15.47:0/1974979821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28c7e00640 1 -- 172.21.15.47:0/1974979821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 msgr2=0x7f28c806cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28c7e00640 1 --2- 172.21.15.47:0/1974979821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c806cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28c7e00640 1 -- 172.21.15.47:0/1974979821 >> v1:172.21.15.47:6789/0 conn(0x7f28c80af4d0 legacy=0x7f28c8071ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28c7e00640 1 -- 172.21.15.47:0/1974979821 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c806bfc0 con 0x7f28c80ad520 2024-04-20T10:30:17.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.468+0000 7f28c7400640 1 -- 172.21.15.47:0/1974979821 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2899028440 0 0) 0x7f28c806bce0 con 0x7f28c80af4d0 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.470+0000 7f28c7e00640 1 --2- 172.21.15.47:0/1974979821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c80aef90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f28c806ca70 tx=0x7f28b002b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7d7f2072561b80 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.470+0000 7f28c7400640 1 -- 172.21.15.47:0/1974979821 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0035020 con 0x7f28c80ad520 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.470+0000 7f28c7400640 1 -- 172.21.15.47:0/1974979821 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b0031800 con 0x7f28c80ad520 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.470+0000 7f28c7400640 1 -- 172.21.15.47:0/1974979821 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0031bd0 con 0x7f28c80ad520 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.470+0000 7f28cfa00640 1 -- 172.21.15.47:0/1974979821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 msgr2=0x7f28c80aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.470+0000 7f28cfa00640 1 --2- 172.21.15.47:0/1974979821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c80aef90 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f28c806ca70 tx=0x7f28b002b8a0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.471+0000 7f28cfa00640 1 -- 172.21.15.47:0/1974979821 shutdown_connections 2024-04-20T10:30:17.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.471+0000 7f28cfa00640 1 --2- 172.21.15.47:0/1974979821 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c80aef90 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.471+0000 7f28cfa00640 1 --2- 172.21.15.47:0/1974979821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c806cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.471+0000 7f28cfa00640 1 -- 172.21.15.47:0/1974979821 >> 172.21.15.47:0/1974979821 conn(0x7f28c80b1d70 msgr2=0x7f28c80b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.471+0000 7f28cfa00640 1 -- 172.21.15.47:0/1974979821 shutdown_connections 2024-04-20T10:30:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.471+0000 7f28cfa00640 1 -- 172.21.15.47:0/1974979821 wait complete. 2024-04-20T10:30:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.472+0000 7f28cfa00640 1 Processor -- start 2024-04-20T10:30:17.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.473+0000 7f28cfa00640 1 -- start start 2024-04-20T10:30:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cfa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cfa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c80af4d0 0x7f28c81507b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cfa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c815ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cfa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f28c80af900 con 0x7f28c80ad520 2024-04-20T10:30:17.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cfa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28c806c580 con 0x7f28c80af4d0 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cfa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28c80ad190 con 0x7f28c815a050 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cd000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c815a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55640/0 (socket says 172.21.15.47:55640) 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cd000640 1 -- 172.21.15.47:0/2157427069 learned_addr learned my addr 172.21.15.47:0/2157427069 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28c7e00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c80af4d0 0x7f28c81507b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cda00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c815ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.474+0000 7f28cd000640 1 -- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 msgr2=0x7f28c815ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28cd000640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c815ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28cd000640 1 -- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c80af4d0 msgr2=0x7f28c81507b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28cd000640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c80af4d0 0x7f28c81507b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28cd000640 1 -- 172.21.15.47:0/2157427069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c8157590 con 0x7f28c80ad520 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28cd000640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c815a510 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f28b800d830 tx=0x7f28b800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:17.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28c4c00640 1 -- 172.21.15.47:0/2157427069 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b800b6c0 con 0x7f28c80ad520 2024-04-20T10:30:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28c4c00640 1 -- 172.21.15.47:0/2157427069 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28b8019070 con 0x7f28c80ad520 2024-04-20T10:30:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28c4c00640 1 -- 172.21.15.47:0/2157427069 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b800ba80 con 0x7f28c80ad520 2024-04-20T10:30:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c8157870 con 0x7f28c80ad520 2024-04-20T10:30:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.475+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c8157e30 con 0x7f28c80ad520 2024-04-20T10:30:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.477+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c8069090 con 0x7f28c80ad520 2024-04-20T10:30:17.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.477+0000 7f28c4c00640 1 -- 172.21.15.47:0/2157427069 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28b801e460 con 0x7f28c80ad520 2024-04-20T10:30:17.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.479+0000 7f28c4c00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:17.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.479+0000 7f28c4c00640 1 -- 172.21.15.47:0/2157427069 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f28b808c110 con 0x7f28c80ad520 2024-04-20T10:30:17.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.480+0000 7f28c7e00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:17.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.481+0000 7f28c7e00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a406c8f0 0x7f28a406edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28b0036000 tx=0x7f28b003c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:17.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.484+0000 7f28c4c00640 1 -- 172.21.15.47:0/2157427069 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28c8069090 con 0x7f28c80ad520 2024-04-20T10:30:17.761 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.761+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f28c8068980 con 0x7f28c80ad520 2024-04-20T10:30:17.762 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.762+0000 7f28c4c00640 1 -- 172.21.15.47:0/2157427069 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f28b805a660 con 0x7f28c80ad520 2024-04-20T10:30:17.762 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:17.762 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:30:17.762 INFO:teuthology.orchestra.run.smithi047.stdout: "auth_dump": [ 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.a", 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmFJVUGBAAWWv9IYGbahiOT/gLniKBOw==", 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.763 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.b", 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmTUa9NRAAqv2igp+zzxGgHeyPdWTaMg==", 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.764 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.c", 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmSGsVJxAA9UQVq15Djoww3cb3RYpnvw==", 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mds.d", 2024-04-20T10:30:17.765 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNm6XxVCBAA2JRaNbycMKfG+H5MRYd/BQ==", 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.0", 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBZmCNmA30rIRAAVq4KbsC5TmRgR2fuF6Ioow==", 2024-04-20T10:30:17.766 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.1", 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBcmCNmTJtrLxAAJmSqkjORGtnWDI0g4to51A==", 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.767 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.2", 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBfmCNmxQDLORAAVyQxgnBHyR13yRBAGF+zNQ==", 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.3", 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBjmCNmL4FwChAA11Oo/q6Csl2I4KAhh7pPDg==", 2024-04-20T10:30:17.768 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.4", 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBpmCNmc0UlHhAA1/xDr2QsraxlgUXGj5zKJA==", 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.769 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.5", 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBsmCNmyZ92ORAAkP7I8i8eANOvmITsE4Sv8Q==", 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.770 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.6", 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBwmCNmag3+CRAAr+SJC24tUSlkRpbdRnZydA==", 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "osd.7", 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBzmCNmF+LYFBAAXXFXRj5Har31kqZtljDnWg==", 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow profile osd", 2024-04-20T10:30:17.771 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile osd", 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.0", 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNmrHlNFRAAY90gnx6m90huEzUUDh9rtw==", 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow r", 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.772 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.1", 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNmaFnFJxAAIjkRNWpCrcCA8aq28sHHqQ==", 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.2", 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNmSmJdHhAAsLDDN4yZBlkSAAPckAd0rw==", 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.773 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.3", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBXmCNm4Nm7MRAA8yPbfNcKnJjd0QiaBTOcMw==", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow r", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow rw", 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow rwx" 2024-04-20T10:30:17.774 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "client.admin", 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBVmCNmv2ymFRAAJV17P7iHLBSO40PM78whWA==", 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mgr": "allow *", 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow *", 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.x", 2024-04-20T10:30:17.775 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNmVaHQDRAA1jKTsPvvO24I4zZ996gBQg==", 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "entity": "mgr.y", 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "key": "AQBWmCNm862fAxAA3IcbrK5u7lUlerIjzUqIwA==", 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "caps": { 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "mds": "allow *", 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "mon": "allow profile mgr", 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: "osd": "allow *" 2024-04-20T10:30:17.776 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.777 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:17.777 INFO:teuthology.orchestra.run.smithi047.stdout: ] 2024-04-20T10:30:17.777 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a406c8f0 msgr2=0x7f28a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a406c8f0 0x7f28a406edb0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f28b0036000 tx=0x7f28b003c040 comp rx=0 tx=0).stop 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 msgr2=0x7f28c815a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c815a510 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f28b800d830 tx=0x7f28b800dcf0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 shutdown_connections 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28c815a050 0x7f28c815ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c80af4d0 0x7f28c81507b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28a406c8f0 0x7f28a406edb0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 --2- 172.21.15.47:0/2157427069 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28c80ad520 0x7f28c815a510 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:17.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 >> 172.21.15.47:0/2157427069 conn(0x7f28c80b1d70 msgr2=0x7f28c8154810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:17.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 shutdown_connections 2024-04-20T10:30:17.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.765+0000 7f28cfa00640 1 -- 172.21.15.47:0/2157427069 wait complete. 2024-04-20T10:30:17.784 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.997+0000 7f53fc200640 1 Processor -- start 2024-04-20T10:30:17.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.997+0000 7f53fc200640 1 -- start start 2024-04-20T10:30:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53fc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 0x7f53f40d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53fc200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f4141c60 con 0x7f53f415ca40 2024-04-20T10:30:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53fc200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f53f40adab0 con 0x7f53f4156c20 2024-04-20T10:30:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53fc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53f406c010 con 0x7f53f40d0360 2024-04-20T10:30:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53f8e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f53f415ca40 0x7f53f40ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45510/0 (socket says 172.21.15.47:45510) 2024-04-20T10:30:17.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53f8e00640 1 -- 172.21.15.47:0/1030075251 learned_addr learned my addr 172.21.15.47:0/1030075251 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2296370911 0 0) 0x7f53f4141c60 con 0x7f53f415ca40 2024-04-20T10:30:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:17.999+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f53d8003520 con 0x7f53f415ca40 2024-04-20T10:30:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53fa200640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 0x7f53f40d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2970978804 0 0) 0x7f53d8003520 con 0x7f53f415ca40 2024-04-20T10:30:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 msgr2=0x7f53f40d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:18.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 0x7f53f40d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 >> v1:172.21.15.175:6789/0 conn(0x7f53f4156c20 legacy=0x7f53f406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f415cf90 con 0x7f53f415ca40 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f53e0001670 con 0x7f53f415ca40 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 >> v1:172.21.15.47:6789/0 conn(0x7f53f415ca40 legacy=0x7f53f40ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 0x7f53d80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004060 0x7f53d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d8004d40 0x7f53d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53f406c010 con 0x7f53d8004d40 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53d800bc00 con 0x7f53d8004060 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53d800be00 con 0x7f53f40d0360 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.000+0000 7f53fa200640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d8004d40 0x7f53d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.001+0000 7f53f8e00640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004060 0x7f53d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.001+0000 7f53f9800640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 0x7f53d80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.001+0000 7f53fa200640 1 -- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 msgr2=0x7f53d80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.001+0000 7f53fa200640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f40d0360 0x7f53d80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.001+0000 7f53fa200640 1 -- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004060 msgr2=0x7f53d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.001+0000 7f53fa200640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004060 0x7f53d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.001+0000 7f53fa200640 1 -- 172.21.15.47:0/1030075251 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d8003520 con 0x7f53d8004d40 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.002+0000 7f53fa200640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d8004d40 0x7f53d80051c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00ba30 tx=0x7f53d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42124e13f73fd681 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.002+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec036020 con 0x7f53d8004d40 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.002+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec032640 con 0x7f53d8004d40 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.002+0000 7f53f3e00640 1 -- 172.21.15.47:0/1030075251 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00b600 con 0x7f53d8004d40 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.003+0000 7f53fc200640 1 -- 172.21.15.47:0/1030075251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d8004d40 msgr2=0x7f53d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.003+0000 7f53fc200640 1 --2- 172.21.15.47:0/1030075251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53d8004d40 0x7f53d80051c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00ba30 tx=0x7f53d8005830 comp rx=0 tx=0).stop 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.003+0000 7f53fa200640 1 -- 172.21.15.47:0/1030075251 reap_dead start 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.003+0000 7f53fc200640 1 -- 172.21.15.47:0/1030075251 shutdown_connections 2024-04-20T10:30:18.004 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.003+0000 7f53fc200640 1 -- 172.21.15.47:0/1030075251 >> 172.21.15.47:0/1030075251 conn(0x7f53f40b3350 msgr2=0x7f53f40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.003+0000 7f53fc200640 1 -- 172.21.15.47:0/1030075251 shutdown_connections 2024-04-20T10:30:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.003+0000 7f53fc200640 1 -- 172.21.15.47:0/1030075251 wait complete. 2024-04-20T10:30:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.005+0000 7f53fc200640 1 Processor -- start 2024-04-20T10:30:18.005 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.005+0000 7f53fc200640 1 -- start start 2024-04-20T10:30:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.006+0000 7f53fc200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004d40 0x7f53f4145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.006+0000 7f53fc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f41509b0 0x7f53f4150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.006+0000 7f53fc200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53f4153160 0x7f53f41535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.006+0000 7f53fc200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f53f415d830 con 0x7f53f4153160 2024-04-20T10:30:18.007 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.006+0000 7f53fc200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53f415d550 con 0x7f53d8004d40 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.006+0000 7f53fc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53f40d07f0 con 0x7f53f41509b0 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.006+0000 7f53fa200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53f4153160 0x7f53f41535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53f4153160 0x7f53f41535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55656/0 (socket says 172.21.15.47:55656) 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 -- 172.21.15.47:0/823243861 learned_addr learned my addr 172.21.15.47:0/823243861 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53f9800640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004d40 0x7f53f4145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53f8e00640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f41509b0 0x7f53f4150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 -- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f41509b0 msgr2=0x7f53f4150e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f41509b0 0x7f53f4150e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 -- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004d40 msgr2=0x7f53f4145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004d40 0x7f53f4145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 -- 172.21.15.47:0/823243861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53f4155cd0 con 0x7f53f4153160 2024-04-20T10:30:18.008 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.007+0000 7f53fa200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53f4153160 0x7f53f41535e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00bb50 tx=0x7f53ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.008+0000 7f53f1600640 1 -- 172.21.15.47:0/823243861 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec036040 con 0x7f53f4153160 2024-04-20T10:30:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.008+0000 7f53f1600640 1 -- 172.21.15.47:0/823243861 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f53ec03f070 con 0x7f53f4153160 2024-04-20T10:30:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.008+0000 7f53f1600640 1 -- 172.21.15.47:0/823243861 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00b600 con 0x7f53f4153160 2024-04-20T10:30:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.008+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53f415f700 con 0x7f53f4153160 2024-04-20T10:30:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.008+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53f4152960 con 0x7f53f4153160 2024-04-20T10:30:18.009 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.009+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53f40afa70 con 0x7f53f4153160 2024-04-20T10:30:18.010 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.010+0000 7f53f1600640 1 -- 172.21.15.47:0/823243861 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f53ec039020 con 0x7f53f4153160 2024-04-20T10:30:18.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.012+0000 7f53f1600640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53d406c920 0x7f53d406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.012+0000 7f53f1600640 1 -- 172.21.15.47:0/823243861 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7f53ec0adad0 con 0x7f53f4153160 2024-04-20T10:30:18.012 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.013+0000 7f53f9800640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53d406c920 0x7f53d406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.014+0000 7f53f9800640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53d406c920 0x7f53d406ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f53f4074020 tx=0x7f53e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.017+0000 7f53f1600640 1 -- 172.21.15.47:0/823243861 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f53f40afa70 con 0x7f53f4153160 2024-04-20T10:30:18.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.224+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53f4068980 con 0x7f53f4153160 2024-04-20T10:30:18.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.233+0000 7f53f1600640 1 -- 172.21.15.47:0/823243861 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) ==== 74+0+13176 (secure 0 0 0) 0x7f53ec04b020 con 0x7f53f4153160 2024-04-20T10:30:18.233 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:18.233 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":50,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:17.253606+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.236+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53d406c920 msgr2=0x7f53d406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.236+0000 7f53fc200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53d406c920 0x7f53d406ede0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f53f4074020 tx=0x7f53e4009040 comp rx=0 tx=0).stop 2024-04-20T10:30:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.236+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53f4153160 msgr2=0x7f53f41535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.236+0000 7f53fc200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53f4153160 0x7f53f41535e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f53ec00bb50 tx=0x7f53ec004730 comp rx=0 tx=0).stop 2024-04-20T10:30:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 shutdown_connections 2024-04-20T10:30:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f53d406c920 0x7f53d406ede0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f53f4153160 0x7f53f41535e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53f41509b0 0x7f53f4150e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 --2- 172.21.15.47:0/823243861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53d8004d40 0x7f53f4145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 >> 172.21.15.47:0/823243861 conn(0x7f53f40b3350 msgr2=0x7f53f415e590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 shutdown_connections 2024-04-20T10:30:18.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.237+0000 7f53fc200640 1 -- 172.21.15.47:0/823243861 wait complete. 2024-04-20T10:30:18.259 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:30:18.260 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:30:18.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.466+0000 7fee41c00640 1 Processor -- start 2024-04-20T10:30:18.466 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.466+0000 7fee41c00640 1 -- start start 2024-04-20T10:30:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee41c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee3c154fe0 0x7fee3c13e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee41c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fee3c0cad70 0x7fee3c0ad7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee41c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0add20 0x7fee3c15b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee41c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fee3c15e5d0 con 0x7fee3c154fe0 2024-04-20T10:30:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee41c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fee3c0ae100 con 0x7fee3c0cad70 2024-04-20T10:30:18.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee41c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c15e870 con 0x7fee3c0add20 2024-04-20T10:30:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee3be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee3c154fe0 0x7fee3c13e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee3be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee3c154fe0 0x7fee3c13e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55670/0 (socket says 172.21.15.47:55670) 2024-04-20T10:30:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.468+0000 7fee3be00640 1 -- 172.21.15.47:0/3224575989 learned_addr learned my addr 172.21.15.47:0/3224575989 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.469+0000 7fee3b400640 1 --2- 172.21.15.47:0/3224575989 >> v2:172.21.15.175:3300/0 conn(0x7fee3c0cad70 0x7fee3c0ad7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.469+0000 7fee3be00640 1 -- 172.21.15.47:0/3224575989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0add20 msgr2=0x7fee3c15b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.469+0000 7fee3be00640 1 --2- 172.21.15.47:0/3224575989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0add20 0x7fee3c15b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.469+0000 7fee3be00640 1 -- 172.21.15.47:0/3224575989 >> v2:172.21.15.175:3300/0 conn(0x7fee3c0cad70 msgr2=0x7fee3c0ad7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.469+0000 7fee3be00640 1 --2- 172.21.15.47:0/3224575989 >> v2:172.21.15.175:3300/0 conn(0x7fee3c0cad70 0x7fee3c0ad7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.469+0000 7fee3be00640 1 -- 172.21.15.47:0/3224575989 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c15eb50 con 0x7fee3c154fe0 2024-04-20T10:30:18.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.470+0000 7fee3be00640 1 --2- 172.21.15.47:0/3224575989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c154fe0 0x7fee3c13e1b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fee3c158bb0 tx=0x7fee2c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f220ba923cf2b059 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.471+0000 7fee3aa00640 1 -- 172.21.15.47:0/3224575989 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c02ca80 con 0x7fee3c154fe0 2024-04-20T10:30:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.471+0000 7fee3aa00640 1 -- 172.21.15.47:0/3224575989 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2c02cc20 con 0x7fee3c154fe0 2024-04-20T10:30:18.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.471+0000 7fee3aa00640 1 -- 172.21.15.47:0/3224575989 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2c0343f0 con 0x7fee3c154fe0 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.471+0000 7fee41c00640 1 -- 172.21.15.47:0/3224575989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c154fe0 msgr2=0x7fee3c13e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.471+0000 7fee41c00640 1 --2- 172.21.15.47:0/3224575989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c154fe0 0x7fee3c13e1b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fee3c158bb0 tx=0x7fee2c02bee0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.472+0000 7fee41c00640 1 -- 172.21.15.47:0/3224575989 shutdown_connections 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.472+0000 7fee41c00640 1 --2- 172.21.15.47:0/3224575989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c0add20 0x7fee3c15b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.472+0000 7fee41c00640 1 --2- 172.21.15.47:0/3224575989 >> v2:172.21.15.175:3300/0 conn(0x7fee3c0cad70 0x7fee3c0ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.472+0000 7fee41c00640 1 --2- 172.21.15.47:0/3224575989 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c154fe0 0x7fee3c13e1b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.472+0000 7fee41c00640 1 -- 172.21.15.47:0/3224575989 >> 172.21.15.47:0/3224575989 conn(0x7fee3c0cffa0 msgr2=0x7fee3c0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.472+0000 7fee41c00640 1 -- 172.21.15.47:0/3224575989 shutdown_connections 2024-04-20T10:30:18.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.472+0000 7fee41c00640 1 -- 172.21.15.47:0/3224575989 wait complete. 2024-04-20T10:30:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.473+0000 7fee41c00640 1 Processor -- start 2024-04-20T10:30:18.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.474+0000 7fee41c00640 1 -- start start 2024-04-20T10:30:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee41c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c0add20 0x7fee3c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee41c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c0cad70 0x7fee3c151d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee41c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c154fe0 0x7fee3c166a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee41c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c15f3f0 con 0x7fee3c0cad70 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee41c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee3c15f110 con 0x7fee3c0add20 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee41c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee3c1553c0 con 0x7fee3c154fe0 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c0cad70 0x7fee3c151d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c0cad70 0x7fee3c151d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55684/0 (socket says 172.21.15.47:55684) 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3b400640 1 -- 172.21.15.47:0/3959738557 learned_addr learned my addr 172.21.15.47:0/3959738557 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3be00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c0add20 0x7fee3c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee41200640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c154fe0 0x7fee3c166a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3b400640 1 -- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c154fe0 msgr2=0x7fee3c166a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3b400640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c154fe0 0x7fee3c166a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3b400640 1 -- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c0add20 msgr2=0x7fee3c06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.475+0000 7fee3b400640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c0add20 0x7fee3c06f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.476+0000 7fee3b400640 1 -- 172.21.15.47:0/3959738557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3c06fce0 con 0x7fee3c0cad70 2024-04-20T10:30:18.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.476+0000 7fee3b400640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c0cad70 0x7fee3c151d50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fee2400d830 tx=0x7fee2400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.476+0000 7fee33e00640 1 -- 172.21.15.47:0/3959738557 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2400b660 con 0x7fee3c0cad70 2024-04-20T10:30:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.476+0000 7fee33e00640 1 -- 172.21.15.47:0/3959738557 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee24019070 con 0x7fee3c0cad70 2024-04-20T10:30:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.476+0000 7fee33e00640 1 -- 172.21.15.47:0/3959738557 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2400ba20 con 0x7fee3c0cad70 2024-04-20T10:30:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.476+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee3c06dbf0 con 0x7fee3c0cad70 2024-04-20T10:30:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.476+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee3c06e1b0 con 0x7fee3c0cad70 2024-04-20T10:30:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.477+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3c075b30 con 0x7fee3c0cad70 2024-04-20T10:30:18.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.479+0000 7fee33e00640 1 -- 172.21.15.47:0/3959738557 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee2401e460 con 0x7fee3c0cad70 2024-04-20T10:30:18.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.480+0000 7fee33e00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1806c8f0 0x7fee1806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.480+0000 7fee33e00640 1 -- 172.21.15.47:0/3959738557 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7fee2408bf80 con 0x7fee3c0cad70 2024-04-20T10:30:18.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.480+0000 7fee3be00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1806c8f0 0x7fee1806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.481+0000 7fee3be00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1806c8f0 0x7fee1806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee2c035000 tx=0x7fee2c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.484+0000 7fee33e00640 1 -- 172.21.15.47:0/3959738557 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee2405a600 con 0x7fee3c0cad70 2024-04-20T10:30:18.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.686+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fee3c06cd00 con 0x7fee3c0cad70 2024-04-20T10:30:18.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.687+0000 7fee33e00640 1 -- 172.21.15.47:0/3959738557 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) ==== 93+0+10 (secure 0 0 0) 0x7fee24059f00 con 0x7fee3c0cad70 2024-04-20T10:30:18.687 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:30:18.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.689+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1806c8f0 msgr2=0x7fee1806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.689+0000 7fee41c00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1806c8f0 0x7fee1806edb0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fee2c035000 tx=0x7fee2c030000 comp rx=0 tx=0).stop 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.689+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c0cad70 msgr2=0x7fee3c151d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.689+0000 7fee41c00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c0cad70 0x7fee3c151d50 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fee2400d830 tx=0x7fee2400dcf0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 shutdown_connections 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee3c154fe0 0x7fee3c166a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1806c8f0 0x7fee1806edb0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee3c0cad70 0x7fee3c151d50 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 --2- 172.21.15.47:0/3959738557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3c0add20 0x7fee3c06f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 >> 172.21.15.47:0/3959738557 conn(0x7fee3c0cffa0 msgr2=0x7fee3c0cdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 shutdown_connections 2024-04-20T10:30:18.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.690+0000 7fee41c00640 1 -- 172.21.15.47:0/3959738557 wait complete. 2024-04-20T10:30:18.710 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-20T10:30:18.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.921+0000 7efbfe800640 1 Processor -- start 2024-04-20T10:30:18.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.921+0000 7efbfe800640 1 -- start start 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.922+0000 7efbfe800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efbf8161d50 0x7efbf80d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.922+0000 7efbfe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.922+0000 7efbfe800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efbf806c190 con 0x7efbf806c6d0 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.922+0000 7efbfe800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efbf8162130 con 0x7efbf8161d50 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.922+0000 7efbfe800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efbf806c470 con 0x7efbf80b3330 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efbf8161d50 0x7efbf80d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfd400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efbf8161d50 0x7efbf80d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55698/0 (socket says 172.21.15.47:55698) 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfd400640 1 -- 172.21.15.47:0/1189789187 learned_addr learned my addr 172.21.15.47:0/1189789187 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:18.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfde00640 1 --1- 172.21.15.47:0/1189789187 >> v1:172.21.15.47:6789/0 conn(0x7efbf806c6d0 0x7efbf80cfa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45520/0 (socket says 172.21.15.47:45520) 2024-04-20T10:30:18.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfd400640 1 -- 172.21.15.47:0/1189789187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 msgr2=0x7efbf80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:18.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfca00640 1 --2- 172.21.15.47:0/1189789187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfd400640 1 --2- 172.21.15.47:0/1189789187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfd400640 1 -- 172.21.15.47:0/1189789187 >> v1:172.21.15.47:6789/0 conn(0x7efbf806c6d0 legacy=0x7efbf80cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.923+0000 7efbfd400640 1 -- 172.21.15.47:0/1189789187 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf80d2e00 con 0x7efbf8161d50 2024-04-20T10:30:18.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.924+0000 7efbfd400640 1 --2- 172.21.15.47:0/1189789187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80d38e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009870 tx=0x7efbe802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2eb55d404fb68d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.925+0000 7efbf7e00640 1 -- 172.21.15.47:0/1189789187 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8034070 con 0x7efbf8161d50 2024-04-20T10:30:18.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.925+0000 7efbf7e00640 1 -- 172.21.15.47:0/1189789187 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbe8030840 con 0x7efbf8161d50 2024-04-20T10:30:18.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.925+0000 7efbf7e00640 1 -- 172.21.15.47:0/1189789187 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbe8030c10 con 0x7efbf8161d50 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 -- 172.21.15.47:0/1189789187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 msgr2=0x7efbf80d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 --2- 172.21.15.47:0/1189789187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80d38e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7efbe8009870 tx=0x7efbe802b980 comp rx=0 tx=0).stop 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 -- 172.21.15.47:0/1189789187 shutdown_connections 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 --2- 172.21.15.47:0/1189789187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 --2- 172.21.15.47:0/1189789187 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80d38e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 -- 172.21.15.47:0/1189789187 >> 172.21.15.47:0/1189789187 conn(0x7efbf80d4450 msgr2=0x7efbf80d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 -- 172.21.15.47:0/1189789187 shutdown_connections 2024-04-20T10:30:18.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.926+0000 7efbfe800640 1 -- 172.21.15.47:0/1189789187 wait complete. 2024-04-20T10:30:18.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.928+0000 7efbfe800640 1 Processor -- start 2024-04-20T10:30:18.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.928+0000 7efbfe800640 1 -- start start 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.929+0000 7efbfe800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf806c6d0 0x7efbf814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.929+0000 7efbfe800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.929+0000 7efbfe800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfe800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efbf806cb60 con 0x7efbf8161d50 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfe800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efbf80d33c0 con 0x7efbf806c6d0 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfe800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efbf806cce0 con 0x7efbf80b3330 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80cd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55704/0 (socket says 172.21.15.47:55704) 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 -- 172.21.15.47:0/3121731936 learned_addr learned my addr 172.21.15.47:0/3121731936 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:18.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfd400640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf806c6d0 0x7efbf814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfca00640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 -- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 msgr2=0x7efbf80cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 -- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf806c6d0 msgr2=0x7efbf814a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf806c6d0 0x7efbf814a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfde00640 1 -- 172.21.15.47:0/3121731936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf815d8e0 con 0x7efbf8161d50 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.930+0000 7efbfca00640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.931+0000 7efbfde00640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80cd190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efbf000c430 tx=0x7efbf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.931+0000 7efbf5600640 1 -- 172.21.15.47:0/3121731936 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf00074e0 con 0x7efbf8161d50 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.931+0000 7efbf5600640 1 -- 172.21.15.47:0/3121731936 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efbf0007680 con 0x7efbf8161d50 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.931+0000 7efbf5600640 1 -- 172.21.15.47:0/3121731936 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf00049e0 con 0x7efbf8161d50 2024-04-20T10:30:18.933 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.931+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf815dbc0 con 0x7efbf8161d50 2024-04-20T10:30:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.931+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf80c8780 con 0x7efbf8161d50 2024-04-20T10:30:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.933+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf80c8980 con 0x7efbf8161d50 2024-04-20T10:30:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.934+0000 7efbf5600640 1 -- 172.21.15.47:0/3121731936 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efbf001a020 con 0x7efbf8161d50 2024-04-20T10:30:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.934+0000 7efbf5600640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efbd806c8f0 0x7efbd806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:18.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.935+0000 7efbf5600640 1 -- 172.21.15.47:0/3121731936 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7efbf008b200 con 0x7efbf8161d50 2024-04-20T10:30:18.935 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.935+0000 7efbfd400640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efbd806c8f0 0x7efbd806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:18.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.936+0000 7efbfd400640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efbd806c8f0 0x7efbd806edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efbe8036000 tx=0x7efbe803b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:18.938 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:18.939+0000 7efbf5600640 1 -- 172.21.15.47:0/3121731936 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efbf80c8980 con 0x7efbf8161d50 2024-04-20T10:30:19.146 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.146+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7efbf80cb2e0 con 0x7efbf8161d50 2024-04-20T10:30:19.147 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.147+0000 7efbf5600640 1 -- 172.21.15.47:0/3121731936 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) ==== 84+0+65 (secure 0 0 0) 0x7efbf0010300 con 0x7efbf8161d50 2024-04-20T10:30:19.147 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:19.147 INFO:teuthology.orchestra.run.smithi047.stdout:[ 2024-04-20T10:30:19.147 INFO:teuthology.orchestra.run.smithi047.stdout: { 2024-04-20T10:30:19.147 INFO:teuthology.orchestra.run.smithi047.stdout: "poolnum": 1, 2024-04-20T10:30:19.147 INFO:teuthology.orchestra.run.smithi047.stdout: "poolname": ".mgr" 2024-04-20T10:30:19.148 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:19.148 INFO:teuthology.orchestra.run.smithi047.stdout:] 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.149+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efbd806c8f0 msgr2=0x7efbd806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.149+0000 7efbfe800640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efbd806c8f0 0x7efbd806edb0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7efbe8036000 tx=0x7efbe803b040 comp rx=0 tx=0).stop 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.149+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 msgr2=0x7efbf80cd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.149+0000 7efbfe800640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80cd190 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7efbf000c430 tx=0x7efbf000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 shutdown_connections 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efbd806c8f0 0x7efbd806edb0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efbf8161d50 0x7efbf80cd190 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.150 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efbf80b3330 0x7efbf80cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 --2- 172.21.15.47:0/3121731936 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efbf806c6d0 0x7efbf814a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 >> 172.21.15.47:0/3121731936 conn(0x7efbf80d4450 msgr2=0x7efbf80d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 shutdown_connections 2024-04-20T10:30:19.151 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.150+0000 7efbfe800640 1 -- 172.21.15.47:0/3121731936 wait complete. 2024-04-20T10:30:19.169 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-20T10:30:19.169 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-20T10:30:19.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.377+0000 7fa459200640 1 Processor -- start 2024-04-20T10:30:19.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.378+0000 7fa459200640 1 -- start start 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa459200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa45415c400 0x7fa45406f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa459200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fa45406d920 0x7fa45406c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa459200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa45406de90 con 0x7fa45406ce50 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa459200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa45406e170 con 0x7fa45415c400 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa459200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fa45415d7f0 con 0x7fa45406d920 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa453400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa45415c400 0x7fa45406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa453400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa45415c400 0x7fa45406f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55720/0 (socket says 172.21.15.47:55720) 2024-04-20T10:30:19.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa453e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa45406ce50 0x7fa4540cdb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45524/0 (socket says 172.21.15.47:45524) 2024-04-20T10:30:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.379+0000 7fa453400640 1 -- 172.21.15.47:0/2986669364 learned_addr learned my addr 172.21.15.47:0/2986669364 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.380+0000 7fa452a00640 1 --2- 172.21.15.47:0/2986669364 >> v2:172.21.15.175:3300/0 conn(0x7fa45406d920 0x7fa45406c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.380+0000 7fa453400640 1 -- 172.21.15.47:0/2986669364 >> v2:172.21.15.175:3300/0 conn(0x7fa45406d920 msgr2=0x7fa45406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.380+0000 7fa453400640 1 --2- 172.21.15.47:0/2986669364 >> v2:172.21.15.175:3300/0 conn(0x7fa45406d920 0x7fa45406c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.380+0000 7fa453400640 1 -- 172.21.15.47:0/2986669364 >> v1:172.21.15.47:6789/0 conn(0x7fa45406ce50 legacy=0x7fa4540cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.380+0000 7fa453400640 1 -- 172.21.15.47:0/2986669364 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa454072130 con 0x7fa45415c400 2024-04-20T10:30:19.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.380+0000 7fa452000640 1 -- 172.21.15.47:0/2986669364 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1232415603 0 0) 0x7fa45406de90 con 0x7fa45406ce50 2024-04-20T10:30:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.381+0000 7fa453400640 1 --2- 172.21.15.47:0/2986669364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45415c400 0x7fa45406f570 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa4540cf730 tx=0x7fa44402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20007b74ff50f14c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.381+0000 7fa452000640 1 -- 172.21.15.47:0/2986669364 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444034070 con 0x7fa45415c400 2024-04-20T10:30:19.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.381+0000 7fa452000640 1 -- 172.21.15.47:0/2986669364 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa444030860 con 0x7fa45415c400 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa452000640 1 -- 172.21.15.47:0/2986669364 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa444030c30 con 0x7fa45415c400 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 -- 172.21.15.47:0/2986669364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45415c400 msgr2=0x7fa45406f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 --2- 172.21.15.47:0/2986669364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45415c400 0x7fa45406f570 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fa4540cf730 tx=0x7fa44402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 -- 172.21.15.47:0/2986669364 shutdown_connections 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 --2- 172.21.15.47:0/2986669364 >> v2:172.21.15.175:3300/0 conn(0x7fa45406d920 0x7fa45406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 --2- 172.21.15.47:0/2986669364 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45415c400 0x7fa45406f570 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 -- 172.21.15.47:0/2986669364 >> 172.21.15.47:0/2986669364 conn(0x7fa4540cc9b0 msgr2=0x7fa4540ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 -- 172.21.15.47:0/2986669364 shutdown_connections 2024-04-20T10:30:19.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.382+0000 7fa459200640 1 -- 172.21.15.47:0/2986669364 wait complete. 2024-04-20T10:30:19.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.384+0000 7fa459200640 1 Processor -- start 2024-04-20T10:30:19.384 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.384+0000 7fa459200640 1 -- start start 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa459200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa45406ce50 0x7fa4540ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa459200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45406d920 0x7fa4541ef440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa459200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45415c400 0x7fa45406c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa459200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa454072780 con 0x7fa45406d920 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa459200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa454071ee0 con 0x7fa45415c400 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa459200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa454071bd0 con 0x7fa45406ce50 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa452a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45406d920 0x7fa4541ef440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa452a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45406d920 0x7fa4541ef440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55730/0 (socket says 172.21.15.47:55730) 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.385+0000 7fa452a00640 1 -- 172.21.15.47:0/1687960196 learned_addr learned my addr 172.21.15.47:0/1687960196 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.386+0000 7fa453e00640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45415c400 0x7fa45406c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.386+0000 7fa452a00640 1 -- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa45406ce50 msgr2=0x7fa4540ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:19.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.386+0000 7fa452a00640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa45406ce50 0x7fa4540ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.386+0000 7fa452a00640 1 -- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45415c400 msgr2=0x7fa45406c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.386+0000 7fa452a00640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45415c400 0x7fa45406c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.386+0000 7fa452a00640 1 -- 172.21.15.47:0/1687960196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4540ad650 con 0x7fa45406d920 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.386+0000 7fa452a00640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45406d920 0x7fa4541ef440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007f20 tx=0x7fa43c00e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.387+0000 7fa44b400640 1 -- 172.21.15.47:0/1687960196 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c014620 con 0x7fa45406d920 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.387+0000 7fa44b400640 1 -- 172.21.15.47:0/1687960196 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa43c019070 con 0x7fa45406d920 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.387+0000 7fa44b400640 1 -- 172.21.15.47:0/1687960196 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa43c014a00 con 0x7fa45406d920 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.387+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4540ad930 con 0x7fa45406d920 2024-04-20T10:30:19.388 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.387+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4541f3350 con 0x7fa45406d920 2024-04-20T10:30:19.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.388+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4540ade50 con 0x7fa45406d920 2024-04-20T10:30:19.389 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.389+0000 7fa44b400640 1 -- 172.21.15.47:0/1687960196 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa43c014ba0 con 0x7fa45406d920 2024-04-20T10:30:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.390+0000 7fa44b400640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa430070d00 0x7fa4300731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.390+0000 7fa44b400640 1 -- 172.21.15.47:0/1687960196 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5249+0+0 (secure 0 0 0) 0x7fa43c08c370 con 0x7fa45406d920 2024-04-20T10:30:19.390 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.391+0000 7fa453400640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa430070d00 0x7fa4300731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:19.392 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.392+0000 7fa453400640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa430070d00 0x7fa4300731c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa444036000 tx=0x7fa444030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:19.394 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.394+0000 7fa44b400640 1 -- 172.21.15.47:0/1687960196 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa43c05a8c0 con 0x7fa45406d920 2024-04-20T10:30:19.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:19.595+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa454075b30 con 0x7fa45406d920 2024-04-20T10:30:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.254+0000 7fa44b400640 1 -- 172.21.15.47:0/1687960196 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) ==== 138+0+0 (secure 0 0 0) 0x7fa43c05a1c0 con 0x7fa45406d920 2024-04-20T10:30:20.254 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' created 2024-04-20T10:30:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa430070d00 msgr2=0x7fa4300731c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa430070d00 0x7fa4300731c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa444036000 tx=0x7fa444030000 comp rx=0 tx=0).stop 2024-04-20T10:30:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45406d920 msgr2=0x7fa4541ef440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45406d920 0x7fa4541ef440 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa43c007f20 tx=0x7fa43c00e570 comp rx=0 tx=0).stop 2024-04-20T10:30:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 shutdown_connections 2024-04-20T10:30:20.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa45415c400 0x7fa45406c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa430070d00 0x7fa4300731c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa45406d920 0x7fa4541ef440 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.257+0000 7fa459200640 1 --2- 172.21.15.47:0/1687960196 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa45406ce50 0x7fa4540ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.258+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 >> 172.21.15.47:0/1687960196 conn(0x7fa4540cc9b0 msgr2=0x7fa4540affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.258+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 shutdown_connections 2024-04-20T10:30:20.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.258+0000 7fa459200640 1 -- 172.21.15.47:0/1687960196 wait complete. 2024-04-20T10:30:20.284 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-20T10:30:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.484+0000 7f57faa00640 1 Processor -- start 2024-04-20T10:30:20.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.484+0000 7f57faa00640 1 -- start start 2024-04-20T10:30:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57faa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f57f40b2f70 0x7f57f40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57faa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57f406b900 0x7f57f406bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57faa00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f4071a80 con 0x7f57f40adf60 2024-04-20T10:30:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57faa00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f57f40ae3f0 con 0x7f57f406b900 2024-04-20T10:30:20.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57faa00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f57f406c270 con 0x7f57f40b2f70 2024-04-20T10:30:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57fa000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57f406b900 0x7f57f406bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57fa000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f57f406b900 0x7f57f406bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55732/0 (socket says 172.21.15.47:55732) 2024-04-20T10:30:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.486+0000 7f57fa000640 1 -- 172.21.15.47:0/785880272 learned_addr learned my addr 172.21.15.47:0/785880272 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.487+0000 7f57f9600640 1 --2- 172.21.15.47:0/785880272 >> v2:172.21.15.175:3300/0 conn(0x7f57f40b2f70 0x7f57f40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:20.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.487+0000 7f57fa000640 1 -- 172.21.15.47:0/785880272 >> v2:172.21.15.175:3300/0 conn(0x7f57f40b2f70 msgr2=0x7f57f40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.487+0000 7f57fa000640 1 --2- 172.21.15.47:0/785880272 >> v2:172.21.15.175:3300/0 conn(0x7f57f40b2f70 0x7f57f40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.487+0000 7f57fa000640 1 -- 172.21.15.47:0/785880272 >> v1:172.21.15.175:6789/0 conn(0x7f57f40adf60 legacy=0x7f57f415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.487+0000 7f57fa000640 1 -- 172.21.15.47:0/785880272 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4071cf0 con 0x7f57f406b900 2024-04-20T10:30:20.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.488+0000 7f57fa000640 1 --2- 172.21.15.47:0/785880272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f406b900 0x7f57f406bd30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f57f40695f0 tx=0x7f57ec02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11552c5f988b69a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.488+0000 7f57f3e00640 1 -- 172.21.15.47:0/785880272 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec02ca80 con 0x7f57f406b900 2024-04-20T10:30:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57f3e00640 1 -- 172.21.15.47:0/785880272 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec02cc20 con 0x7f57f406b900 2024-04-20T10:30:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57f3e00640 1 -- 172.21.15.47:0/785880272 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0343f0 con 0x7f57f406b900 2024-04-20T10:30:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57faa00640 1 -- 172.21.15.47:0/785880272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f406b900 msgr2=0x7f57f406bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:20.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57faa00640 1 --2- 172.21.15.47:0/785880272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f406b900 0x7f57f406bd30 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f57f40695f0 tx=0x7f57ec02bee0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57faa00640 1 -- 172.21.15.47:0/785880272 shutdown_connections 2024-04-20T10:30:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57faa00640 1 --2- 172.21.15.47:0/785880272 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f406b900 0x7f57f406bd30 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57faa00640 1 --2- 172.21.15.47:0/785880272 >> v2:172.21.15.175:3300/0 conn(0x7f57f40b2f70 0x7f57f40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.489+0000 7f57faa00640 1 -- 172.21.15.47:0/785880272 >> 172.21.15.47:0/785880272 conn(0x7f57f40d0040 msgr2=0x7f57f40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.490+0000 7f57faa00640 1 -- 172.21.15.47:0/785880272 shutdown_connections 2024-04-20T10:30:20.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.490+0000 7f57faa00640 1 -- 172.21.15.47:0/785880272 wait complete. 2024-04-20T10:30:20.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.491+0000 7f57faa00640 1 Processor -- start 2024-04-20T10:30:20.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.492+0000 7f57faa00640 1 -- start start 2024-04-20T10:30:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.492+0000 7f57faa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f406b900 0x7f57f4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57faa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f40adf60 0x7f57f409ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57faa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f40b2f70 0x7f57f415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57faa00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40736c0 con 0x7f57f40b2f70 2024-04-20T10:30:20.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57faa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57f40739c0 con 0x7f57f40adf60 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57faa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57f415fcc0 con 0x7f57f406b900 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f40b2f70 0x7f57f415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f40b2f70 0x7f57f415b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55744/0 (socket says 172.21.15.47:55744) 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 -- 172.21.15.47:0/3836110039 learned_addr learned my addr 172.21.15.47:0/3836110039 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57f9600640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f406b900 0x7f57f4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 -- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f406b900 msgr2=0x7f57f4151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f406b900 0x7f57f4151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 -- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f40adf60 msgr2=0x7f57f409ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57f8c00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f40adf60 0x7f57f409ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f40adf60 0x7f57f409ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57fa000640 1 -- 172.21.15.47:0/3836110039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4151f80 con 0x7f57f40b2f70 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.493+0000 7f57f8c00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f40adf60 0x7f57f409ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:20.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.494+0000 7f57fa000640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f40b2f70 0x7f57f415b7f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009f40 tx=0x7f57ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.494+0000 7f57f1600640 1 -- 172.21.15.47:0/3836110039 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec043020 con 0x7f57f40b2f70 2024-04-20T10:30:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.494+0000 7f57f1600640 1 -- 172.21.15.47:0/3836110039 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec03e450 con 0x7f57f40b2f70 2024-04-20T10:30:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.494+0000 7f57f1600640 1 -- 172.21.15.47:0/3836110039 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec009510 con 0x7f57f40b2f70 2024-04-20T10:30:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.494+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f40ca270 con 0x7f57f40b2f70 2024-04-20T10:30:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.494+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f40ca830 con 0x7f57f40b2f70 2024-04-20T10:30:20.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.496+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f4075b60 con 0x7f57f40b2f70 2024-04-20T10:30:20.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.497+0000 7f57f1600640 1 -- 172.21.15.47:0/3836110039 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57ec046020 con 0x7f57f40b2f70 2024-04-20T10:30:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.498+0000 7f57f1600640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.498+0000 7f57f1600640 1 -- 172.21.15.47:0/3836110039 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5619+0+0 (secure 0 0 0) 0x7f57ec0ae720 con 0x7f57f40b2f70 2024-04-20T10:30:20.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.499+0000 7f57f9600640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:20.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.500+0000 7f57f9600640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f57e8006fd0 tx=0x7f57e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.502+0000 7f57f1600640 1 -- 172.21.15.47:0/3836110039 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f57ec04d030 con 0x7f57f40b2f70 2024-04-20T10:30:20.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:20.709+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f57f415faa0 con 0x7f57f40b2f70 2024-04-20T10:30:21.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.260+0000 7f57f1600640 1 -- 172.21.15.47:0/3836110039 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) ==== 170+0+0 (secure 0 0 0) 0x7f57ec07cca0 con 0x7f57f40b2f70 2024-04-20T10:30:21.260 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' created 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 msgr2=0x7f57d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f57e8006fd0 tx=0x7f57e8008040 comp rx=0 tx=0).stop 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f40b2f70 msgr2=0x7f57f415b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f40b2f70 0x7f57f415b7f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009f40 tx=0x7f57ec004a40 comp rx=0 tx=0).stop 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 shutdown_connections 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f40b2f70 0x7f57f415b7f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:21.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f40adf60 0x7f57f409ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 --2- 172.21.15.47:0/3836110039 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f406b900 0x7f57f4151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.265+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 >> 172.21.15.47:0/3836110039 conn(0x7f57f40d0040 msgr2=0x7f57f40b3ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.266+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 shutdown_connections 2024-04-20T10:30:21.266 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.266+0000 7f57faa00640 1 -- 172.21.15.47:0/3836110039 wait complete. 2024-04-20T10:30:21.300 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-20T10:30:21.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.479+0000 7f4982800640 1 Processor -- start 2024-04-20T10:30:21.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.480+0000 7f4982800640 1 -- start start 2024-04-20T10:30:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.481+0000 7f4982800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c15e9c0 0x7f497c0d1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.481+0000 7f4982800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c0d23b0 con 0x7f497c0d4930 2024-04-20T10:30:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.481+0000 7f4982800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f497c0d2690 con 0x7f497c0b11c0 2024-04-20T10:30:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.481+0000 7f4982800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0d0f20 con 0x7f497c15e9c0 2024-04-20T10:30:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.482+0000 7f497be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f497c0d4930 0x7f497c0d3170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45534/0 (socket says 172.21.15.47:45534) 2024-04-20T10:30:21.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.482+0000 7f497be00640 1 -- 172.21.15.47:0/1606118571 learned_addr learned my addr 172.21.15.47:0/1606118571 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.482+0000 7f497b400640 1 --2- 172.21.15.47:0/1606118571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c15e9c0 0x7f497c0d1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.482+0000 7f497a000640 1 -- 172.21.15.47:0/1606118571 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1436205048 0 0) 0x7f497c0d23b0 con 0x7f497c0d4930 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.482+0000 7f497a000640 1 -- 172.21.15.47:0/1606118571 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f4960003520 con 0x7f497c0d4930 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.483+0000 7f497a000640 1 -- 172.21.15.47:0/1606118571 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1951191159 0 0) 0x7f497c0d2690 con 0x7f497c0b11c0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.483+0000 7f497a000640 1 -- 172.21.15.47:0/1606118571 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f497c0d23b0 con 0x7f497c0b11c0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.483+0000 7f497b400640 1 -- 172.21.15.47:0/1606118571 >> v1:172.21.15.175:6789/0 conn(0x7f497c0b11c0 legacy=0x7f497c0d0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.483+0000 7f497b400640 1 -- 172.21.15.47:0/1606118571 >> v1:172.21.15.47:6789/0 conn(0x7f497c0d4930 legacy=0x7f497c0d3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.483+0000 7f497b400640 1 -- 172.21.15.47:0/1606118571 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c0d2970 con 0x7f497c15e9c0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.484+0000 7f497b400640 1 --2- 172.21.15.47:0/1606118571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c15e9c0 0x7f497c0d1ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f496c009dc0 tx=0x7f496c02be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b5759ba3a2acd85 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f497a000640 1 -- 172.21.15.47:0/1606118571 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c039070 con 0x7f497c15e9c0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f497a000640 1 -- 172.21.15.47:0/1606118571 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f496c02cc30 con 0x7f497c15e9c0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f497a000640 1 -- 172.21.15.47:0/1606118571 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f496c034440 con 0x7f497c15e9c0 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f4982800640 1 -- 172.21.15.47:0/1606118571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c15e9c0 msgr2=0x7f497c0d1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:21.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f4982800640 1 --2- 172.21.15.47:0/1606118571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c15e9c0 0x7f497c0d1ba0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f496c009dc0 tx=0x7f496c02be80 comp rx=0 tx=0).stop 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f4982800640 1 -- 172.21.15.47:0/1606118571 shutdown_connections 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f4982800640 1 --2- 172.21.15.47:0/1606118571 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c15e9c0 0x7f497c0d1ba0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f4982800640 1 -- 172.21.15.47:0/1606118571 >> 172.21.15.47:0/1606118571 conn(0x7f497c0b9600 msgr2=0x7f497c0b9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f4982800640 1 -- 172.21.15.47:0/1606118571 shutdown_connections 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.485+0000 7f4982800640 1 -- 172.21.15.47:0/1606118571 wait complete. 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.487+0000 7f4982800640 1 Processor -- start 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.487+0000 7f4982800640 1 -- start start 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.488+0000 7f4982800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f497c0b11c0 0x7f497c161480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.488+0000 7f4982800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c0d4930 0x7f497c0c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.488+0000 7f4982800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f497c15e9c0 0x7f497c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.488+0000 7f4982800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0d51f0 con 0x7f497c15e9c0 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.488+0000 7f4982800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f497c0d5070 con 0x7f497c0b11c0 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.488+0000 7f4982800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f497c0d2ce0 con 0x7f497c0d4930 2024-04-20T10:30:21.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f497c15e9c0 0x7f497c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f497c15e9c0 0x7f497c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55758/0 (socket says 172.21.15.47:55758) 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 -- 172.21.15.47:0/4273066230 learned_addr learned my addr 172.21.15.47:0/4273066230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497b400640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f497c0b11c0 0x7f497c161480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497aa00640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c0d4930 0x7f497c0c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 -- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c0d4930 msgr2=0x7f497c0c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c0d4930 0x7f497c0c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 -- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f497c0b11c0 msgr2=0x7f497c161480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f497c0b11c0 0x7f497c161480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 -- 172.21.15.47:0/4273066230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f497c0c8450 con 0x7f497c15e9c0 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.489+0000 7f497be00640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f497c15e9c0 0x7f497c0cceb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f497400c460 tx=0x7f497400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.490+0000 7f4973400640 1 -- 172.21.15.47:0/4273066230 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974017070 con 0x7f497c15e9c0 2024-04-20T10:30:21.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.490+0000 7f4973400640 1 -- 172.21.15.47:0/4273066230 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4974005ce0 con 0x7f497c15e9c0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.490+0000 7f4973400640 1 -- 172.21.15.47:0/4273066230 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974007290 con 0x7f497c15e9c0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.490+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f497c0c8730 con 0x7f497c15e9c0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.490+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f497c0c8cf0 con 0x7f497c15e9c0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.491+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f497c068980 con 0x7f497c15e9c0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.492+0000 7f4973400640 1 -- 172.21.15.47:0/4273066230 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f497401a070 con 0x7f497c15e9c0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.493+0000 7f4973400640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f495806c8f0 0x7f495806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.493+0000 7f4973400640 1 -- 172.21.15.47:0/4273066230 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 6001+0+0 (secure 0 0 0) 0x7f497408b100 con 0x7f497c15e9c0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.494+0000 7f497b400640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f495806c8f0 0x7f495806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.495+0000 7f497b400640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f495806c8f0 0x7f495806edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f496c036000 tx=0x7f496c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:21.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.498+0000 7f4973400640 1 -- 172.21.15.47:0/4273066230 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4974059360 con 0x7f497c15e9c0 2024-04-20T10:30:21.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:21.743+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f497c0b2830 con 0x7f497c15e9c0 2024-04-20T10:30:22.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.314+0000 7f4973400640 1 -- 172.21.15.47:0/4273066230 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v29) ==== 172+0+0 (secure 0 0 0) 0x7f4974058c60 con 0x7f497c15e9c0 2024-04-20T10:30:22.315 INFO:teuthology.orchestra.run.smithi047.stderr:new fs with metadata pool 7 and data pool 8 2024-04-20T10:30:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f495806c8f0 msgr2=0x7f495806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f495806c8f0 0x7f495806edb0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f496c036000 tx=0x7f496c030000 comp rx=0 tx=0).stop 2024-04-20T10:30:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f497c15e9c0 msgr2=0x7f497c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f497c15e9c0 0x7f497c0cceb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f497400c460 tx=0x7f497400c920 comp rx=0 tx=0).stop 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 shutdown_connections 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f495806c8f0 0x7f495806edb0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f497c15e9c0 0x7f497c0cceb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f497c0d4930 0x7f497c0c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 --2- 172.21.15.47:0/4273066230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f497c0b11c0 0x7f497c161480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.327+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 >> 172.21.15.47:0/4273066230 conn(0x7f497c0b9600 msgr2=0x7f497c0d16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.328+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 shutdown_connections 2024-04-20T10:30:22.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.328+0000 7f4982800640 1 -- 172.21.15.47:0/4273066230 wait complete. 2024-04-20T10:30:22.358 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.531+0000 7f5afee00640 1 Processor -- start 2024-04-20T10:30:22.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.532+0000 7f5afee00640 1 -- start start 2024-04-20T10:30:22.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.533+0000 7f5afee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4270 0x7f5af80d1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.533 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.533+0000 7f5afee00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f5af80d4650 0x7f5af80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.533+0000 7f5afee00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5af80b17d0 0x7f5af80d0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.533+0000 7f5afee00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f5af80d40c0 con 0x7f5af80b17d0 2024-04-20T10:30:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.533+0000 7f5afee00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5af80d3820 con 0x7f5af80d4650 2024-04-20T10:30:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.533+0000 7f5afee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5af80d3b00 con 0x7f5af80d4270 2024-04-20T10:30:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5af80b17d0 0x7f5af80d0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f5af80b17d0 0x7f5af80d0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55772/0 (socket says 172.21.15.47:55772) 2024-04-20T10:30:22.534 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 -- 172.21.15.47:0/2199476227 learned_addr learned my addr 172.21.15.47:0/2199476227 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5aefe00640 1 --2- 172.21.15.47:0/2199476227 >> v2:172.21.15.175:3300/0 conn(0x7f5af80d4650 0x7f5af80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5af7e00640 1 --2- 172.21.15.47:0/2199476227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4270 0x7f5af80d1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 -- 172.21.15.47:0/2199476227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4270 msgr2=0x7f5af80d1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 --2- 172.21.15.47:0/2199476227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4270 0x7f5af80d1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 -- 172.21.15.47:0/2199476227 >> v2:172.21.15.175:3300/0 conn(0x7f5af80d4650 msgr2=0x7f5af80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 --2- 172.21.15.47:0/2199476227 >> v2:172.21.15.175:3300/0 conn(0x7f5af80d4650 0x7f5af80b1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5afce00640 1 -- 172.21.15.47:0/2199476227 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af80d3de0 con 0x7f5af80b17d0 2024-04-20T10:30:22.535 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.534+0000 7f5af7e00640 1 --2- 172.21.15.47:0/2199476227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4270 0x7f5af80d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.535+0000 7f5afce00640 1 --2- 172.21.15.47:0/2199476227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80b17d0 0x7f5af80d0380 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5af80683c0 tx=0x7f5af002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8130e1df5bf3f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.536+0000 7f5af7400640 1 -- 172.21.15.47:0/2199476227 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af002ca70 con 0x7f5af80b17d0 2024-04-20T10:30:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.536+0000 7f5af7400640 1 -- 172.21.15.47:0/2199476227 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af002cc10 con 0x7f5af80b17d0 2024-04-20T10:30:22.536 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.536+0000 7f5af7400640 1 -- 172.21.15.47:0/2199476227 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5af00343f0 con 0x7f5af80b17d0 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.536+0000 7f5afee00640 1 -- 172.21.15.47:0/2199476227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80b17d0 msgr2=0x7f5af80d0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.536+0000 7f5afee00640 1 --2- 172.21.15.47:0/2199476227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80b17d0 0x7f5af80d0380 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5af80683c0 tx=0x7f5af002bed0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.537+0000 7f5afee00640 1 -- 172.21.15.47:0/2199476227 shutdown_connections 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.537+0000 7f5afee00640 1 --2- 172.21.15.47:0/2199476227 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80b17d0 0x7f5af80d0380 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.537+0000 7f5afee00640 1 --2- 172.21.15.47:0/2199476227 >> v2:172.21.15.175:3300/0 conn(0x7f5af80d4650 0x7f5af80b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.537+0000 7f5afee00640 1 --2- 172.21.15.47:0/2199476227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4270 0x7f5af80d1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.537+0000 7f5afee00640 1 -- 172.21.15.47:0/2199476227 >> 172.21.15.47:0/2199476227 conn(0x7f5af80b99c0 msgr2=0x7f5af80b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.537+0000 7f5afee00640 1 -- 172.21.15.47:0/2199476227 shutdown_connections 2024-04-20T10:30:22.537 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.537+0000 7f5afee00640 1 -- 172.21.15.47:0/2199476227 wait complete. 2024-04-20T10:30:22.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.539+0000 7f5afee00640 1 Processor -- start 2024-04-20T10:30:22.539 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.539+0000 7f5afee00640 1 -- start start 2024-04-20T10:30:22.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.540+0000 7f5afee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af80b17d0 0x7f5af81f89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5afee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80d4270 0x7f5af806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5afee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4650 0x7f5af80ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5afee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5af8161f70 con 0x7f5af80d4270 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5afee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5af80d53b0 con 0x7f5af80b17d0 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5afee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5af81620f0 con 0x7f5af80d4650 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80d4270 0x7f5af806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80d4270 0x7f5af806e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55786/0 (socket says 172.21.15.47:55786) 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 -- 172.21.15.47:0/315673071 learned_addr learned my addr 172.21.15.47:0/315673071 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5af7e00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af80b17d0 0x7f5af81f89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 -- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4650 msgr2=0x7f5af80ca040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4650 0x7f5af80ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 -- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af80b17d0 msgr2=0x7f5af81f89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af80b17d0 0x7f5af81f89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.541+0000 7f5aefe00640 1 -- 172.21.15.47:0/315673071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af80caac0 con 0x7f5af80d4270 2024-04-20T10:30:22.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.542+0000 7f5aefe00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80d4270 0x7f5af806e360 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d830 tx=0x7f5ae000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:22.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.542+0000 7f5af4c00640 1 -- 172.21.15.47:0/315673071 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000b6c0 con 0x7f5af80d4270 2024-04-20T10:30:22.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.542+0000 7f5af4c00640 1 -- 172.21.15.47:0/315673071 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5ae0019070 con 0x7f5af80d4270 2024-04-20T10:30:22.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.542+0000 7f5af4c00640 1 -- 172.21.15.47:0/315673071 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000ba80 con 0x7f5af80d4270 2024-04-20T10:30:22.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.542+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af80ca7e0 con 0x7f5af80d4270 2024-04-20T10:30:22.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.542+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af81f81c0 con 0x7f5af80d4270 2024-04-20T10:30:22.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.544+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5af80c78e0 con 0x7f5af80d4270 2024-04-20T10:30:22.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.544+0000 7f5af4c00640 1 -- 172.21.15.47:0/315673071 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5ae001e460 con 0x7f5af80d4270 2024-04-20T10:30:22.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.545+0000 7f5af4c00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ad406c920 0x7f5ad406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.545+0000 7f5af4c00640 1 -- 172.21.15.47:0/315673071 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f5ae008c280 con 0x7f5af80d4270 2024-04-20T10:30:22.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.546+0000 7f5af7e00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ad406c920 0x7f5ad406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.547 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.547+0000 7f5af7e00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ad406c920 0x7f5ad406ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5af80d0d50 tx=0x7f5ae8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:22.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.550+0000 7f5af4c00640 1 -- 172.21.15.47:0/315673071 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f5af80c78e0 con 0x7f5af80d4270 2024-04-20T10:30:22.756 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.756+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5af8068980 con 0x7f5af80d4270 2024-04-20T10:30:22.760 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.760+0000 7f5af4c00640 1 -- 172.21.15.47:0/315673071 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16907 (secure 0 0 0) 0x7f5ae005a5b0 con 0x7f5af80d4270 2024-04-20T10:30:22.760 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:22.761 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:21.745554+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:30:19.610070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-20T10:30:20.722858+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.762+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ad406c920 msgr2=0x7f5ad406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.762+0000 7f5afee00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ad406c920 0x7f5ad406ede0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5af80d0d50 tx=0x7f5ae8008040 comp rx=0 tx=0).stop 2024-04-20T10:30:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.762+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80d4270 msgr2=0x7f5af806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80d4270 0x7f5af806e360 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d830 tx=0x7f5ae000dcf0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 shutdown_connections 2024-04-20T10:30:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5af80d4650 0x7f5af80ca040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f5ad406c920 0x7f5ad406ede0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f5af80d4270 0x7f5af806e360 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 --2- 172.21.15.47:0/315673071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5af80b17d0 0x7f5af81f89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 >> 172.21.15.47:0/315673071 conn(0x7f5af80b99c0 msgr2=0x7f5af8161130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 shutdown_connections 2024-04-20T10:30:22.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.763+0000 7f5afee00640 1 -- 172.21.15.47:0/315673071 wait complete. 2024-04-20T10:30:22.782 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.983+0000 7f93ee600640 1 Processor -- start 2024-04-20T10:30:22.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.983+0000 7f93ee600640 1 -- start start 2024-04-20T10:30:22.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.984+0000 7f93ee600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 0x7f93e806b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.984+0000 7f93ee600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f93e815ca40 0x7f93e80ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.984+0000 7f93ee600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f93e8141c60 con 0x7f93e80ad8e0 2024-04-20T10:30:22.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.984+0000 7f93ee600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f93e815cf90 con 0x7f93e815ca40 2024-04-20T10:30:22.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.984+0000 7f93ee600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93e815d270 con 0x7f93e8156c20 2024-04-20T10:30:22.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.985+0000 7f93e7e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f93e80ad8e0 0x7f93e80d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:45550/0 (socket says 172.21.15.47:45550) 2024-04-20T10:30:22.985 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.985+0000 7f93e7e00640 1 -- 172.21.15.47:0/4209487715 learned_addr learned my addr 172.21.15.47:0/4209487715 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.985+0000 7f93e7400640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 0x7f93e806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.985+0000 7f93dea00640 1 --2- 172.21.15.47:0/4209487715 >> v2:172.21.15.175:3300/0 conn(0x7f93e815ca40 0x7f93e80ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.985+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 203966117 0 0) 0x7f93e8141c60 con 0x7f93e80ad8e0 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.985+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f93cc003520 con 0x7f93e80ad8e0 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3462644636 0 0) 0x7f93cc003520 con 0x7f93e80ad8e0 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 msgr2=0x7f93e806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 0x7f93e806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 >> v2:172.21.15.175:3300/0 conn(0x7f93e815ca40 msgr2=0x7f93e80ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 --2- 172.21.15.47:0/4209487715 >> v2:172.21.15.175:3300/0 conn(0x7f93e815ca40 0x7f93e80ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e815d550 con 0x7f93e80ad8e0 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f93e00019f0 con 0x7f93e80ad8e0 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 >> v1:172.21.15.47:6789/0 conn(0x7f93e80ad8e0 legacy=0x7f93e80d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 0x7f93cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93cc003f30 0x7f93cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93cc004d40 0x7f93cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93e815cf90 con 0x7f93cc003f30 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93e815d270 con 0x7f93cc004d40 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93cc00ba10 con 0x7f93e8156c20 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93dea00640 1 --2- 172.21.15.47:0/4209487715 >> v2:172.21.15.175:3300/0 conn(0x7f93e815ca40 0x7f93e80ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.986+0000 7f93dea00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93cc003f30 0x7f93cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.987+0000 7f93e7400640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 0x7f93cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.987+0000 7f93e7e00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93cc004d40 0x7f93cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.987+0000 7f93dea00640 1 -- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 msgr2=0x7f93cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.987+0000 7f93dea00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93e8156c20 0x7f93cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.987+0000 7f93dea00640 1 -- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93cc004d40 msgr2=0x7f93cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.987+0000 7f93dea00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93cc004d40 0x7f93cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.987+0000 7f93dea00640 1 -- 172.21.15.47:0/4209487715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93cc003520 con 0x7f93cc003f30 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.988+0000 7f93dea00640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93cc003f30 0x7f93cc004800 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f93d00098a0 tx=0x7f93cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f4bb425e928884 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.988+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d000f040 con 0x7f93cc003f30 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.988+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93d0037600 con 0x7f93cc003f30 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.988+0000 7f93e6a00640 1 -- 172.21.15.47:0/4209487715 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d00379b0 con 0x7f93cc003f30 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.989+0000 7f93ee600640 1 -- 172.21.15.47:0/4209487715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93cc003f30 msgr2=0x7f93cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.989+0000 7f93ee600640 1 --2- 172.21.15.47:0/4209487715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93cc003f30 0x7f93cc004800 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f93d00098a0 tx=0x7f93cc005830 comp rx=0 tx=0).stop 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.989+0000 7f93e7e00640 1 -- 172.21.15.47:0/4209487715 reap_dead start 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.989+0000 7f93ee600640 1 -- 172.21.15.47:0/4209487715 shutdown_connections 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.989+0000 7f93ee600640 1 -- 172.21.15.47:0/4209487715 >> 172.21.15.47:0/4209487715 conn(0x7f93e80b3350 msgr2=0x7f93e80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:22.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.989+0000 7f93ee600640 1 -- 172.21.15.47:0/4209487715 shutdown_connections 2024-04-20T10:30:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.989+0000 7f93ee600640 1 -- 172.21.15.47:0/4209487715 wait complete. 2024-04-20T10:30:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.991+0000 7f93ee600640 1 Processor -- start 2024-04-20T10:30:22.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.992+0000 7f93ee600640 1 -- start start 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93ee600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93cc003f30 0x7f93e806e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93ee600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e80cd440 0x7f93e806f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93ee600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e809dda0 0x7f93e806bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93ee600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80d0bf0 con 0x7f93e809dda0 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93ee600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93e80d0a70 con 0x7f93e80cd440 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93ee600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93e806b900 con 0x7f93cc003f30 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e809dda0 0x7f93e806bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93e7e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e809dda0 0x7f93e806bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55798/0 (socket says 172.21.15.47:55798) 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.993+0000 7f93e7e00640 1 -- 172.21.15.47:0/169488110 learned_addr learned my addr 172.21.15.47:0/169488110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:22.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93dea00640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e80cd440 0x7f93e806f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7400640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93cc003f30 0x7f93e806e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7e00640 1 -- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93cc003f30 msgr2=0x7f93e806e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7e00640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93cc003f30 0x7f93e806e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7e00640 1 -- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e80cd440 msgr2=0x7f93e806f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7e00640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e80cd440 0x7f93e806f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7e00640 1 -- 172.21.15.47:0/169488110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e806e7a0 con 0x7f93e809dda0 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93dea00640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e80cd440 0x7f93e806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7400640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93cc003f30 0x7f93e806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:22.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93e7e00640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e809dda0 0x7f93e806bce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000f80 tx=0x7f93e000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93dfe00640 1 -- 172.21.15.47:0/169488110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e000eb30 con 0x7f93e809dda0 2024-04-20T10:30:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93dfe00640 1 -- 172.21.15.47:0/169488110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f93e000ecf0 con 0x7f93e809dda0 2024-04-20T10:30:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93dfe00640 1 -- 172.21.15.47:0/169488110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e000eea0 con 0x7f93e809dda0 2024-04-20T10:30:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.994+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e806ea80 con 0x7f93e809dda0 2024-04-20T10:30:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.995+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e806f040 con 0x7f93e809dda0 2024-04-20T10:30:22.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.996+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e8068980 con 0x7f93e809dda0 2024-04-20T10:30:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.997+0000 7f93dfe00640 1 -- 172.21.15.47:0/169488110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f93e0014670 con 0x7f93e809dda0 2024-04-20T10:30:22.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.998+0000 7f93dfe00640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f93c406c920 0x7f93c406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:22.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.998+0000 7f93dfe00640 1 -- 172.21.15.47:0/169488110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 6101+0+0 (secure 0 0 0) 0x7f93e008baa0 con 0x7f93e809dda0 2024-04-20T10:30:22.998 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:22.998+0000 7f93e7400640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f93c406c920 0x7f93c406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:22.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.000+0000 7f93e7400640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f93c406c920 0x7f93c406ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006ce0 tx=0x7f93d8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:23.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.002+0000 7f93dfe00640 1 -- 172.21.15.47:0/169488110 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f93e0059d30 con 0x7f93e809dda0 2024-04-20T10:30:23.210 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.210+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f93e806dfa0 con 0x7f93e809dda0 2024-04-20T10:30:23.213 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.213+0000 7f93dfe00640 1 -- 172.21.15.47:0/169488110 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) ==== 74+0+16907 (secure 0 0 0) 0x7f93e0059630 con 0x7f93e809dda0 2024-04-20T10:30:23.214 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:23.214 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":53,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:21.745554+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:30:19.610070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-20T10:30:20.722858+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f93c406c920 msgr2=0x7f93c406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f93c406c920 0x7f93c406ede0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f93d8006ce0 tx=0x7f93d8008040 comp rx=0 tx=0).stop 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e809dda0 msgr2=0x7f93e806bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e809dda0 0x7f93e806bce0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f93e0000f80 tx=0x7f93e000e530 comp rx=0 tx=0).stop 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 shutdown_connections 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f93c406c920 0x7f93c406ede0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f93e809dda0 0x7f93e806bce0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.217 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93e80cd440 0x7f93e806f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 --2- 172.21.15.47:0/169488110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93cc003f30 0x7f93e806e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.216+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 >> 172.21.15.47:0/169488110 conn(0x7f93e80b3350 msgr2=0x7f93e815f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.217+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 shutdown_connections 2024-04-20T10:30:23.218 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.217+0000 7f93ee600640 1 -- 172.21.15.47:0/169488110 wait complete. 2024-04-20T10:30:23.234 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-20T10:30:23.251 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-04-20T10:30:23.251+0000 7ff74d000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:30:23.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.435+0000 7f28d7200640 1 Processor -- start 2024-04-20T10:30:23.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.435+0000 7f28d7200640 1 -- start start 2024-04-20T10:30:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.436+0000 7f28d7200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28d815d4f0 0x7f28d80ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.436+0000 7f28d7200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d806b900 con 0x7f28d80aee60 2024-04-20T10:30:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d7200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f28d806bbe0 con 0x7f28d806e060 2024-04-20T10:30:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d7200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f28d806c510 con 0x7f28d815d4f0 2024-04-20T10:30:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d5400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28d815d4f0 0x7f28d80ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d5400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f28d815d4f0 0x7f28d80ad0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55814/0 (socket says 172.21.15.47:55814) 2024-04-20T10:30:23.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d5400640 1 -- 172.21.15.47:0/1195724597 learned_addr learned my addr 172.21.15.47:0/1195724597 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d5400640 1 -- 172.21.15.47:0/1195724597 >> v1:172.21.15.175:6789/0 conn(0x7f28d806e060 legacy=0x7f28d806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d5400640 1 -- 172.21.15.47:0/1195724597 >> v1:172.21.15.47:6789/0 conn(0x7f28d80aee60 legacy=0x7f28d80af240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:23.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.437+0000 7f28d5400640 1 -- 172.21.15.47:0/1195724597 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d806bec0 con 0x7f28d815d4f0 2024-04-20T10:30:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.438+0000 7f28d5400640 1 --2- 172.21.15.47:0/1195724597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d815d4f0 0x7f28d80ad0f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009870 tx=0x7f28c402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74016a9641f5923e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.439+0000 7f28d4a00640 1 -- 172.21.15.47:0/1195724597 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4034070 con 0x7f28d815d4f0 2024-04-20T10:30:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.439+0000 7f28d4a00640 1 -- 172.21.15.47:0/1195724597 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c4030860 con 0x7f28d815d4f0 2024-04-20T10:30:23.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.439+0000 7f28d4a00640 1 -- 172.21.15.47:0/1195724597 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4030c30 con 0x7f28d815d4f0 2024-04-20T10:30:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.440+0000 7f28d7200640 1 -- 172.21.15.47:0/1195724597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d815d4f0 msgr2=0x7f28d80ad0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.440+0000 7f28d7200640 1 --2- 172.21.15.47:0/1195724597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d815d4f0 0x7f28d80ad0f0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f28c4009870 tx=0x7f28c402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.440+0000 7f28d7200640 1 -- 172.21.15.47:0/1195724597 shutdown_connections 2024-04-20T10:30:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.440+0000 7f28d7200640 1 --2- 172.21.15.47:0/1195724597 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d815d4f0 0x7f28d80ad0f0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.440+0000 7f28d7200640 1 -- 172.21.15.47:0/1195724597 >> 172.21.15.47:0/1195724597 conn(0x7f28d80b5270 msgr2=0x7f28d80b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.440+0000 7f28d7200640 1 -- 172.21.15.47:0/1195724597 shutdown_connections 2024-04-20T10:30:23.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.440+0000 7f28d7200640 1 -- 172.21.15.47:0/1195724597 wait complete. 2024-04-20T10:30:23.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.441+0000 7f28d7200640 1 Processor -- start 2024-04-20T10:30:23.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.442+0000 7f28d7200640 1 -- start start 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.442+0000 7f28d7200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d806e060 0x7f28d815e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.442+0000 7f28d7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28d80aee60 0x7f28d814c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d815d4f0 0x7f28d81e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d7200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f28d80ad630 con 0x7f28d806e060 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28d806c9a0 con 0x7f28d815d4f0 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28d80ad7b0 con 0x7f28d80aee60 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d806e060 0x7f28d815e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d806e060 0x7f28d815e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:55828/0 (socket says 172.21.15.47:55828) 2024-04-20T10:30:23.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 -- 172.21.15.47:0/1850058496 learned_addr learned my addr 172.21.15.47:0/1850058496 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 -- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28d80aee60 msgr2=0x7f28d814c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28d80aee60 0x7f28d814c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 -- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d815d4f0 msgr2=0x7f28d81e8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:23.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d815d4f0 0x7f28d81e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.443+0000 7f28d5e00640 1 -- 172.21.15.47:0/1850058496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d815ed90 con 0x7f28d806e060 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.444+0000 7f28d5e00640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d806e060 0x7f28d815e7b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c400 tx=0x7f28c800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.444+0000 7f28ce000640 1 -- 172.21.15.47:0/1850058496 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c80052d0 con 0x7f28d806e060 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.444+0000 7f28ce000640 1 -- 172.21.15.47:0/1850058496 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f28c8005490 con 0x7f28d806e060 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.444+0000 7f28ce000640 1 -- 172.21.15.47:0/1850058496 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c8007260 con 0x7f28d806e060 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.444+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d815f070 con 0x7f28d806e060 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.444+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d815f630 con 0x7f28d806e060 2024-04-20T10:30:23.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.445+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28d8075b30 con 0x7f28d806e060 2024-04-20T10:30:23.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.446+0000 7f28ce000640 1 -- 172.21.15.47:0/1850058496 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f28c8007400 con 0x7f28d806e060 2024-04-20T10:30:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.447+0000 7f28ce000640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28b806c8f0 0x7f28b806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:23.447 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.447+0000 7f28ce000640 1 -- 172.21.15.47:0/1850058496 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f28c808b280 con 0x7f28d806e060 2024-04-20T10:30:23.448 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.448+0000 7f28d5400640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28b806c8f0 0x7f28b806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:23.449 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.449+0000 7f28d5400640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28b806c8f0 0x7f28b806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f28c4036000 tx=0x7f28c403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:23.452 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.452+0000 7f28ce000640 1 -- 172.21.15.47:0/1850058496 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f28c8059510 con 0x7f28d806e060 2024-04-20T10:30:23.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:23.697+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f28d806ceb0 con 0x7f28d806e060 2024-04-20T10:30:24.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.305+0000 7f28ce000640 1 -- 172.21.15.47:0/1850058496 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v31) ==== 118+0+0 (secure 0 0 0) 0x7f28c8058e10 con 0x7f28d806e060 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.309+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28b806c8f0 msgr2=0x7f28b806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.309+0000 7f28d7200640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28b806c8f0 0x7f28b806edb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f28c4036000 tx=0x7f28c403b040 comp rx=0 tx=0).stop 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.309+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d806e060 msgr2=0x7f28d815e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.309+0000 7f28d7200640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d806e060 0x7f28d815e7b0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f28c800c400 tx=0x7f28c800c8c0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.309+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 shutdown_connections 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.310+0000 7f28d7200640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28d815d4f0 0x7f28d81e8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.310+0000 7f28d7200640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28d80aee60 0x7f28d814c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.310+0000 7f28d7200640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f28b806c8f0 0x7f28b806edb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.310+0000 7f28d7200640 1 --2- 172.21.15.47:0/1850058496 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f28d806e060 0x7f28d815e7b0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.310+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 >> 172.21.15.47:0/1850058496 conn(0x7f28d80b5270 msgr2=0x7f28d806e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:24.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.310+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 shutdown_connections 2024-04-20T10:30:24.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.310+0000 7f28d7200640 1 -- 172.21.15.47:0/1850058496 wait complete. 2024-04-20T10:30:24.333 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-20T10:30:24.334 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-20T10:30:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.596+0000 7f6044400640 1 Processor -- start 2024-04-20T10:30:24.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.596+0000 7f6044400640 1 -- start start 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.597+0000 7f6044400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f603c15e8a0 0x7f603c06e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.597+0000 7f6044400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f603c0d0a70 0x7f603c0ad680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.597+0000 7f6044400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f603c06f060 con 0x7f603c06dbb0 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.597+0000 7f6044400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f603c06e7c0 con 0x7f603c15e8a0 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.597+0000 7f6044400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f603c06eaa0 con 0x7f603c0d0a70 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6041a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f603c15e8a0 0x7f603c06e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6041a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f603c15e8a0 0x7f603c06e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41102/0 (socket says 172.21.15.47:41102) 2024-04-20T10:30:24.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6041a00640 1 -- 172.21.15.47:0/775396005 learned_addr learned my addr 172.21.15.47:0/775396005 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6042400640 1 --2- 172.21.15.47:0/775396005 >> v2:172.21.15.175:3300/0 conn(0x7f603c0d0a70 0x7f603c0ad680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6041a00640 1 -- 172.21.15.47:0/775396005 >> v2:172.21.15.175:3300/0 conn(0x7f603c0d0a70 msgr2=0x7f603c0ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6041a00640 1 --2- 172.21.15.47:0/775396005 >> v2:172.21.15.175:3300/0 conn(0x7f603c0d0a70 0x7f603c0ad680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6041a00640 1 -- 172.21.15.47:0/775396005 >> v1:172.21.15.175:6789/0 conn(0x7f603c06dbb0 legacy=0x7f603c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:24.599 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.598+0000 7f6041a00640 1 -- 172.21.15.47:0/775396005 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c06ed80 con 0x7f603c15e8a0 2024-04-20T10:30:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.600+0000 7f6041a00640 1 --2- 172.21.15.47:0/775396005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c06e1e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f603c0d0f70 tx=0x7f602c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374977a493767f0f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.600+0000 7f603be00640 1 -- 172.21.15.47:0/775396005 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c02ca80 con 0x7f603c15e8a0 2024-04-20T10:30:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.600+0000 7f603be00640 1 -- 172.21.15.47:0/775396005 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f602c02cc20 con 0x7f603c15e8a0 2024-04-20T10:30:24.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.600+0000 7f603be00640 1 -- 172.21.15.47:0/775396005 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0343f0 con 0x7f603c15e8a0 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 -- 172.21.15.47:0/775396005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 msgr2=0x7f603c06e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 --2- 172.21.15.47:0/775396005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c06e1e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f603c0d0f70 tx=0x7f602c02bee0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 -- 172.21.15.47:0/775396005 shutdown_connections 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 --2- 172.21.15.47:0/775396005 >> v2:172.21.15.175:3300/0 conn(0x7f603c0d0a70 0x7f603c0ad680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 --2- 172.21.15.47:0/775396005 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c06e1e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 -- 172.21.15.47:0/775396005 >> 172.21.15.47:0/775396005 conn(0x7f603c0b3350 msgr2=0x7f603c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 -- 172.21.15.47:0/775396005 shutdown_connections 2024-04-20T10:30:24.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.601+0000 7f6044400640 1 -- 172.21.15.47:0/775396005 wait complete. 2024-04-20T10:30:24.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.603+0000 7f6044400640 1 Processor -- start 2024-04-20T10:30:24.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.603+0000 7f6044400640 1 -- start start 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.604+0000 7f6044400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c06dbb0 0x7f603c099a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.604+0000 7f6044400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f603c0d0a70 0x7f603c14dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.604+0000 7f6044400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c1603b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.604+0000 7f6044400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f603c06fef0 con 0x7f603c15e8a0 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.604+0000 7f6044400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f603c06fd70 con 0x7f603c06dbb0 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.604+0000 7f6044400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f603c0adbc0 con 0x7f603c0d0a70 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6042400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c1603b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6042400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c1603b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41108/0 (socket says 172.21.15.47:41108) 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6042400640 1 -- 172.21.15.47:0/4123062581 learned_addr learned my addr 172.21.15.47:0/4123062581 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:24.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6041a00640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c06dbb0 0x7f603c099a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6041000640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f603c0d0a70 0x7f603c14dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6042400640 1 -- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f603c0d0a70 msgr2=0x7f603c14dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6042400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f603c0d0a70 0x7f603c14dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6042400640 1 -- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c06dbb0 msgr2=0x7f603c099a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.605+0000 7f6042400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c06dbb0 0x7f603c099a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.606+0000 7f6042400640 1 -- 172.21.15.47:0/4123062581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603c150e90 con 0x7f603c15e8a0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.606+0000 7f6042400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c1603b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f603400d4f0 tx=0x7f603400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.606+0000 7f6039600640 1 -- 172.21.15.47:0/4123062581 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340125f0 con 0x7f603c15e8a0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.606+0000 7f6039600640 1 -- 172.21.15.47:0/4123062581 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6034019070 con 0x7f603c15e8a0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.606+0000 7f6039600640 1 -- 172.21.15.47:0/4123062581 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60340129f0 con 0x7f603c15e8a0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.606+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603c152cb0 con 0x7f603c15e8a0 2024-04-20T10:30:24.607 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.606+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f603c150a20 con 0x7f603c15e8a0 2024-04-20T10:30:24.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.607+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f603c068980 con 0x7f603c15e8a0 2024-04-20T10:30:24.608 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.608+0000 7f6039600640 1 -- 172.21.15.47:0/4123062581 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6034012b90 con 0x7f603c15e8a0 2024-04-20T10:30:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.609+0000 7f6039600640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f601c06c8f0 0x7f601c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:24.609 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.610+0000 7f6039600640 1 -- 172.21.15.47:0/4123062581 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f603408bff0 con 0x7f603c15e8a0 2024-04-20T10:30:24.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.610+0000 7f6041a00640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f601c06c8f0 0x7f601c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:24.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.611+0000 7f6041a00640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f601c06c8f0 0x7f601c06edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f602c035000 tx=0x7f602c030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:24.614 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.614+0000 7f6039600640 1 -- 172.21.15.47:0/4123062581 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f603405a2e0 con 0x7f603c15e8a0 2024-04-20T10:30:24.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:24.858+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f603c06d160 con 0x7f603c15e8a0 2024-04-20T10:30:25.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.309+0000 7f6039600640 1 -- 172.21.15.47:0/4123062581 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f6034059be0 con 0x7f603c15e8a0 2024-04-20T10:30:25.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f601c06c8f0 msgr2=0x7f601c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f601c06c8f0 0x7f601c06edb0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f602c035000 tx=0x7f602c030000 comp rx=0 tx=0).stop 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 msgr2=0x7f603c1603b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c1603b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f603400d4f0 tx=0x7f603400d9b0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 shutdown_connections 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f601c06c8f0 0x7f601c06edb0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f603c15e8a0 0x7f603c1603b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f603c0d0a70 0x7f603c14dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 --2- 172.21.15.47:0/4123062581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f603c06dbb0 0x7f603c099a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.313+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 >> 172.21.15.47:0/4123062581 conn(0x7f603c0b3350 msgr2=0x7f603c15da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.314+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 shutdown_connections 2024-04-20T10:30:25.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.314+0000 7f6044400640 1 -- 172.21.15.47:0/4123062581 wait complete. 2024-04-20T10:30:25.338 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-20T10:30:25.338 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:25.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.549+0000 7f225a800640 1 Processor -- start 2024-04-20T10:30:25.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.550+0000 7f225a800640 1 -- start start 2024-04-20T10:30:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.551+0000 7f225a800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2254156c20 0x7f225406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:25.551 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.551+0000 7f225a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225415ca40 0x7f22540ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.551+0000 7f225a800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2254141c60 con 0x7f22540ad8e0 2024-04-20T10:30:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.551+0000 7f225a800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f225415cf90 con 0x7f2254156c20 2024-04-20T10:30:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.551+0000 7f225a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f225415d270 con 0x7f225415ca40 2024-04-20T10:30:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.551+0000 7f2253400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2254156c20 0x7f225406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.551+0000 7f2253e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f22540ad8e0 0x7f22540d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53434/0 (socket says 172.21.15.47:53434) 2024-04-20T10:30:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f2253400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2254156c20 0x7f225406b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41114/0 (socket says 172.21.15.47:41114) 2024-04-20T10:30:25.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f2253e00640 1 -- 172.21.15.47:0/314561291 learned_addr learned my addr 172.21.15.47:0/314561291 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:25.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f2253400640 1 -- 172.21.15.47:0/314561291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225415ca40 msgr2=0x7f22540ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:25.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f224be00640 1 --2- 172.21.15.47:0/314561291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225415ca40 0x7f22540ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:25.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f2253400640 1 --2- 172.21.15.47:0/314561291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225415ca40 0x7f22540ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f2253400640 1 -- 172.21.15.47:0/314561291 >> v1:172.21.15.47:6789/0 conn(0x7f22540ad8e0 legacy=0x7f22540d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f2253400640 1 -- 172.21.15.47:0/314561291 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225415d550 con 0x7f2254156c20 2024-04-20T10:30:25.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.552+0000 7f224be00640 1 --2- 172.21.15.47:0/314561291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225415ca40 0x7f22540ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:25.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.553+0000 7f2253400640 1 --2- 172.21.15.47:0/314561291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254156c20 0x7f225406b900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f22440097a0 tx=0x7f224402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e844fbde7a005 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:25.554 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.553+0000 7f2252a00640 1 -- 172.21.15.47:0/314561291 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244035020 con 0x7f2254156c20 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f2252a00640 1 -- 172.21.15.47:0/314561291 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2244031810 con 0x7f2254156c20 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f2252a00640 1 -- 172.21.15.47:0/314561291 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2244031be0 con 0x7f2254156c20 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 -- 172.21.15.47:0/314561291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254156c20 msgr2=0x7f225406b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 --2- 172.21.15.47:0/314561291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254156c20 0x7f225406b900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f22440097a0 tx=0x7f224402b8b0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 -- 172.21.15.47:0/314561291 shutdown_connections 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 --2- 172.21.15.47:0/314561291 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225415ca40 0x7f22540ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 --2- 172.21.15.47:0/314561291 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2254156c20 0x7f225406b900 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 -- 172.21.15.47:0/314561291 >> 172.21.15.47:0/314561291 conn(0x7f22540b3350 msgr2=0x7f22540b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 -- 172.21.15.47:0/314561291 shutdown_connections 2024-04-20T10:30:25.555 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.554+0000 7f225a800640 1 -- 172.21.15.47:0/314561291 wait complete. 2024-04-20T10:30:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.556+0000 7f225a800640 1 Processor -- start 2024-04-20T10:30:25.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.556+0000 7f225a800640 1 -- start start 2024-04-20T10:30:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.557+0000 7f225a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22540ad8e0 0x7f225414b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.557+0000 7f225a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2254156c20 0x7f22540732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.557+0000 7f225a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f225415ca40 0x7f225414e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.557+0000 7f225a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f225406bfc0 con 0x7f225415ca40 2024-04-20T10:30:25.558 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.557+0000 7f225a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f225406be40 con 0x7f22540ad8e0 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.557+0000 7f225a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22540d0a70 con 0x7f2254156c20 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.557+0000 7f2253e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f225415ca40 0x7f225414e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f225415ca40 0x7f225414e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41130/0 (socket says 172.21.15.47:41130) 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 -- 172.21.15.47:0/4038418784 learned_addr learned my addr 172.21.15.47:0/4038418784 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253400640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22540ad8e0 0x7f225414b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f224be00640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2254156c20 0x7f22540732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 -- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2254156c20 msgr2=0x7f22540732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2254156c20 0x7f22540732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 -- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22540ad8e0 msgr2=0x7f225414b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22540ad8e0 0x7f225414b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 -- 172.21.15.47:0/4038418784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22540989f0 con 0x7f225415ca40 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f224be00640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2254156c20 0x7f22540732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:25.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.558+0000 7f2253e00640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f225415ca40 0x7f225414e6c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f224c00caa0 tx=0x7f224c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.559+0000 7f224b400640 1 -- 172.21.15.47:0/4038418784 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c017070 con 0x7f225415ca40 2024-04-20T10:30:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.559+0000 7f224b400640 1 -- 172.21.15.47:0/4038418784 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f224c00aba0 con 0x7f225415ca40 2024-04-20T10:30:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.559+0000 7f224b400640 1 -- 172.21.15.47:0/4038418784 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f224c012400 con 0x7f225415ca40 2024-04-20T10:30:25.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.559+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22540af0a0 con 0x7f225415ca40 2024-04-20T10:30:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.559+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22540993f0 con 0x7f225415ca40 2024-04-20T10:30:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.560+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22540af270 con 0x7f225415ca40 2024-04-20T10:30:25.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.561+0000 7f224b400640 1 -- 172.21.15.47:0/4038418784 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f224c01a070 con 0x7f225415ca40 2024-04-20T10:30:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.562+0000 7f224b400640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f223006c940 0x7f223006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.562+0000 7f224b400640 1 -- 172.21.15.47:0/4038418784 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f224c08b8e0 con 0x7f225415ca40 2024-04-20T10:30:25.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.563+0000 7f2253400640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f223006c940 0x7f223006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:25.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.564+0000 7f2253400640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f223006c940 0x7f223006ee00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2244036000 tx=0x7f2244008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:25.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.566+0000 7f224b400640 1 -- 172.21.15.47:0/4038418784 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f22540af270 con 0x7f225415ca40 2024-04-20T10:30:25.815 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.815+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2254072ba0 con 0x7f225415ca40 2024-04-20T10:30:25.816 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.816+0000 7f224b400640 1 -- 172.21.15.47:0/4038418784 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5202 (secure 0 0 0) 0x7f224c0595a0 con 0x7f225415ca40 2024-04-20T10:30:25.816 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:25.817 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3182036779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3182036779},{"type":"v1","addr":"172.21.15.175:6835","nonce":3182036779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5398,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1207907337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1207907337},{"type":"v1","addr":"172.21.15.175:6837","nonce":1207907337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/793057653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":793057653},{"type":"v1","addr":"172.21.15.47:6833","nonce":793057653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:24.860182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6057":{"gid":6057,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3814505199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3814505199},{"type":"v1","addr":"172.21.15.47:6835","nonce":3814505199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6057,"qdb_cluster":[6057]},"id":3}]} 2024-04-20T10:30:25.817 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-04-20T10:30:25.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f223006c940 msgr2=0x7f223006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f223006c940 0x7f223006ee00 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f2244036000 tx=0x7f2244008030 comp rx=0 tx=0).stop 2024-04-20T10:30:25.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f225415ca40 msgr2=0x7f225414e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:25.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f225415ca40 0x7f225414e6c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f224c00caa0 tx=0x7f224c00cf60 comp rx=0 tx=0).stop 2024-04-20T10:30:25.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 shutdown_connections 2024-04-20T10:30:25.819 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f223006c940 0x7f223006ee00 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f225415ca40 0x7f225414e6c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2254156c20 0x7f22540732d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 --2- 172.21.15.47:0/4038418784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22540ad8e0 0x7f225414b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 >> 172.21.15.47:0/4038418784 conn(0x7f22540b3350 msgr2=0x7f225415f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 shutdown_connections 2024-04-20T10:30:25.820 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:25.819+0000 7f225a800640 1 -- 172.21.15.47:0/4038418784 wait complete. 2024-04-20T10:30:25.839 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:30:26.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.053+0000 7f77c6e00640 1 Processor -- start 2024-04-20T10:30:26.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.053+0000 7f77c6e00640 1 -- start start 2024-04-20T10:30:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.054+0000 7f77c6e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f77c015e8a0 0x7f77c006e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.054+0000 7f77c6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 0x7f77c00d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.054+0000 7f77c6e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f77c006f060 con 0x7f77c00d08a0 2024-04-20T10:30:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.054+0000 7f77c6e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f77c006e7c0 con 0x7f77c015e8a0 2024-04-20T10:30:26.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.054+0000 7f77c6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77c006eaa0 con 0x7f77c006dbb0 2024-04-20T10:30:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.054+0000 7f77c4e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f77c00d08a0 0x7f77c006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53436/0 (socket says 172.21.15.47:53436) 2024-04-20T10:30:26.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.054+0000 7f77c4e00640 1 -- 172.21.15.47:0/2000658485 learned_addr learned my addr 172.21.15.47:0/2000658485 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bfe00640 1 --2- 172.21.15.47:0/2000658485 >> v2:172.21.15.175:3300/0 conn(0x7f77c015e8a0 0x7f77c006e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3425595713 0 0) 0x7f77c006f060 con 0x7f77c00d08a0 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f77a4003520 con 0x7f77c00d08a0 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77b7e00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 0x7f77c00d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1509493088 0 0) 0x7f77a4003520 con 0x7f77c00d08a0 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 msgr2=0x7f77c00d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 0x7f77c00d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 >> v2:172.21.15.175:3300/0 conn(0x7f77c015e8a0 msgr2=0x7f77c006e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 --2- 172.21.15.47:0/2000658485 >> v2:172.21.15.175:3300/0 conn(0x7f77c015e8a0 0x7f77c006e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c006ed80 con 0x7f77c00d08a0 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f77b80019f0 con 0x7f77c00d08a0 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 >> v1:172.21.15.47:6789/0 conn(0x7f77c00d08a0 legacy=0x7f77c006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a40039b0 0x7f77a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 0x7f77a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77c006e7c0 con 0x7f77a4004800 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77c006eaa0 con 0x7f77a40039b0 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.055+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77a400ba10 con 0x7f77c006dbb0 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77b7e00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77bfe00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a40039b0 0x7f77a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77b7e00640 1 -- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 msgr2=0x7f77a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77c4e00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 0x7f77a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77b7e00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c006dbb0 0x7f77a40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77b7e00640 1 -- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a40039b0 msgr2=0x7f77a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77b7e00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a40039b0 0x7f77a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.056+0000 7f77b7e00640 1 -- 172.21.15.47:0/2000658485 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77a4003520 con 0x7f77a4004800 2024-04-20T10:30:26.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.057+0000 7f77b7e00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f77a800ba30 tx=0x7f77a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5004c3459175978 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8036020 con 0x7f77a4004800 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77a8032640 con 0x7f77a4004800 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77bf400640 1 -- 172.21.15.47:0/2000658485 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a800b600 con 0x7f77a4004800 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77c6e00640 1 -- 172.21.15.47:0/2000658485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 msgr2=0x7f77a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77c6e00640 1 --2- 172.21.15.47:0/2000658485 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77a4004c80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f77a800ba30 tx=0x7f77a4005830 comp rx=0 tx=0).stop 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77c4e00640 1 -- 172.21.15.47:0/2000658485 reap_dead start 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77c6e00640 1 -- 172.21.15.47:0/2000658485 shutdown_connections 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.058+0000 7f77c6e00640 1 -- 172.21.15.47:0/2000658485 >> 172.21.15.47:0/2000658485 conn(0x7f77c00b3350 msgr2=0x7f77c00b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.059+0000 7f77c6e00640 1 -- 172.21.15.47:0/2000658485 shutdown_connections 2024-04-20T10:30:26.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.059+0000 7f77c6e00640 1 -- 172.21.15.47:0/2000658485 wait complete. 2024-04-20T10:30:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.060+0000 7f77c6e00640 1 Processor -- start 2024-04-20T10:30:26.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.061+0000 7f77c6e00640 1 -- start start 2024-04-20T10:30:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77c6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77c0155d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77c6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c00b1780 0x7f77c01f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77c6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77c0152bb0 0x7f77c0159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77c6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f77c006fef0 con 0x7f77a4004800 2024-04-20T10:30:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77c6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77c006fd70 con 0x7f77c0152bb0 2024-04-20T10:30:26.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77c6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77c00ae690 con 0x7f77c00b1780 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77c0155d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77c0155d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41148/0 (socket says 172.21.15.47:41148) 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 -- 172.21.15.47:0/1298366104 learned_addr learned my addr 172.21.15.47:0/1298366104 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 -- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c00b1780 msgr2=0x7f77c01f74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77b7e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c00b1780 0x7f77c01f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c00b1780 0x7f77c01f74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 -- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77c0152bb0 msgr2=0x7f77c0159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77c0152bb0 0x7f77c0159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.062+0000 7f77bfe00640 1 -- 172.21.15.47:0/1298366104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77c0156550 con 0x7f77a4004800 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.063+0000 7f77bfe00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77c0155d00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f77b00116d0 tx=0x7f77b0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.063+0000 7f77bcc00640 1 -- 172.21.15.47:0/1298366104 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b000f9c0 con 0x7f77a4004800 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.063+0000 7f77bcc00640 1 -- 172.21.15.47:0/1298366104 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f77b000fb60 con 0x7f77a4004800 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.063+0000 7f77bcc00640 1 -- 172.21.15.47:0/1298366104 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b0015460 con 0x7f77a4004800 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.063+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77c0156830 con 0x7f77a4004800 2024-04-20T10:30:26.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.063+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77c0156df0 con 0x7f77a4004800 2024-04-20T10:30:26.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.064+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77c0068980 con 0x7f77a4004800 2024-04-20T10:30:26.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.066+0000 7f77bcc00640 1 -- 172.21.15.47:0/1298366104 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f77b0015600 con 0x7f77a4004800 2024-04-20T10:30:26.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.067+0000 7f77bcc00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f779c06c8f0 0x7f779c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.067+0000 7f77bcc00640 1 -- 172.21.15.47:0/1298366104 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f77b008c770 con 0x7f77a4004800 2024-04-20T10:30:26.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.067+0000 7f77b7e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f779c06c8f0 0x7f779c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.068+0000 7f77b7e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f779c06c8f0 0x7f779c06edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77a800ba00 tx=0x7f77a8030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:26.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.071+0000 7f77bcc00640 1 -- 172.21.15.47:0/1298366104 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f77b0019030 con 0x7f77a4004800 2024-04-20T10:30:26.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.277+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77c006d160 con 0x7f77a4004800 2024-04-20T10:30:26.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.280+0000 7f77bcc00640 1 -- 172.21.15.47:0/1298366104 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) ==== 74+0+16894 (secure 0 0 0) 0x7f77b005a970 con 0x7f77a4004800 2024-04-20T10:30:26.281 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:26.281 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":54,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:23.283233+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:30:19.610070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-20T10:30:20.722858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:30:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f779c06c8f0 msgr2=0x7f779c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f779c06c8f0 0x7f779c06edb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f77a800ba00 tx=0x7f77a8030000 comp rx=0 tx=0).stop 2024-04-20T10:30:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 msgr2=0x7f77c0155d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.283 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77c0155d00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f77b00116d0 tx=0x7f77b0011b90 comp rx=0 tx=0).stop 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 shutdown_connections 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77c0152bb0 0x7f77c0159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77c00b1780 0x7f77c01f74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f779c06c8f0 0x7f779c06edb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 --2- 172.21.15.47:0/1298366104 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f77a4004800 0x7f77c0155d00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 >> 172.21.15.47:0/1298366104 conn(0x7f77c00b3350 msgr2=0x7f77c0144440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 shutdown_connections 2024-04-20T10:30:26.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.283+0000 7f77c6e00640 1 -- 172.21.15.47:0/1298366104 wait complete. 2024-04-20T10:30:26.302 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-20T10:30:26.302 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:30:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.512+0000 7fa842400640 1 Processor -- start 2024-04-20T10:30:26.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.512+0000 7fa842400640 1 -- start start 2024-04-20T10:30:26.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa842400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c159e10 0x7fa83c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa842400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa83c15a1f0 0x7fa83c071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa842400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa83c15cc90 con 0x7fa83c06da90 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa842400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa83c072900 con 0x7fa83c15a1f0 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa842400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa83c15ca40 con 0x7fa83c159e10 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa83be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa83c15a1f0 0x7fa83c071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa841a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa83c06da90 0x7fa83c160030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53446/0 (socket says 172.21.15.47:53446) 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa83be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa83c15a1f0 0x7fa83c071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41156/0 (socket says 172.21.15.47:41156) 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa841a00640 1 -- 172.21.15.47:0/125328763 learned_addr learned my addr 172.21.15.47:0/125328763 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.514+0000 7fa841000640 1 --2- 172.21.15.47:0/125328763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c159e10 0x7fa83c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.515+0000 7fa83be00640 1 -- 172.21.15.47:0/125328763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c159e10 msgr2=0x7fa83c071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.515+0000 7fa83be00640 1 --2- 172.21.15.47:0/125328763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c159e10 0x7fa83c071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.515+0000 7fa83be00640 1 -- 172.21.15.47:0/125328763 >> v1:172.21.15.47:6789/0 conn(0x7fa83c06da90 legacy=0x7fa83c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.515+0000 7fa83be00640 1 -- 172.21.15.47:0/125328763 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c15cf70 con 0x7fa83c15a1f0 2024-04-20T10:30:26.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.515+0000 7fa83b400640 1 -- 172.21.15.47:0/125328763 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 899025405 0 0) 0x7fa83c15cc90 con 0x7fa83c06da90 2024-04-20T10:30:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.516+0000 7fa83be00640 1 --2- 172.21.15.47:0/125328763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c15a1f0 0x7fa83c071fc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa83c15cbc0 tx=0x7fa82c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa0683051af182b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.516+0000 7fa83b400640 1 -- 172.21.15.47:0/125328763 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c039070 con 0x7fa83c15a1f0 2024-04-20T10:30:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.516+0000 7fa83b400640 1 -- 172.21.15.47:0/125328763 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82c02cc40 con 0x7fa83c15a1f0 2024-04-20T10:30:26.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.516+0000 7fa83b400640 1 -- 172.21.15.47:0/125328763 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c034440 con 0x7fa83c15a1f0 2024-04-20T10:30:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 -- 172.21.15.47:0/125328763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c15a1f0 msgr2=0x7fa83c071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 --2- 172.21.15.47:0/125328763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c15a1f0 0x7fa83c071fc0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fa83c15cbc0 tx=0x7fa82c02be90 comp rx=0 tx=0).stop 2024-04-20T10:30:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 -- 172.21.15.47:0/125328763 shutdown_connections 2024-04-20T10:30:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 --2- 172.21.15.47:0/125328763 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c15a1f0 0x7fa83c071fc0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 --2- 172.21.15.47:0/125328763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c159e10 0x7fa83c071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 -- 172.21.15.47:0/125328763 >> 172.21.15.47:0/125328763 conn(0x7fa83c0d0040 msgr2=0x7fa83c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 -- 172.21.15.47:0/125328763 shutdown_connections 2024-04-20T10:30:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.517+0000 7fa842400640 1 -- 172.21.15.47:0/125328763 wait complete. 2024-04-20T10:30:26.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.519+0000 7fa842400640 1 Processor -- start 2024-04-20T10:30:26.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.519+0000 7fa842400640 1 -- start start 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa842400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c06da90 0x7fa83c153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa842400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c159e10 0x7fa83c0b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa842400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83c15a1f0 0x7fa83c0b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa842400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c15d810 con 0x7fa83c159e10 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa842400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c15d530 con 0x7fa83c15a1f0 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa842400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa83c06b900 con 0x7fa83c06da90 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa83be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c159e10 0x7fa83c0b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa83be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c159e10 0x7fa83c0b3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41162/0 (socket says 172.21.15.47:41162) 2024-04-20T10:30:26.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa83be00640 1 -- 172.21.15.47:0/3369172812 learned_addr learned my addr 172.21.15.47:0/3369172812 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa83be00640 1 -- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c06da90 msgr2=0x7fa83c153890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.520+0000 7fa841000640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c06da90 0x7fa83c153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa841a00640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83c15a1f0 0x7fa83c0b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa83be00640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c06da90 0x7fa83c153890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa83be00640 1 -- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83c15a1f0 msgr2=0x7fa83c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa83be00640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83c15a1f0 0x7fa83c0b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa83be00640 1 -- 172.21.15.47:0/3369172812 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c159250 con 0x7fa83c159e10 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa841000640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c06da90 0x7fa83c153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa83be00640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c159e10 0x7fa83c0b3070 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009e80 tx=0x7fa82c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:26.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa838c00640 1 -- 172.21.15.47:0/3369172812 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c039050 con 0x7fa83c159e10 2024-04-20T10:30:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa838c00640 1 -- 172.21.15.47:0/3369172812 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa82c03a040 con 0x7fa83c159e10 2024-04-20T10:30:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa838c00640 1 -- 172.21.15.47:0/3369172812 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c009450 con 0x7fa83c159e10 2024-04-20T10:30:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.521+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83c152ad0 con 0x7fa83c159e10 2024-04-20T10:30:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.522+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83c153090 con 0x7fa83c159e10 2024-04-20T10:30:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.523+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83c075b60 con 0x7fa83c159e10 2024-04-20T10:30:26.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.524+0000 7fa838c00640 1 -- 172.21.15.47:0/3369172812 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa82c047020 con 0x7fa83c159e10 2024-04-20T10:30:26.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.525+0000 7fa838c00640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81806c8f0 0x7fa81806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.525+0000 7fa838c00640 1 -- 172.21.15.47:0/3369172812 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fa82c0ade40 con 0x7fa83c159e10 2024-04-20T10:30:26.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.526+0000 7fa841000640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81806c8f0 0x7fa81806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:26.526 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.527+0000 7fa841000640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81806c8f0 0x7fa81806edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:26.529 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.529+0000 7fa838c00640 1 -- 172.21.15.47:0/3369172812 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa82c04e020 con 0x7fa83c159e10 2024-04-20T10:30:26.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.724+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa83c073b00 con 0x7fa81806c8f0 2024-04-20T10:30:26.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.742+0000 7fa838c00640 1 -- 172.21.15.47:0/3369172812 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245415 (secure 0 0 0) 0x7fa83c073b00 con 0x7fa81806c8f0 2024-04-20T10:30:26.746 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:26.747 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.747+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81806c8f0 msgr2=0x7fa81806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.747+0000 7fa842400640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81806c8f0 0x7fa81806edb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824008040 comp rx=0 tx=0).stop 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.747+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c159e10 msgr2=0x7fa83c0b3070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.747+0000 7fa842400640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c159e10 0x7fa83c0b3070 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009e80 tx=0x7fa82c004a40 comp rx=0 tx=0).stop 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 shutdown_connections 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa83c15a1f0 0x7fa83c0b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa81806c8f0 0x7fa81806edb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa83c159e10 0x7fa83c0b3070 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 --2- 172.21.15.47:0/3369172812 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa83c06da90 0x7fa83c153890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 >> 172.21.15.47:0/3369172812 conn(0x7fa83c0d0040 msgr2=0x7fa83c06cd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:26.749 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 shutdown_connections 2024-04-20T10:30:26.750 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:26.748+0000 7fa842400640 1 -- 172.21.15.47:0/3369172812 wait complete. 2024-04-20T10:30:26.767 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-04-20T10:30:25.129304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592036,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":189,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143,"ondisk_log_size":143,"up":112,"acting":112,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":51,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":238816,"kb_used_data":5976,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735904,"statfs":{"total":773094113280,"available":772849565696,"internally_reserved":0,"allocated":6119424,"data_stored":3315256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.872388"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303289+0000","last_change":"2024-04-20T10:30:22.329273+0000","last_active":"2024-04-20T10:30:23.303289+0000","last_peered":"2024-04-20T10:30:23.303289+0000","last_clean":"2024-04-20T10:30:23.303289+0000","last_became_active":"2024-04-20T10:30:22.328446+0000","last_became_peered":"2024-04-20T10:30:22.328446+0000","last_unstale":"2024-04-20T10:30:23.303289+0000","last_undegraded":"2024-04-20T10:30:23.303289+0000","last_fullsized":"2024-04-20T10:30:23.303289+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:51.725829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304416+0000","last_change":"2024-04-20T10:30:22.307386+0000","last_active":"2024-04-20T10:30:23.304416+0000","last_peered":"2024-04-20T10:30:23.304416+0000","last_clean":"2024-04-20T10:30:23.304416+0000","last_became_active":"2024-04-20T10:30:22.306653+0000","last_became_peered":"2024-04-20T10:30:22.306653+0000","last_unstale":"2024-04-20T10:30:23.304416+0000","last_undegraded":"2024-04-20T10:30:23.304416+0000","last_fullsized":"2024-04-20T10:30:23.304416+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:12.364253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481932+0000","last_change":"2024-04-20T10:30:22.323316+0000","last_active":"2024-04-20T10:30:23.481932+0000","last_peered":"2024-04-20T10:30:23.481932+0000","last_clean":"2024-04-20T10:30:23.481932+0000","last_became_active":"2024-04-20T10:30:22.322941+0000","last_became_peered":"2024-04-20T10:30:22.322941+0000","last_unstale":"2024-04-20T10:30:23.481932+0000","last_undegraded":"2024-04-20T10:30:23.481932+0000","last_fullsized":"2024-04-20T10:30:23.481932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:10:34.132958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339337+0000","last_change":"2024-04-20T10:30:22.325907+0000","last_active":"2024-04-20T10:30:23.339337+0000","last_peered":"2024-04-20T10:30:23.339337+0000","last_clean":"2024-04-20T10:30:23.339337+0000","last_became_active":"2024-04-20T10:30:22.321317+0000","last_became_peered":"2024-04-20T10:30:22.321317+0000","last_unstale":"2024-04-20T10:30:23.339337+0000","last_undegraded":"2024-04-20T10:30:23.339337+0000","last_fullsized":"2024-04-20T10:30:23.339337+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:25.070053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481679+0000","last_change":"2024-04-20T10:30:22.323164+0000","last_active":"2024-04-20T10:30:23.481679+0000","last_peered":"2024-04-20T10:30:23.481679+0000","last_clean":"2024-04-20T10:30:23.481679+0000","last_became_active":"2024-04-20T10:30:22.322853+0000","last_became_peered":"2024-04-20T10:30:22.322853+0000","last_unstale":"2024-04-20T10:30:23.481679+0000","last_undegraded":"2024-04-20T10:30:23.481679+0000","last_fullsized":"2024-04-20T10:30:23.481679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:28:18.396065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304856+0000","last_change":"2024-04-20T10:30:22.310232+0000","last_active":"2024-04-20T10:30:23.304856+0000","last_peered":"2024-04-20T10:30:23.304856+0000","last_clean":"2024-04-20T10:30:23.304856+0000","last_became_active":"2024-04-20T10:30:22.309893+0000","last_became_peered":"2024-04-20T10:30:22.309893+0000","last_unstale":"2024-04-20T10:30:23.304856+0000","last_undegraded":"2024-04-20T10:30:23.304856+0000","last_fullsized":"2024-04-20T10:30:23.304856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:19.761399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340340+0000","last_change":"2024-04-20T10:30:22.334859+0000","last_active":"2024-04-20T10:30:23.340340+0000","last_peered":"2024-04-20T10:30:23.340340+0000","last_clean":"2024-04-20T10:30:23.340340+0000","last_became_active":"2024-04-20T10:30:22.334532+0000","last_became_peered":"2024-04-20T10:30:22.334532+0000","last_unstale":"2024-04-20T10:30:23.340340+0000","last_undegraded":"2024-04-20T10:30:23.340340+0000","last_fullsized":"2024-04-20T10:30:23.340340+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:22.708741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302516+0000","last_change":"2024-04-20T10:30:22.328972+0000","last_active":"2024-04-20T10:30:23.302516+0000","last_peered":"2024-04-20T10:30:23.302516+0000","last_clean":"2024-04-20T10:30:23.302516+0000","last_became_active":"2024-04-20T10:30:22.328579+0000","last_became_peered":"2024-04-20T10:30:22.328579+0000","last_unstale":"2024-04-20T10:30:23.302516+0000","last_undegraded":"2024-04-20T10:30:23.302516+0000","last_fullsized":"2024-04-20T10:30:23.302516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:22.708741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482160+0000","last_change":"2024-04-20T10:30:22.334770+0000","last_active":"2024-04-20T10:30:23.482160+0000","last_peered":"2024-04-20T10:30:23.482160+0000","last_clean":"2024-04-20T10:30:23.482160+0000","last_became_active":"2024-04-20T10:30:22.334386+0000","last_became_peered":"2024-04-20T10:30:22.334386+0000","last_unstale":"2024-04-20T10:30:23.482160+0000","last_undegraded":"2024-04-20T10:30:23.482160+0000","last_fullsized":"2024-04-20T10:30:23.482160+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.268934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482127+0000","last_change":"2024-04-20T10:30:22.329131+0000","last_active":"2024-04-20T10:30:23.482127+0000","last_peered":"2024-04-20T10:30:23.482127+0000","last_clean":"2024-04-20T10:30:23.482127+0000","last_became_active":"2024-04-20T10:30:22.327492+0000","last_became_peered":"2024-04-20T10:30:22.327492+0000","last_unstale":"2024-04-20T10:30:23.482127+0000","last_undegraded":"2024-04-20T10:30:23.482127+0000","last_fullsized":"2024-04-20T10:30:23.482127+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:39.381432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302272+0000","last_change":"2024-04-20T10:30:22.330328+0000","last_active":"2024-04-20T10:30:23.302272+0000","last_peered":"2024-04-20T10:30:23.302272+0000","last_clean":"2024-04-20T10:30:23.302272+0000","last_became_active":"2024-04-20T10:30:22.330023+0000","last_became_peered":"2024-04-20T10:30:22.330023+0000","last_unstale":"2024-04-20T10:30:23.302272+0000","last_undegraded":"2024-04-20T10:30:23.302272+0000","last_fullsized":"2024-04-20T10:30:23.302272+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:20:54.410647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481692+0000","last_change":"2024-04-20T10:30:22.332056+0000","last_active":"2024-04-20T10:30:23.481692+0000","last_peered":"2024-04-20T10:30:23.481692+0000","last_clean":"2024-04-20T10:30:23.481692+0000","last_became_active":"2024-04-20T10:30:22.331594+0000","last_became_peered":"2024-04-20T10:30:22.331594+0000","last_unstale":"2024-04-20T10:30:23.481692+0000","last_undegraded":"2024-04-20T10:30:23.481692+0000","last_fullsized":"2024-04-20T10:30:23.481692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:34:52.181995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304105+0000","last_change":"2024-04-20T10:30:22.332431+0000","last_active":"2024-04-20T10:30:23.304105+0000","last_peered":"2024-04-20T10:30:23.304105+0000","last_clean":"2024-04-20T10:30:23.304105+0000","last_became_active":"2024-04-20T10:30:22.332101+0000","last_became_peered":"2024-04-20T10:30:22.332101+0000","last_unstale":"2024-04-20T10:30:23.304105+0000","last_undegraded":"2024-04-20T10:30:23.304105+0000","last_fullsized":"2024-04-20T10:30:23.304105+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:23.991051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304644+0000","last_change":"2024-04-20T10:30:22.319721+0000","last_active":"2024-04-20T10:30:23.304644+0000","last_peered":"2024-04-20T10:30:23.304644+0000","last_clean":"2024-04-20T10:30:23.304644+0000","last_became_active":"2024-04-20T10:30:22.319421+0000","last_became_peered":"2024-04-20T10:30:22.319421+0000","last_unstale":"2024-04-20T10:30:23.304644+0000","last_undegraded":"2024-04-20T10:30:23.304644+0000","last_fullsized":"2024-04-20T10:30:23.304644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:20:55.452831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303687+0000","last_change":"2024-04-20T10:30:22.320705+0000","last_active":"2024-04-20T10:30:23.303687+0000","last_peered":"2024-04-20T10:30:23.303687+0000","last_clean":"2024-04-20T10:30:23.303687+0000","last_became_active":"2024-04-20T10:30:22.320266+0000","last_became_peered":"2024-04-20T10:30:22.320266+0000","last_unstale":"2024-04-20T10:30:23.303687+0000","last_undegraded":"2024-04-20T10:30:23.303687+0000","last_fullsized":"2024-04-20T10:30:23.303687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:17:46.675848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302990+0000","last_change":"2024-04-20T10:30:22.311889+0000","last_active":"2024-04-20T10:30:23.302990+0000","last_peered":"2024-04-20T10:30:23.302990+0000","last_clean":"2024-04-20T10:30:23.302990+0000","last_became_active":"2024-04-20T10:30:22.311588+0000","last_became_peered":"2024-04-20T10:30:22.311588+0000","last_unstale":"2024-04-20T10:30:23.302990+0000","last_undegraded":"2024-04-20T10:30:23.302990+0000","last_fullsized":"2024-04-20T10:30:23.302990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:55.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302524+0000","last_change":"2024-04-20T10:30:22.320501+0000","last_active":"2024-04-20T10:30:23.302524+0000","last_peered":"2024-04-20T10:30:23.302524+0000","last_clean":"2024-04-20T10:30:23.302524+0000","last_became_active":"2024-04-20T10:30:22.320193+0000","last_became_peered":"2024-04-20T10:30:22.320193+0000","last_unstale":"2024-04-20T10:30:23.302524+0000","last_undegraded":"2024-04-20T10:30:23.302524+0000","last_fullsized":"2024-04-20T10:30:23.302524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:17.001959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303346+0000","last_change":"2024-04-20T10:30:22.322865+0000","last_active":"2024-04-20T10:30:23.303346+0000","last_peered":"2024-04-20T10:30:23.303346+0000","last_clean":"2024-04-20T10:30:23.303346+0000","last_became_active":"2024-04-20T10:30:22.322214+0000","last_became_peered":"2024-04-20T10:30:22.322214+0000","last_unstale":"2024-04-20T10:30:23.303346+0000","last_undegraded":"2024-04-20T10:30:23.303346+0000","last_fullsized":"2024-04-20T10:30:23.303346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:53.186380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.305122+0000","last_change":"2024-04-20T10:30:22.325203+0000","last_active":"2024-04-20T10:30:23.305122+0000","last_peered":"2024-04-20T10:30:23.305122+0000","last_clean":"2024-04-20T10:30:23.305122+0000","last_became_active":"2024-04-20T10:30:22.324096+0000","last_became_peered":"2024-04-20T10:30:22.324096+0000","last_unstale":"2024-04-20T10:30:23.305122+0000","last_undegraded":"2024-04-20T10:30:23.305122+0000","last_fullsized":"2024-04-20T10:30:23.305122+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:50.899499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482431+0000","last_change":"2024-04-20T10:30:22.336175+0000","last_active":"2024-04-20T10:30:23.482431+0000","last_peered":"2024-04-20T10:30:23.482431+0000","last_clean":"2024-04-20T10:30:23.482431+0000","last_became_active":"2024-04-20T10:30:22.335854+0000","last_became_peered":"2024-04-20T10:30:22.335854+0000","last_unstale":"2024-04-20T10:30:23.482431+0000","last_undegraded":"2024-04-20T10:30:23.482431+0000","last_fullsized":"2024-04-20T10:30:23.482431+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:11.088283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302262+0000","last_change":"2024-04-20T10:30:22.329311+0000","last_active":"2024-04-20T10:30:23.302262+0000","last_peered":"2024-04-20T10:30:23.302262+0000","last_clean":"2024-04-20T10:30:23.302262+0000","last_became_active":"2024-04-20T10:30:22.328470+0000","last_became_peered":"2024-04-20T10:30:22.328470+0000","last_unstale":"2024-04-20T10:30:23.302262+0000","last_undegraded":"2024-04-20T10:30:23.302262+0000","last_fullsized":"2024-04-20T10:30:23.302262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:58.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303408+0000","last_change":"2024-04-20T10:30:22.328382+0000","last_active":"2024-04-20T10:30:23.303408+0000","last_peered":"2024-04-20T10:30:23.303408+0000","last_clean":"2024-04-20T10:30:23.303408+0000","last_became_active":"2024-04-20T10:30:22.327878+0000","last_became_peered":"2024-04-20T10:30:22.327878+0000","last_unstale":"2024-04-20T10:30:23.303408+0000","last_undegraded":"2024-04-20T10:30:23.303408+0000","last_fullsized":"2024-04-20T10:30:23.303408+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:40:50.945796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482534+0000","last_change":"2024-04-20T10:30:21.305701+0000","last_active":"2024-04-20T10:30:23.482534+0000","last_peered":"2024-04-20T10:30:23.482534+0000","last_clean":"2024-04-20T10:30:23.482534+0000","last_became_active":"2024-04-20T10:30:21.305374+0000","last_became_peered":"2024-04-20T10:30:21.305374+0000","last_unstale":"2024-04-20T10:30:23.482534+0000","last_undegraded":"2024-04-20T10:30:23.482534+0000","last_fullsized":"2024-04-20T10:30:23.482534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:25:54.048126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339955+0000","last_change":"2024-04-20T10:30:21.299006+0000","last_active":"2024-04-20T10:30:23.339955+0000","last_peered":"2024-04-20T10:30:23.339955+0000","last_clean":"2024-04-20T10:30:23.339955+0000","last_became_active":"2024-04-20T10:30:21.298665+0000","last_became_peered":"2024-04-20T10:30:21.298665+0000","last_unstale":"2024-04-20T10:30:23.339955+0000","last_undegraded":"2024-04-20T10:30:23.339955+0000","last_fullsized":"2024-04-20T10:30:23.339955+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:10.074085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339743+0000","last_change":"2024-04-20T10:30:22.323657+0000","last_active":"2024-04-20T10:30:23.339743+0000","last_peered":"2024-04-20T10:30:23.339743+0000","last_clean":"2024-04-20T10:30:23.339743+0000","last_became_active":"2024-04-20T10:30:22.323314+0000","last_became_peered":"2024-04-20T10:30:22.323314+0000","last_unstale":"2024-04-20T10:30:23.339743+0000","last_undegraded":"2024-04-20T10:30:23.339743+0000","last_fullsized":"2024-04-20T10:30:23.339743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:53:17.743013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304203+0000","last_change":"2024-04-20T10:30:21.302553+0000","last_active":"2024-04-20T10:30:23.304203+0000","last_peered":"2024-04-20T10:30:23.304203+0000","last_clean":"2024-04-20T10:30:23.304203+0000","last_became_active":"2024-04-20T10:30:21.302147+0000","last_became_peered":"2024-04-20T10:30:21.302147+0000","last_unstale":"2024-04-20T10:30:23.304203+0000","last_undegraded":"2024-04-20T10:30:23.304203+0000","last_fullsized":"2024-04-20T10:30:23.304203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:25:01.405342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340420+0000","last_change":"2024-04-20T10:30:22.324959+0000","last_active":"2024-04-20T10:30:23.340420+0000","last_peered":"2024-04-20T10:30:23.340420+0000","last_clean":"2024-04-20T10:30:23.340420+0000","last_became_active":"2024-04-20T10:30:22.324664+0000","last_became_peered":"2024-04-20T10:30:22.324664+0000","last_unstale":"2024-04-20T10:30:23.340420+0000","last_undegraded":"2024-04-20T10:30:23.340420+0000","last_fullsized":"2024-04-20T10:30:23.340420+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:31.622075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302446+0000","last_change":"2024-04-20T10:30:21.294347+0000","last_active":"2024-04-20T10:30:23.302446+0000","last_peered":"2024-04-20T10:30:23.302446+0000","last_clean":"2024-04-20T10:30:23.302446+0000","last_became_active":"2024-04-20T10:30:21.294024+0000","last_became_peered":"2024-04-20T10:30:21.294024+0000","last_unstale":"2024-04-20T10:30:23.302446+0000","last_undegraded":"2024-04-20T10:30:23.302446+0000","last_fullsized":"2024-04-20T10:30:23.302446+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:30.247444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302827+0000","last_change":"2024-04-20T10:30:22.311712+0000","last_active":"2024-04-20T10:30:23.302827+0000","last_peered":"2024-04-20T10:30:23.302827+0000","last_clean":"2024-04-20T10:30:23.302827+0000","last_became_active":"2024-04-20T10:30:22.311357+0000","last_became_peered":"2024-04-20T10:30:22.311357+0000","last_unstale":"2024-04-20T10:30:23.302827+0000","last_undegraded":"2024-04-20T10:30:23.302827+0000","last_fullsized":"2024-04-20T10:30:23.302827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:42:25.740328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340300+0000","last_change":"2024-04-20T10:30:21.306997+0000","last_active":"2024-04-20T10:30:23.340300+0000","last_peered":"2024-04-20T10:30:23.340300+0000","last_clean":"2024-04-20T10:30:23.340300+0000","last_became_active":"2024-04-20T10:30:21.306251+0000","last_became_peered":"2024-04-20T10:30:21.306251+0000","last_unstale":"2024-04-20T10:30:23.340300+0000","last_undegraded":"2024-04-20T10:30:23.340300+0000","last_fullsized":"2024-04-20T10:30:23.340300+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:38.367235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482067+0000","last_change":"2024-04-20T10:30:21.299360+0000","last_active":"2024-04-20T10:30:23.482067+0000","last_peered":"2024-04-20T10:30:23.482067+0000","last_clean":"2024-04-20T10:30:23.482067+0000","last_became_active":"2024-04-20T10:30:21.299119+0000","last_became_peered":"2024-04-20T10:30:21.299119+0000","last_unstale":"2024-04-20T10:30:23.482067+0000","last_undegraded":"2024-04-20T10:30:23.482067+0000","last_fullsized":"2024-04-20T10:30:23.482067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:00.556201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302709+0000","last_change":"2024-04-20T10:30:22.322825+0000","last_active":"2024-04-20T10:30:23.302709+0000","last_peered":"2024-04-20T10:30:23.302709+0000","last_clean":"2024-04-20T10:30:23.302709+0000","last_became_active":"2024-04-20T10:30:22.322526+0000","last_became_peered":"2024-04-20T10:30:22.322526+0000","last_unstale":"2024-04-20T10:30:23.302709+0000","last_undegraded":"2024-04-20T10:30:23.302709+0000","last_fullsized":"2024-04-20T10:30:23.302709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:31.622075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482761+0000","last_change":"2024-04-20T10:30:22.329140+0000","last_active":"2024-04-20T10:30:23.482761+0000","last_peered":"2024-04-20T10:30:23.482761+0000","last_clean":"2024-04-20T10:30:23.482761+0000","last_became_active":"2024-04-20T10:30:22.328862+0000","last_became_peered":"2024-04-20T10:30:22.328862+0000","last_unstale":"2024-04-20T10:30:23.482761+0000","last_undegraded":"2024-04-20T10:30:23.482761+0000","last_fullsized":"2024-04-20T10:30:23.482761+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:53.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.305034+0000","last_change":"2024-04-20T10:30:22.333023+0000","last_active":"2024-04-20T10:30:23.305034+0000","last_peered":"2024-04-20T10:30:23.305034+0000","last_clean":"2024-04-20T10:30:23.305034+0000","last_became_active":"2024-04-20T10:30:22.332711+0000","last_became_peered":"2024-04-20T10:30:22.332711+0000","last_unstale":"2024-04-20T10:30:23.305034+0000","last_undegraded":"2024-04-20T10:30:23.305034+0000","last_fullsized":"2024-04-20T10:30:23.305034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:31.579201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482019+0000","last_change":"2024-04-20T10:30:21.305877+0000","last_active":"2024-04-20T10:30:23.482019+0000","last_peered":"2024-04-20T10:30:23.482019+0000","last_clean":"2024-04-20T10:30:23.482019+0000","last_became_active":"2024-04-20T10:30:21.305421+0000","last_became_peered":"2024-04-20T10:30:21.305421+0000","last_unstale":"2024-04-20T10:30:23.482019+0000","last_undegraded":"2024-04-20T10:30:23.482019+0000","last_fullsized":"2024-04-20T10:30:23.482019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:46:34.127797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340018+0000","last_change":"2024-04-20T10:30:22.326109+0000","last_active":"2024-04-20T10:30:23.340018+0000","last_peered":"2024-04-20T10:30:23.340018+0000","last_clean":"2024-04-20T10:30:23.340018+0000","last_became_active":"2024-04-20T10:30:22.325559+0000","last_became_peered":"2024-04-20T10:30:22.325559+0000","last_unstale":"2024-04-20T10:30:23.340018+0000","last_undegraded":"2024-04-20T10:30:23.340018+0000","last_fullsized":"2024-04-20T10:30:23.340018+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:20:54.410647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482942+0000","last_change":"2024-04-20T10:30:21.306532+0000","last_active":"2024-04-20T10:30:23.482942+0000","last_peered":"2024-04-20T10:30:23.482942+0000","last_clean":"2024-04-20T10:30:23.482942+0000","last_became_active":"2024-04-20T10:30:21.305377+0000","last_became_peered":"2024-04-20T10:30:21.305377+0000","last_unstale":"2024-04-20T10:30:23.482942+0000","last_undegraded":"2024-04-20T10:30:23.482942+0000","last_fullsized":"2024-04-20T10:30:23.482942+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:55:25.206126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.301691+0000","last_change":"2024-04-20T10:30:22.319088+0000","last_active":"2024-04-20T10:30:23.301691+0000","last_peered":"2024-04-20T10:30:23.301691+0000","last_clean":"2024-04-20T10:30:23.301691+0000","last_became_active":"2024-04-20T10:30:22.318684+0000","last_became_peered":"2024-04-20T10:30:22.318684+0000","last_unstale":"2024-04-20T10:30:23.301691+0000","last_undegraded":"2024-04-20T10:30:23.301691+0000","last_fullsized":"2024-04-20T10:30:23.301691+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:26.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482736+0000","last_change":"2024-04-20T10:30:21.302031+0000","last_active":"2024-04-20T10:30:23.482736+0000","last_peered":"2024-04-20T10:30:23.482736+0000","last_clean":"2024-04-20T10:30:23.482736+0000","last_became_active":"2024-04-20T10:30:21.301752+0000","last_became_peered":"2024-04-20T10:30:21.301752+0000","last_unstale":"2024-04-20T10:30:23.482736+0000","last_undegraded":"2024-04-20T10:30:23.482736+0000","last_fullsized":"2024-04-20T10:30:23.482736+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:48.262531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481783+0000","last_change":"2024-04-20T10:30:22.334190+0000","last_active":"2024-04-20T10:30:23.481783+0000","last_peered":"2024-04-20T10:30:23.481783+0000","last_clean":"2024-04-20T10:30:23.481783+0000","last_became_active":"2024-04-20T10:30:22.333836+0000","last_became_peered":"2024-04-20T10:30:22.333836+0000","last_unstale":"2024-04-20T10:30:23.481783+0000","last_undegraded":"2024-04-20T10:30:23.481783+0000","last_fullsized":"2024-04-20T10:30:23.481783+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:16.425728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302231+0000","last_change":"2024-04-20T10:30:21.290127+0000","last_active":"2024-04-20T10:30:23.302231+0000","last_peered":"2024-04-20T10:30:23.302231+0000","last_clean":"2024-04-20T10:30:23.302231+0000","last_became_active":"2024-04-20T10:30:21.289602+0000","last_became_peered":"2024-04-20T10:30:21.289602+0000","last_unstale":"2024-04-20T10:30:23.302231+0000","last_undegraded":"2024-04-20T10:30:23.302231+0000","last_fullsized":"2024-04-20T10:30:23.302231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:59.254736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303554+0000","last_change":"2024-04-20T10:30:21.299403+0000","last_active":"2024-04-20T10:30:23.303554+0000","last_peered":"2024-04-20T10:30:23.303554+0000","last_clean":"2024-04-20T10:30:23.303554+0000","last_became_active":"2024-04-20T10:30:21.298263+0000","last_became_peered":"2024-04-20T10:30:21.298263+0000","last_unstale":"2024-04-20T10:30:23.303554+0000","last_undegraded":"2024-04-20T10:30:23.303554+0000","last_fullsized":"2024-04-20T10:30:23.303554+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:48:40.408177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.301994+0000","last_change":"2024-04-20T10:30:22.314238+0000","last_active":"2024-04-20T10:30:23.301994+0000","last_peered":"2024-04-20T10:30:23.301994+0000","last_clean":"2024-04-20T10:30:23.301994+0000","last_became_active":"2024-04-20T10:30:22.313673+0000","last_became_peered":"2024-04-20T10:30:22.313673+0000","last_unstale":"2024-04-20T10:30:23.301994+0000","last_undegraded":"2024-04-20T10:30:23.301994+0000","last_fullsized":"2024-04-20T10:30:23.301994+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:53:17.743013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339315+0000","last_change":"2024-04-20T10:30:21.306163+0000","last_active":"2024-04-20T10:30:23.339315+0000","last_peered":"2024-04-20T10:30:23.339315+0000","last_clean":"2024-04-20T10:30:23.339315+0000","last_became_active":"2024-04-20T10:30:21.305841+0000","last_became_peered":"2024-04-20T10:30:21.305841+0000","last_unstale":"2024-04-20T10:30:23.339315+0000","last_undegraded":"2024-04-20T10:30:23.339315+0000","last_fullsized":"2024-04-20T10:30:23.339315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:10:33.118760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482985+0000","last_change":"2024-04-20T10:30:22.337266+0000","last_active":"2024-04-20T10:30:23.482985+0000","last_peered":"2024-04-20T10:30:23.482985+0000","last_clean":"2024-04-20T10:30:23.482985+0000","last_became_active":"2024-04-20T10:30:22.336913+0000","last_became_peered":"2024-04-20T10:30:22.336913+0000","last_unstale":"2024-04-20T10:30:23.482985+0000","last_undegraded":"2024-04-20T10:30:23.482985+0000","last_fullsized":"2024-04-20T10:30:23.482985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:25.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339487+0000","last_change":"2024-04-20T10:30:21.298825+0000","last_active":"2024-04-20T10:30:23.339487+0000","last_peered":"2024-04-20T10:30:23.339487+0000","last_clean":"2024-04-20T10:30:23.339487+0000","last_became_active":"2024-04-20T10:30:21.298517+0000","last_became_peered":"2024-04-20T10:30:21.298517+0000","last_unstale":"2024-04-20T10:30:23.339487+0000","last_undegraded":"2024-04-20T10:30:23.339487+0000","last_fullsized":"2024-04-20T10:30:23.339487+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:28:17.381867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339394+0000","last_change":"2024-04-20T10:30:22.324315+0000","last_active":"2024-04-20T10:30:23.339394+0000","last_peered":"2024-04-20T10:30:23.339394+0000","last_clean":"2024-04-20T10:30:23.339394+0000","last_became_active":"2024-04-20T10:30:22.324038+0000","last_became_peered":"2024-04-20T10:30:22.324038+0000","last_unstale":"2024-04-20T10:30:23.339394+0000","last_undegraded":"2024-04-20T10:30:23.339394+0000","last_fullsized":"2024-04-20T10:30:23.339394+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:26.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304212+0000","last_change":"2024-04-20T10:30:22.319692+0000","last_active":"2024-04-20T10:30:23.304212+0000","last_peered":"2024-04-20T10:30:23.304212+0000","last_clean":"2024-04-20T10:30:23.304212+0000","last_became_active":"2024-04-20T10:30:22.319379+0000","last_became_peered":"2024-04-20T10:30:22.319379+0000","last_unstale":"2024-04-20T10:30:23.304212+0000","last_undegraded":"2024-04-20T10:30:23.304212+0000","last_fullsized":"2024-04-20T10:30:23.304212+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:27.294334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304094+0000","last_change":"2024-04-20T10:30:21.299031+0000","last_active":"2024-04-20T10:30:23.304094+0000","last_peered":"2024-04-20T10:30:23.304094+0000","last_clean":"2024-04-20T10:30:23.304094+0000","last_became_active":"2024-04-20T10:30:21.298071+0000","last_became_peered":"2024-04-20T10:30:21.298071+0000","last_unstale":"2024-04-20T10:30:23.304094+0000","last_undegraded":"2024-04-20T10:30:23.304094+0000","last_fullsized":"2024-04-20T10:30:23.304094+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:35.344994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339998+0000","last_change":"2024-04-20T10:30:22.330573+0000","last_active":"2024-04-20T10:30:23.339998+0000","last_peered":"2024-04-20T10:30:23.339998+0000","last_clean":"2024-04-20T10:30:23.339998+0000","last_became_active":"2024-04-20T10:30:22.330217+0000","last_became_peered":"2024-04-20T10:30:22.330217+0000","last_unstale":"2024-04-20T10:30:23.339998+0000","last_undegraded":"2024-04-20T10:30:23.339998+0000","last_fullsized":"2024-04-20T10:30:23.339998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:58.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340110+0000","last_change":"2024-04-20T10:30:21.302806+0000","last_active":"2024-04-20T10:30:23.340110+0000","last_peered":"2024-04-20T10:30:23.340110+0000","last_clean":"2024-04-20T10:30:23.340110+0000","last_became_active":"2024-04-20T10:30:21.302383+0000","last_became_peered":"2024-04-20T10:30:21.302383+0000","last_unstale":"2024-04-20T10:30:23.340110+0000","last_undegraded":"2024-04-20T10:30:23.340110+0000","last_fullsized":"2024-04-20T10:30:23.340110+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:15.411531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303861+0000","last_change":"2024-04-20T10:30:21.298661+0000","last_active":"2024-04-20T10:30:23.303861+0000","last_peered":"2024-04-20T10:30:23.303861+0000","last_clean":"2024-04-20T10:30:23.303861+0000","last_became_active":"2024-04-20T10:30:21.298297+0000","last_became_peered":"2024-04-20T10:30:21.298297+0000","last_unstale":"2024-04-20T10:30:23.303861+0000","last_undegraded":"2024-04-20T10:30:23.303861+0000","last_fullsized":"2024-04-20T10:30:23.303861+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:51.803957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340600+0000","last_change":"2024-04-20T10:30:22.329243+0000","last_active":"2024-04-20T10:30:23.340600+0000","last_peered":"2024-04-20T10:30:23.340600+0000","last_clean":"2024-04-20T10:30:23.340600+0000","last_became_active":"2024-04-20T10:30:22.327549+0000","last_became_peered":"2024-04-20T10:30:22.327549+0000","last_unstale":"2024-04-20T10:30:23.340600+0000","last_undegraded":"2024-04-20T10:30:23.340600+0000","last_fullsized":"2024-04-20T10:30:23.340600+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:42:25.740328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482730+0000","last_change":"2024-04-20T10:30:22.334102+0000","last_active":"2024-04-20T10:30:23.482730+0000","last_peered":"2024-04-20T10:30:23.482730+0000","last_clean":"2024-04-20T10:30:23.482730+0000","last_became_active":"2024-04-20T10:30:22.333754+0000","last_became_peered":"2024-04-20T10:30:22.333754+0000","last_unstale":"2024-04-20T10:30:23.482730+0000","last_undegraded":"2024-04-20T10:30:23.482730+0000","last_fullsized":"2024-04-20T10:30:23.482730+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:00.274563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:20.240378+0000","last_change":"2024-04-20T10:30:20.240378+0000","last_active":"2024-04-20T10:30:20.240378+0000","last_peered":"2024-04-20T10:30:20.240378+0000","last_clean":"2024-04-20T10:30:20.240378+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:20.240378+0000","last_undegraded":"2024-04-20T10:30:20.240378+0000","last_fullsized":"2024-04-20T10:30:20.240378+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-20T10:30:21.254575+0000","last_change":"2024-04-20T10:30:21.254575+0000","last_active":"2024-04-20T10:30:21.254575+0000","last_peered":"2024-04-20T10:30:21.254575+0000","last_clean":"2024-04-20T10:30:21.254575+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-20T10:30:21.254575+0000","last_undegraded":"2024-04-20T10:30:21.254575+0000","last_fullsized":"2024-04-20T10:30:21.254575+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.344638+0000","last_change":"2024-04-20T10:30:21.297078+0000","last_active":"2024-04-20T10:30:23.344638+0000","last_peered":"2024-04-20T10:30:23.344638+0000","last_clean":"2024-04-20T10:30:23.344638+0000","last_became_active":"2024-04-20T10:30:21.296174+0000","last_became_peered":"2024-04-20T10:30:21.296174+0000","last_unstale":"2024-04-20T10:30:23.344638+0000","last_undegraded":"2024-04-20T10:30:23.344638+0000","last_fullsized":"2024-04-20T10:30:23.344638+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:59.254736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":222,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-20T10:30:18.248120+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:30:18.248120+0000","last_peered":"2024-04-20T10:30:18.248120+0000","last_clean":"2024-04-20T10:30:18.248120+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:30:18.248120+0000","last_undegraded":"2024-04-20T10:30:18.248120+0000","last_fullsized":"2024-04-20T10:30:18.248120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":76,"acting":76,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":1668,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":4044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":34,"acting":34,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":279733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":280259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1245184,"data_stored":871747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":252942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31760,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340080,"statfs":{"total":96636764160,"available":96604241920,"internally_reserved":0,"allocated":585728,"data_stored":252942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509479,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32336,"kb_used_data":1152,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339504,"statfs":{"total":96636764160,"available":96603652096,"internally_reserved":0,"allocated":1179648,"data_stored":843310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509479,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31752,"kb_used_data":572,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340088,"statfs":{"total":96636764160,"available":96604250112,"internally_reserved":0,"allocated":585728,"data_stored":252942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":281381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:30:26.772 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:30:27.014 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.014+0000 7fb199a00640 1 Processor -- start 2024-04-20T10:30:27.015 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.014+0000 7fb199a00640 1 -- start start 2024-04-20T10:30:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.016+0000 7fb199a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb194163070 0x7fb194071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.016+0000 7fb199a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb19406c310 con 0x7fb194073a70 2024-04-20T10:30:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.016+0000 7fb199a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb19406c5f0 con 0x7fb1940735e0 2024-04-20T10:30:27.016 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.016+0000 7fb199a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fb19406c8d0 con 0x7fb194163070 2024-04-20T10:30:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.016+0000 7fb193e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb194163070 0x7fb194071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.017+0000 7fb193e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fb194163070 0x7fb194071d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41176/0 (socket says 172.21.15.47:41176) 2024-04-20T10:30:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.017+0000 7fb193e00640 1 -- 172.21.15.47:0/3624614153 learned_addr learned my addr 172.21.15.47:0/3624614153 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.017+0000 7fb193e00640 1 -- 172.21.15.47:0/3624614153 >> v1:172.21.15.175:6789/0 conn(0x7fb1940735e0 legacy=0x7fb194163450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.017+0000 7fb193e00640 1 -- 172.21.15.47:0/3624614153 >> v1:172.21.15.47:6789/0 conn(0x7fb194073a70 legacy=0x7fb19416c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.017 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.017+0000 7fb193e00640 1 -- 172.21.15.47:0/3624614153 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19406cbb0 con 0x7fb194163070 2024-04-20T10:30:27.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.017+0000 7fb193400640 1 -- 172.21.15.47:0/3624614153 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2669350287 0 0) 0x7fb19406c310 con 0x7fb194073a70 2024-04-20T10:30:27.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.018+0000 7fb193e00640 1 --2- 172.21.15.47:0/3624614153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb194163070 0x7fb194071d90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb18402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fb6a5e0e1e9529 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:27.018 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb193400640 1 -- 172.21.15.47:0/3624614153 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184035020 con 0x7fb194163070 2024-04-20T10:30:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb193400640 1 -- 172.21.15.47:0/3624614153 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb184031870 con 0x7fb194163070 2024-04-20T10:30:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb193400640 1 -- 172.21.15.47:0/3624614153 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184031c40 con 0x7fb194163070 2024-04-20T10:30:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb199a00640 1 -- 172.21.15.47:0/3624614153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb194163070 msgr2=0x7fb194071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb199a00640 1 --2- 172.21.15.47:0/3624614153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb194163070 0x7fb194071d90 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fb184009870 tx=0x7fb18402b910 comp rx=0 tx=0).stop 2024-04-20T10:30:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb199a00640 1 -- 172.21.15.47:0/3624614153 shutdown_connections 2024-04-20T10:30:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb199a00640 1 --2- 172.21.15.47:0/3624614153 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb194163070 0x7fb194071d90 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.019 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb199a00640 1 -- 172.21.15.47:0/3624614153 >> 172.21.15.47:0/3624614153 conn(0x7fb1940d0040 msgr2=0x7fb1940d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb199a00640 1 -- 172.21.15.47:0/3624614153 shutdown_connections 2024-04-20T10:30:27.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.019+0000 7fb199a00640 1 -- 172.21.15.47:0/3624614153 wait complete. 2024-04-20T10:30:27.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.021+0000 7fb199a00640 1 Processor -- start 2024-04-20T10:30:27.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.021+0000 7fb199a00640 1 -- start start 2024-04-20T10:30:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb199a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1940735e0 0x7fb19415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb199a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb194073a70 0x7fb19409b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.022 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb199a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb194163070 0x7fb19409c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb199a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941651d0 con 0x7fb1940735e0 2024-04-20T10:30:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb199a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1941662a0 con 0x7fb194163070 2024-04-20T10:30:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb199a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb194165350 con 0x7fb194073a70 2024-04-20T10:30:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb193e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1940735e0 0x7fb19415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb193e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1940735e0 0x7fb19415d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41180/0 (socket says 172.21.15.47:41180) 2024-04-20T10:30:27.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.022+0000 7fb193e00640 1 -- 172.21.15.47:0/535773534 learned_addr learned my addr 172.21.15.47:0/535773534 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb193e00640 1 -- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb194073a70 msgr2=0x7fb19409b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb193e00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb194073a70 0x7fb19409b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb199000640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb194163070 0x7fb19409c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb193e00640 1 -- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb194163070 msgr2=0x7fb19409c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb193e00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb194163070 0x7fb19409c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb193e00640 1 -- 172.21.15.47:0/535773534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1940b4d30 con 0x7fb1940735e0 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb199000640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb194163070 0x7fb19409c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.023+0000 7fb193e00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1940735e0 0x7fb19415d390 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb1840090b0 tx=0x7fb1840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:27.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.024+0000 7fb190c00640 1 -- 172.21.15.47:0/535773534 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18403ab50 con 0x7fb1940735e0 2024-04-20T10:30:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.024+0000 7fb190c00640 1 -- 172.21.15.47:0/535773534 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb18403ad10 con 0x7fb1940735e0 2024-04-20T10:30:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.024+0000 7fb190c00640 1 -- 172.21.15.47:0/535773534 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb184004270 con 0x7fb1940735e0 2024-04-20T10:30:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.024+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1940b5010 con 0x7fb1940735e0 2024-04-20T10:30:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.024+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1940b3190 con 0x7fb1940735e0 2024-04-20T10:30:27.025 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.025+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb194075b60 con 0x7fb1940735e0 2024-04-20T10:30:27.026 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.026+0000 7fb190c00640 1 -- 172.21.15.47:0/535773534 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb184038020 con 0x7fb1940735e0 2024-04-20T10:30:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.027+0000 7fb190c00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb17006c8f0 0x7fb17006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.027 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.028+0000 7fb190c00640 1 -- 172.21.15.47:0/535773534 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fb1840adc40 con 0x7fb1940735e0 2024-04-20T10:30:27.028 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.028+0000 7fb18be00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb17006c8f0 0x7fb17006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:27.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.029+0000 7fb18be00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb17006c8f0 0x7fb17006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb194164510 tx=0x7fb17c0079b0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:27.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.032+0000 7fb190c00640 1 -- 172.21.15.47:0/535773534 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb18403f070 con 0x7fb1940735e0 2024-04-20T10:30:27.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.235+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1940b33d0 con 0x7fb17006c8f0 2024-04-20T10:30:27.254 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.254+0000 7fb190c00640 1 -- 172.21.15.47:0/535773534 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251957 (secure 0 0 0) 0x7fb17c014400 con 0x7fb17006c8f0 2024-04-20T10:30:27.257 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:27.258 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:30:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.258+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb17006c8f0 msgr2=0x7fb17006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.258+0000 7fb199a00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb17006c8f0 0x7fb17006edb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb194164510 tx=0x7fb17c0079b0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1940735e0 msgr2=0x7fb19415d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1940735e0 0x7fb19415d390 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fb1840090b0 tx=0x7fb1840046c0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 shutdown_connections 2024-04-20T10:30:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb194163070 0x7fb19409c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb194073a70 0x7fb19409b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb17006c8f0 0x7fb17006edb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 --2- 172.21.15.47:0/535773534 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb1940735e0 0x7fb19415d390 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 >> 172.21.15.47:0/535773534 conn(0x7fb1940d0040 msgr2=0x7fb194071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 shutdown_connections 2024-04-20T10:30:27.261 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.259+0000 7fb199a00640 1 -- 172.21.15.47:0/535773534 wait complete. 2024-04-20T10:30:27.277 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":141,"stamp":"2024-04-20T10:30:27.130169+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":139,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239028,"kb_used_data":6184,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735692,"statfs":{"total":773094113280,"available":772849348608,"internally_reserved":0,"allocated":6332416,"data_stored":3422972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002105"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303289+0000","last_change":"2024-04-20T10:30:22.329273+0000","last_active":"2024-04-20T10:30:23.303289+0000","last_peered":"2024-04-20T10:30:23.303289+0000","last_clean":"2024-04-20T10:30:23.303289+0000","last_became_active":"2024-04-20T10:30:22.328446+0000","last_became_peered":"2024-04-20T10:30:22.328446+0000","last_unstale":"2024-04-20T10:30:23.303289+0000","last_undegraded":"2024-04-20T10:30:23.303289+0000","last_fullsized":"2024-04-20T10:30:23.303289+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:25:51.725829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299768+0000","last_change":"2024-04-20T10:30:22.324897+0000","last_active":"2024-04-20T10:30:23.299768+0000","last_peered":"2024-04-20T10:30:23.299768+0000","last_clean":"2024-04-20T10:30:23.299768+0000","last_became_active":"2024-04-20T10:30:22.324601+0000","last_became_peered":"2024-04-20T10:30:22.324601+0000","last_unstale":"2024-04-20T10:30:23.299768+0000","last_undegraded":"2024-04-20T10:30:23.299768+0000","last_fullsized":"2024-04-20T10:30:23.299768+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:25:13.018022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304416+0000","last_change":"2024-04-20T10:30:22.307386+0000","last_active":"2024-04-20T10:30:23.304416+0000","last_peered":"2024-04-20T10:30:23.304416+0000","last_clean":"2024-04-20T10:30:23.304416+0000","last_became_active":"2024-04-20T10:30:22.306653+0000","last_became_peered":"2024-04-20T10:30:22.306653+0000","last_unstale":"2024-04-20T10:30:23.304416+0000","last_undegraded":"2024-04-20T10:30:23.304416+0000","last_fullsized":"2024-04-20T10:30:23.304416+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:44:12.364253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449762+0000","last_change":"2024-04-20T10:30:22.328822+0000","last_active":"2024-04-20T10:30:23.449762+0000","last_peered":"2024-04-20T10:30:23.449762+0000","last_clean":"2024-04-20T10:30:23.449762+0000","last_became_active":"2024-04-20T10:30:22.325499+0000","last_became_peered":"2024-04-20T10:30:22.325499+0000","last_unstale":"2024-04-20T10:30:23.449762+0000","last_undegraded":"2024-04-20T10:30:23.449762+0000","last_fullsized":"2024-04-20T10:30:23.449762+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:25.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299464+0000","last_change":"2024-04-20T10:30:22.326315+0000","last_active":"2024-04-20T10:30:23.299464+0000","last_peered":"2024-04-20T10:30:23.299464+0000","last_clean":"2024-04-20T10:30:23.299464+0000","last_became_active":"2024-04-20T10:30:22.325905+0000","last_became_peered":"2024-04-20T10:30:22.325905+0000","last_unstale":"2024-04-20T10:30:23.299464+0000","last_undegraded":"2024-04-20T10:30:23.299464+0000","last_fullsized":"2024-04-20T10:30:23.299464+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:25.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481932+0000","last_change":"2024-04-20T10:30:22.323316+0000","last_active":"2024-04-20T10:30:23.481932+0000","last_peered":"2024-04-20T10:30:23.481932+0000","last_clean":"2024-04-20T10:30:23.481932+0000","last_became_active":"2024-04-20T10:30:22.322941+0000","last_became_peered":"2024-04-20T10:30:22.322941+0000","last_unstale":"2024-04-20T10:30:23.481932+0000","last_undegraded":"2024-04-20T10:30:23.481932+0000","last_fullsized":"2024-04-20T10:30:23.481932+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:10:34.132958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339337+0000","last_change":"2024-04-20T10:30:22.325907+0000","last_active":"2024-04-20T10:30:23.339337+0000","last_peered":"2024-04-20T10:30:23.339337+0000","last_clean":"2024-04-20T10:30:23.339337+0000","last_became_active":"2024-04-20T10:30:22.321317+0000","last_became_peered":"2024-04-20T10:30:22.321317+0000","last_unstale":"2024-04-20T10:30:23.339337+0000","last_undegraded":"2024-04-20T10:30:23.339337+0000","last_fullsized":"2024-04-20T10:30:23.339337+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:25.070053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481679+0000","last_change":"2024-04-20T10:30:22.323164+0000","last_active":"2024-04-20T10:30:23.481679+0000","last_peered":"2024-04-20T10:30:23.481679+0000","last_clean":"2024-04-20T10:30:23.481679+0000","last_became_active":"2024-04-20T10:30:22.322853+0000","last_became_peered":"2024-04-20T10:30:22.322853+0000","last_unstale":"2024-04-20T10:30:23.481679+0000","last_undegraded":"2024-04-20T10:30:23.481679+0000","last_fullsized":"2024-04-20T10:30:23.481679+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:28:18.396065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304856+0000","last_change":"2024-04-20T10:30:22.310232+0000","last_active":"2024-04-20T10:30:23.304856+0000","last_peered":"2024-04-20T10:30:23.304856+0000","last_clean":"2024-04-20T10:30:23.304856+0000","last_became_active":"2024-04-20T10:30:22.309893+0000","last_became_peered":"2024-04-20T10:30:22.309893+0000","last_unstale":"2024-04-20T10:30:23.304856+0000","last_undegraded":"2024-04-20T10:30:23.304856+0000","last_fullsized":"2024-04-20T10:30:23.304856+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:23:19.761399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340340+0000","last_change":"2024-04-20T10:30:22.334859+0000","last_active":"2024-04-20T10:30:23.340340+0000","last_peered":"2024-04-20T10:30:23.340340+0000","last_clean":"2024-04-20T10:30:23.340340+0000","last_became_active":"2024-04-20T10:30:22.334532+0000","last_became_peered":"2024-04-20T10:30:22.334532+0000","last_unstale":"2024-04-20T10:30:23.340340+0000","last_undegraded":"2024-04-20T10:30:23.340340+0000","last_fullsized":"2024-04-20T10:30:23.340340+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:22.708741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302516+0000","last_change":"2024-04-20T10:30:22.328972+0000","last_active":"2024-04-20T10:30:23.302516+0000","last_peered":"2024-04-20T10:30:23.302516+0000","last_clean":"2024-04-20T10:30:23.302516+0000","last_became_active":"2024-04-20T10:30:22.328579+0000","last_became_peered":"2024-04-20T10:30:22.328579+0000","last_unstale":"2024-04-20T10:30:23.302516+0000","last_undegraded":"2024-04-20T10:30:23.302516+0000","last_fullsized":"2024-04-20T10:30:23.302516+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:22.708741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482160+0000","last_change":"2024-04-20T10:30:22.334770+0000","last_active":"2024-04-20T10:30:23.482160+0000","last_peered":"2024-04-20T10:30:23.482160+0000","last_clean":"2024-04-20T10:30:23.482160+0000","last_became_active":"2024-04-20T10:30:22.334386+0000","last_became_peered":"2024-04-20T10:30:22.334386+0000","last_unstale":"2024-04-20T10:30:23.482160+0000","last_undegraded":"2024-04-20T10:30:23.482160+0000","last_fullsized":"2024-04-20T10:30:23.482160+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.268934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482127+0000","last_change":"2024-04-20T10:30:22.329131+0000","last_active":"2024-04-20T10:30:23.482127+0000","last_peered":"2024-04-20T10:30:23.482127+0000","last_clean":"2024-04-20T10:30:23.482127+0000","last_became_active":"2024-04-20T10:30:22.327492+0000","last_became_peered":"2024-04-20T10:30:22.327492+0000","last_unstale":"2024-04-20T10:30:23.482127+0000","last_undegraded":"2024-04-20T10:30:23.482127+0000","last_fullsized":"2024-04-20T10:30:23.482127+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:39.381432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302272+0000","last_change":"2024-04-20T10:30:22.330328+0000","last_active":"2024-04-20T10:30:23.302272+0000","last_peered":"2024-04-20T10:30:23.302272+0000","last_clean":"2024-04-20T10:30:23.302272+0000","last_became_active":"2024-04-20T10:30:22.330023+0000","last_became_peered":"2024-04-20T10:30:22.330023+0000","last_unstale":"2024-04-20T10:30:23.302272+0000","last_undegraded":"2024-04-20T10:30:23.302272+0000","last_fullsized":"2024-04-20T10:30:23.302272+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:20:54.410647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449318+0000","last_change":"2024-04-20T10:30:22.332397+0000","last_active":"2024-04-20T10:30:23.449318+0000","last_peered":"2024-04-20T10:30:23.449318+0000","last_clean":"2024-04-20T10:30:23.449318+0000","last_became_active":"2024-04-20T10:30:22.332041+0000","last_became_peered":"2024-04-20T10:30:22.332041+0000","last_unstale":"2024-04-20T10:30:23.449318+0000","last_undegraded":"2024-04-20T10:30:23.449318+0000","last_fullsized":"2024-04-20T10:30:23.449318+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:39.381432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481692+0000","last_change":"2024-04-20T10:30:22.332056+0000","last_active":"2024-04-20T10:30:23.481692+0000","last_peered":"2024-04-20T10:30:23.481692+0000","last_clean":"2024-04-20T10:30:23.481692+0000","last_became_active":"2024-04-20T10:30:22.331594+0000","last_became_peered":"2024-04-20T10:30:22.331594+0000","last_unstale":"2024-04-20T10:30:23.481692+0000","last_undegraded":"2024-04-20T10:30:23.481692+0000","last_fullsized":"2024-04-20T10:30:23.481692+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:34:52.181995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304105+0000","last_change":"2024-04-20T10:30:22.332431+0000","last_active":"2024-04-20T10:30:23.304105+0000","last_peered":"2024-04-20T10:30:23.304105+0000","last_clean":"2024-04-20T10:30:23.304105+0000","last_became_active":"2024-04-20T10:30:22.332101+0000","last_became_peered":"2024-04-20T10:30:22.332101+0000","last_unstale":"2024-04-20T10:30:23.304105+0000","last_undegraded":"2024-04-20T10:30:23.304105+0000","last_fullsized":"2024-04-20T10:30:23.304105+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:23.991051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449563+0000","last_change":"2024-04-20T10:30:22.325678+0000","last_active":"2024-04-20T10:30:23.449563+0000","last_peered":"2024-04-20T10:30:23.449563+0000","last_clean":"2024-04-20T10:30:23.449563+0000","last_became_active":"2024-04-20T10:30:22.324710+0000","last_became_peered":"2024-04-20T10:30:22.324710+0000","last_unstale":"2024-04-20T10:30:23.449563+0000","last_undegraded":"2024-04-20T10:30:23.449563+0000","last_fullsized":"2024-04-20T10:30:23.449563+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:11.088283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304644+0000","last_change":"2024-04-20T10:30:22.319721+0000","last_active":"2024-04-20T10:30:23.304644+0000","last_peered":"2024-04-20T10:30:23.304644+0000","last_clean":"2024-04-20T10:30:23.304644+0000","last_became_active":"2024-04-20T10:30:22.319421+0000","last_became_peered":"2024-04-20T10:30:22.319421+0000","last_unstale":"2024-04-20T10:30:23.304644+0000","last_undegraded":"2024-04-20T10:30:23.304644+0000","last_fullsized":"2024-04-20T10:30:23.304644+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:20:55.452831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299688+0000","last_change":"2024-04-20T10:30:22.324018+0000","last_active":"2024-04-20T10:30:23.299688+0000","last_peered":"2024-04-20T10:30:23.299688+0000","last_clean":"2024-04-20T10:30:23.299688+0000","last_became_active":"2024-04-20T10:30:22.323569+0000","last_became_peered":"2024-04-20T10:30:22.323569+0000","last_unstale":"2024-04-20T10:30:23.299688+0000","last_undegraded":"2024-04-20T10:30:23.299688+0000","last_fullsized":"2024-04-20T10:30:23.299688+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:40:57.935329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449374+0000","last_change":"2024-04-20T10:30:22.334240+0000","last_active":"2024-04-20T10:30:23.449374+0000","last_peered":"2024-04-20T10:30:23.449374+0000","last_clean":"2024-04-20T10:30:23.449374+0000","last_became_active":"2024-04-20T10:30:22.333946+0000","last_became_peered":"2024-04-20T10:30:22.333946+0000","last_unstale":"2024-04-20T10:30:23.449374+0000","last_undegraded":"2024-04-20T10:30:23.449374+0000","last_fullsized":"2024-04-20T10:30:23.449374+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:10:00.268934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303687+0000","last_change":"2024-04-20T10:30:22.320705+0000","last_active":"2024-04-20T10:30:23.303687+0000","last_peered":"2024-04-20T10:30:23.303687+0000","last_clean":"2024-04-20T10:30:23.303687+0000","last_became_active":"2024-04-20T10:30:22.320266+0000","last_became_peered":"2024-04-20T10:30:22.320266+0000","last_unstale":"2024-04-20T10:30:23.303687+0000","last_undegraded":"2024-04-20T10:30:23.303687+0000","last_fullsized":"2024-04-20T10:30:23.303687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:17:46.675848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302990+0000","last_change":"2024-04-20T10:30:22.311889+0000","last_active":"2024-04-20T10:30:23.302990+0000","last_peered":"2024-04-20T10:30:23.302990+0000","last_clean":"2024-04-20T10:30:23.302990+0000","last_became_active":"2024-04-20T10:30:22.311588+0000","last_became_peered":"2024-04-20T10:30:22.311588+0000","last_unstale":"2024-04-20T10:30:23.302990+0000","last_undegraded":"2024-04-20T10:30:23.302990+0000","last_fullsized":"2024-04-20T10:30:23.302990+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T13:32:55.222947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302524+0000","last_change":"2024-04-20T10:30:22.320501+0000","last_active":"2024-04-20T10:30:23.302524+0000","last_peered":"2024-04-20T10:30:23.302524+0000","last_clean":"2024-04-20T10:30:23.302524+0000","last_became_active":"2024-04-20T10:30:22.320193+0000","last_became_peered":"2024-04-20T10:30:22.320193+0000","last_unstale":"2024-04-20T10:30:23.302524+0000","last_undegraded":"2024-04-20T10:30:23.302524+0000","last_fullsized":"2024-04-20T10:30:23.302524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:51:17.001959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303346+0000","last_change":"2024-04-20T10:30:22.322865+0000","last_active":"2024-04-20T10:30:23.303346+0000","last_peered":"2024-04-20T10:30:23.303346+0000","last_clean":"2024-04-20T10:30:23.303346+0000","last_became_active":"2024-04-20T10:30:22.322214+0000","last_became_peered":"2024-04-20T10:30:22.322214+0000","last_unstale":"2024-04-20T10:30:23.303346+0000","last_undegraded":"2024-04-20T10:30:23.303346+0000","last_fullsized":"2024-04-20T10:30:23.303346+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:20:53.186380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.305122+0000","last_change":"2024-04-20T10:30:22.325203+0000","last_active":"2024-04-20T10:30:23.305122+0000","last_peered":"2024-04-20T10:30:23.305122+0000","last_clean":"2024-04-20T10:30:23.305122+0000","last_became_active":"2024-04-20T10:30:22.324096+0000","last_became_peered":"2024-04-20T10:30:22.324096+0000","last_unstale":"2024-04-20T10:30:23.305122+0000","last_undegraded":"2024-04-20T10:30:23.305122+0000","last_fullsized":"2024-04-20T10:30:23.305122+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:13:50.899499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482431+0000","last_change":"2024-04-20T10:30:22.336175+0000","last_active":"2024-04-20T10:30:23.482431+0000","last_peered":"2024-04-20T10:30:23.482431+0000","last_clean":"2024-04-20T10:30:23.482431+0000","last_became_active":"2024-04-20T10:30:22.335854+0000","last_became_peered":"2024-04-20T10:30:22.335854+0000","last_unstale":"2024-04-20T10:30:23.482431+0000","last_undegraded":"2024-04-20T10:30:23.482431+0000","last_fullsized":"2024-04-20T10:30:23.482431+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:11.088283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299789+0000","last_change":"2024-04-20T10:30:22.332616+0000","last_active":"2024-04-20T10:30:23.299789+0000","last_peered":"2024-04-20T10:30:23.299789+0000","last_clean":"2024-04-20T10:30:23.299789+0000","last_became_active":"2024-04-20T10:30:22.331862+0000","last_became_peered":"2024-04-20T10:30:22.331862+0000","last_unstale":"2024-04-20T10:30:23.299789+0000","last_undegraded":"2024-04-20T10:30:23.299789+0000","last_fullsized":"2024-04-20T10:30:23.299789+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:46.072670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302262+0000","last_change":"2024-04-20T10:30:22.329311+0000","last_active":"2024-04-20T10:30:23.302262+0000","last_peered":"2024-04-20T10:30:23.302262+0000","last_clean":"2024-04-20T10:30:23.302262+0000","last_became_active":"2024-04-20T10:30:22.328470+0000","last_became_peered":"2024-04-20T10:30:22.328470+0000","last_unstale":"2024-04-20T10:30:23.302262+0000","last_undegraded":"2024-04-20T10:30:23.302262+0000","last_fullsized":"2024-04-20T10:30:23.302262+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:58.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480119+0000","last_change":"2024-04-20T10:30:22.321402+0000","last_active":"2024-04-20T10:30:23.480119+0000","last_peered":"2024-04-20T10:30:23.480119+0000","last_clean":"2024-04-20T10:30:23.480119+0000","last_became_active":"2024-04-20T10:30:22.321086+0000","last_became_peered":"2024-04-20T10:30:22.321086+0000","last_unstale":"2024-04-20T10:30:23.480119+0000","last_undegraded":"2024-04-20T10:30:23.480119+0000","last_fullsized":"2024-04-20T10:30:23.480119+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T10:52:07.312630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303408+0000","last_change":"2024-04-20T10:30:22.328382+0000","last_active":"2024-04-20T10:30:23.303408+0000","last_peered":"2024-04-20T10:30:23.303408+0000","last_clean":"2024-04-20T10:30:23.303408+0000","last_became_active":"2024-04-20T10:30:22.327878+0000","last_became_peered":"2024-04-20T10:30:22.327878+0000","last_unstale":"2024-04-20T10:30:23.303408+0000","last_undegraded":"2024-04-20T10:30:23.303408+0000","last_fullsized":"2024-04-20T10:30:23.303408+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:40:50.945796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.298738+0000","last_change":"2024-04-20T10:30:22.308150+0000","last_active":"2024-04-20T10:30:23.298738+0000","last_peered":"2024-04-20T10:30:23.298738+0000","last_clean":"2024-04-20T10:30:23.298738+0000","last_became_active":"2024-04-20T10:30:22.307372+0000","last_became_peered":"2024-04-20T10:30:22.307372+0000","last_unstale":"2024-04-20T10:30:23.298738+0000","last_undegraded":"2024-04-20T10:30:23.298738+0000","last_fullsized":"2024-04-20T10:30:23.298738+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:11.088283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480550+0000","last_change":"2024-04-20T10:30:22.324198+0000","last_active":"2024-04-20T10:30:23.480550+0000","last_peered":"2024-04-20T10:30:23.480550+0000","last_clean":"2024-04-20T10:30:23.480550+0000","last_became_active":"2024-04-20T10:30:22.319764+0000","last_became_peered":"2024-04-20T10:30:22.319764+0000","last_unstale":"2024-04-20T10:30:23.480550+0000","last_undegraded":"2024-04-20T10:30:23.480550+0000","last_fullsized":"2024-04-20T10:30:23.480550+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:47:37.783903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.346511+0000","last_change":"2024-04-20T10:30:21.292596+0000","last_active":"2024-04-20T10:30:23.346511+0000","last_peered":"2024-04-20T10:30:23.346511+0000","last_clean":"2024-04-20T10:30:23.346511+0000","last_became_active":"2024-04-20T10:30:21.291679+0000","last_became_peered":"2024-04-20T10:30:21.291679+0000","last_unstale":"2024-04-20T10:30:23.346511+0000","last_undegraded":"2024-04-20T10:30:23.346511+0000","last_fullsized":"2024-04-20T10:30:23.346511+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:24.601472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480062+0000","last_change":"2024-04-20T10:30:22.334602+0000","last_active":"2024-04-20T10:30:23.480062+0000","last_peered":"2024-04-20T10:30:23.480062+0000","last_clean":"2024-04-20T10:30:23.480062+0000","last_became_active":"2024-04-20T10:30:22.334325+0000","last_became_peered":"2024-04-20T10:30:22.334325+0000","last_unstale":"2024-04-20T10:30:23.480062+0000","last_undegraded":"2024-04-20T10:30:23.480062+0000","last_fullsized":"2024-04-20T10:30:23.480062+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:21:19.475972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482534+0000","last_change":"2024-04-20T10:30:21.305701+0000","last_active":"2024-04-20T10:30:23.482534+0000","last_peered":"2024-04-20T10:30:23.482534+0000","last_clean":"2024-04-20T10:30:23.482534+0000","last_became_active":"2024-04-20T10:30:21.305374+0000","last_became_peered":"2024-04-20T10:30:21.305374+0000","last_unstale":"2024-04-20T10:30:23.482534+0000","last_undegraded":"2024-04-20T10:30:23.482534+0000","last_fullsized":"2024-04-20T10:30:23.482534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:25:54.048126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.479958+0000","last_change":"2024-04-20T10:30:22.320138+0000","last_active":"2024-04-20T10:30:23.479958+0000","last_peered":"2024-04-20T10:30:23.479958+0000","last_clean":"2024-04-20T10:30:23.479958+0000","last_became_active":"2024-04-20T10:30:22.319823+0000","last_became_peered":"2024-04-20T10:30:22.319823+0000","last_unstale":"2024-04-20T10:30:23.479958+0000","last_undegraded":"2024-04-20T10:30:23.479958+0000","last_fullsized":"2024-04-20T10:30:23.479958+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:40:44.913550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339955+0000","last_change":"2024-04-20T10:30:21.299006+0000","last_active":"2024-04-20T10:30:23.339955+0000","last_peered":"2024-04-20T10:30:23.339955+0000","last_clean":"2024-04-20T10:30:23.339955+0000","last_became_active":"2024-04-20T10:30:21.298665+0000","last_became_peered":"2024-04-20T10:30:21.298665+0000","last_unstale":"2024-04-20T10:30:23.339955+0000","last_undegraded":"2024-04-20T10:30:23.339955+0000","last_fullsized":"2024-04-20T10:30:23.339955+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:12:10.074085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339743+0000","last_change":"2024-04-20T10:30:22.323657+0000","last_active":"2024-04-20T10:30:23.339743+0000","last_peered":"2024-04-20T10:30:23.339743+0000","last_clean":"2024-04-20T10:30:23.339743+0000","last_became_active":"2024-04-20T10:30:22.323314+0000","last_became_peered":"2024-04-20T10:30:22.323314+0000","last_unstale":"2024-04-20T10:30:23.339743+0000","last_undegraded":"2024-04-20T10:30:23.339743+0000","last_fullsized":"2024-04-20T10:30:23.339743+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:53:17.743013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299946+0000","last_change":"2024-04-20T10:30:21.288809+0000","last_active":"2024-04-20T10:30:23.299946+0000","last_peered":"2024-04-20T10:30:23.299946+0000","last_clean":"2024-04-20T10:30:23.299946+0000","last_became_active":"2024-04-20T10:30:21.288378+0000","last_became_peered":"2024-04-20T10:30:21.288378+0000","last_unstale":"2024-04-20T10:30:23.299946+0000","last_undegraded":"2024-04-20T10:30:23.299946+0000","last_fullsized":"2024-04-20T10:30:23.299946+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:37:15.106115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.346293+0000","last_change":"2024-04-20T10:30:22.328111+0000","last_active":"2024-04-20T10:30:23.346293+0000","last_peered":"2024-04-20T10:30:23.346293+0000","last_clean":"2024-04-20T10:30:23.346293+0000","last_became_active":"2024-04-20T10:30:22.327038+0000","last_became_peered":"2024-04-20T10:30:22.327038+0000","last_unstale":"2024-04-20T10:30:23.346293+0000","last_undegraded":"2024-04-20T10:30:23.346293+0000","last_fullsized":"2024-04-20T10:30:23.346293+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:27:25.070053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304203+0000","last_change":"2024-04-20T10:30:21.302553+0000","last_active":"2024-04-20T10:30:23.304203+0000","last_peered":"2024-04-20T10:30:23.304203+0000","last_clean":"2024-04-20T10:30:23.304203+0000","last_became_active":"2024-04-20T10:30:21.302147+0000","last_became_peered":"2024-04-20T10:30:21.302147+0000","last_unstale":"2024-04-20T10:30:23.304203+0000","last_undegraded":"2024-04-20T10:30:23.304203+0000","last_fullsized":"2024-04-20T10:30:23.304203+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:25:01.405342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480314+0000","last_change":"2024-04-20T10:30:22.315964+0000","last_active":"2024-04-20T10:30:23.480314+0000","last_peered":"2024-04-20T10:30:23.480314+0000","last_clean":"2024-04-20T10:30:23.480314+0000","last_became_active":"2024-04-20T10:30:22.315667+0000","last_became_peered":"2024-04-20T10:30:22.315667+0000","last_unstale":"2024-04-20T10:30:23.480314+0000","last_undegraded":"2024-04-20T10:30:23.480314+0000","last_fullsized":"2024-04-20T10:30:23.480314+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:45:55.281029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480112+0000","last_change":"2024-04-20T10:30:21.292948+0000","last_active":"2024-04-20T10:30:23.480112+0000","last_peered":"2024-04-20T10:30:23.480112+0000","last_clean":"2024-04-20T10:30:23.480112+0000","last_became_active":"2024-04-20T10:30:21.292660+0000","last_became_peered":"2024-04-20T10:30:21.292660+0000","last_unstale":"2024-04-20T10:30:23.480112+0000","last_undegraded":"2024-04-20T10:30:23.480112+0000","last_fullsized":"2024-04-20T10:30:23.480112+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:23:49.074077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340420+0000","last_change":"2024-04-20T10:30:22.324959+0000","last_active":"2024-04-20T10:30:23.340420+0000","last_peered":"2024-04-20T10:30:23.340420+0000","last_clean":"2024-04-20T10:30:23.340420+0000","last_became_active":"2024-04-20T10:30:22.324664+0000","last_became_peered":"2024-04-20T10:30:22.324664+0000","last_unstale":"2024-04-20T10:30:23.340420+0000","last_undegraded":"2024-04-20T10:30:23.340420+0000","last_fullsized":"2024-04-20T10:30:23.340420+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:31.622075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.479872+0000","last_change":"2024-04-20T10:30:21.293158+0000","last_active":"2024-04-20T10:30:23.479872+0000","last_peered":"2024-04-20T10:30:23.479872+0000","last_clean":"2024-04-20T10:30:23.479872+0000","last_became_active":"2024-04-20T10:30:21.292880+0000","last_became_peered":"2024-04-20T10:30:21.292880+0000","last_unstale":"2024-04-20T10:30:23.479872+0000","last_undegraded":"2024-04-20T10:30:23.479872+0000","last_fullsized":"2024-04-20T10:30:23.479872+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:56:10.697220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449641+0000","last_change":"2024-04-20T10:30:22.331158+0000","last_active":"2024-04-20T10:30:23.449641+0000","last_peered":"2024-04-20T10:30:23.449641+0000","last_clean":"2024-04-20T10:30:23.449641+0000","last_became_active":"2024-04-20T10:30:22.330864+0000","last_became_peered":"2024-04-20T10:30:22.330864+0000","last_unstale":"2024-04-20T10:30:23.449641+0000","last_undegraded":"2024-04-20T10:30:23.449641+0000","last_fullsized":"2024-04-20T10:30:23.449641+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:00.274563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302446+0000","last_change":"2024-04-20T10:30:21.294347+0000","last_active":"2024-04-20T10:30:23.302446+0000","last_peered":"2024-04-20T10:30:23.302446+0000","last_clean":"2024-04-20T10:30:23.302446+0000","last_became_active":"2024-04-20T10:30:21.294024+0000","last_became_peered":"2024-04-20T10:30:21.294024+0000","last_unstale":"2024-04-20T10:30:23.302446+0000","last_undegraded":"2024-04-20T10:30:23.302446+0000","last_fullsized":"2024-04-20T10:30:23.302446+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:30.247444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.345642+0000","last_change":"2024-04-20T10:30:22.324683+0000","last_active":"2024-04-20T10:30:23.345642+0000","last_peered":"2024-04-20T10:30:23.345642+0000","last_clean":"2024-04-20T10:30:23.345642+0000","last_became_active":"2024-04-20T10:30:22.324398+0000","last_became_peered":"2024-04-20T10:30:22.324398+0000","last_unstale":"2024-04-20T10:30:23.345642+0000","last_undegraded":"2024-04-20T10:30:23.345642+0000","last_fullsized":"2024-04-20T10:30:23.345642+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:44:46.072670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449759+0000","last_change":"2024-04-20T10:30:21.291573+0000","last_active":"2024-04-20T10:30:23.449759+0000","last_peered":"2024-04-20T10:30:23.449759+0000","last_clean":"2024-04-20T10:30:23.449759+0000","last_became_active":"2024-04-20T10:30:21.291286+0000","last_became_peered":"2024-04-20T10:30:21.291286+0000","last_unstale":"2024-04-20T10:30:23.449759+0000","last_undegraded":"2024-04-20T10:30:23.449759+0000","last_fullsized":"2024-04-20T10:30:23.449759+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:37:49.257466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449306+0000","last_change":"2024-04-20T10:30:22.326580+0000","last_active":"2024-04-20T10:30:23.449306+0000","last_peered":"2024-04-20T10:30:23.449306+0000","last_clean":"2024-04-20T10:30:23.449306+0000","last_became_active":"2024-04-20T10:30:22.326307+0000","last_became_peered":"2024-04-20T10:30:22.326307+0000","last_unstale":"2024-04-20T10:30:23.449306+0000","last_undegraded":"2024-04-20T10:30:23.449306+0000","last_fullsized":"2024-04-20T10:30:23.449306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:10:34.132958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449520+0000","last_change":"2024-04-20T10:30:21.298825+0000","last_active":"2024-04-20T10:30:23.449520+0000","last_peered":"2024-04-20T10:30:23.449520+0000","last_clean":"2024-04-20T10:30:23.449520+0000","last_became_active":"2024-04-20T10:30:21.298551+0000","last_became_peered":"2024-04-20T10:30:21.298551+0000","last_unstale":"2024-04-20T10:30:23.449520+0000","last_undegraded":"2024-04-20T10:30:23.449520+0000","last_fullsized":"2024-04-20T10:30:23.449520+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:51:51.570079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302827+0000","last_change":"2024-04-20T10:30:22.311712+0000","last_active":"2024-04-20T10:30:23.302827+0000","last_peered":"2024-04-20T10:30:23.302827+0000","last_clean":"2024-04-20T10:30:23.302827+0000","last_became_active":"2024-04-20T10:30:22.311357+0000","last_became_peered":"2024-04-20T10:30:22.311357+0000","last_unstale":"2024-04-20T10:30:23.302827+0000","last_undegraded":"2024-04-20T10:30:23.302827+0000","last_fullsized":"2024-04-20T10:30:23.302827+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:42:25.740328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480400+0000","last_change":"2024-04-20T10:30:21.300138+0000","last_active":"2024-04-20T10:30:23.480400+0000","last_peered":"2024-04-20T10:30:23.480400+0000","last_clean":"2024-04-20T10:30:23.480400+0000","last_became_active":"2024-04-20T10:30:21.299612+0000","last_became_peered":"2024-04-20T10:30:21.299612+0000","last_unstale":"2024-04-20T10:30:23.480400+0000","last_undegraded":"2024-04-20T10:30:23.480400+0000","last_fullsized":"2024-04-20T10:30:23.480400+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:34:25.754831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299272+0000","last_change":"2024-04-20T10:30:22.319367+0000","last_active":"2024-04-20T10:30:23.299272+0000","last_peered":"2024-04-20T10:30:23.299272+0000","last_clean":"2024-04-20T10:30:23.299272+0000","last_became_active":"2024-04-20T10:30:22.318764+0000","last_became_peered":"2024-04-20T10:30:22.318764+0000","last_unstale":"2024-04-20T10:30:23.299272+0000","last_undegraded":"2024-04-20T10:30:23.299272+0000","last_fullsized":"2024-04-20T10:30:23.299272+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:53.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340300+0000","last_change":"2024-04-20T10:30:21.306997+0000","last_active":"2024-04-20T10:30:23.340300+0000","last_peered":"2024-04-20T10:30:23.340300+0000","last_clean":"2024-04-20T10:30:23.340300+0000","last_became_active":"2024-04-20T10:30:21.306251+0000","last_became_peered":"2024-04-20T10:30:21.306251+0000","last_unstale":"2024-04-20T10:30:23.340300+0000","last_undegraded":"2024-04-20T10:30:23.340300+0000","last_fullsized":"2024-04-20T10:30:23.340300+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:18:38.367235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480320+0000","last_change":"2024-04-20T10:30:22.312201+0000","last_active":"2024-04-20T10:30:23.480320+0000","last_peered":"2024-04-20T10:30:23.480320+0000","last_clean":"2024-04-20T10:30:23.480320+0000","last_became_active":"2024-04-20T10:30:22.311635+0000","last_became_peered":"2024-04-20T10:30:22.311635+0000","last_unstale":"2024-04-20T10:30:23.480320+0000","last_undegraded":"2024-04-20T10:30:23.480320+0000","last_fullsized":"2024-04-20T10:30:23.480320+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:33:23.961725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482067+0000","last_change":"2024-04-20T10:30:21.299360+0000","last_active":"2024-04-20T10:30:23.482067+0000","last_peered":"2024-04-20T10:30:23.482067+0000","last_clean":"2024-04-20T10:30:23.482067+0000","last_became_active":"2024-04-20T10:30:21.299119+0000","last_became_peered":"2024-04-20T10:30:21.299119+0000","last_unstale":"2024-04-20T10:30:23.482067+0000","last_undegraded":"2024-04-20T10:30:23.482067+0000","last_fullsized":"2024-04-20T10:30:23.482067+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T22:00:00.556201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302709+0000","last_change":"2024-04-20T10:30:22.322825+0000","last_active":"2024-04-20T10:30:23.302709+0000","last_peered":"2024-04-20T10:30:23.302709+0000","last_clean":"2024-04-20T10:30:23.302709+0000","last_became_active":"2024-04-20T10:30:22.322526+0000","last_became_peered":"2024-04-20T10:30:22.322526+0000","last_unstale":"2024-04-20T10:30:23.302709+0000","last_undegraded":"2024-04-20T10:30:23.302709+0000","last_fullsized":"2024-04-20T10:30:23.302709+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:35:31.622075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449348+0000","last_change":"2024-04-20T10:30:21.296780+0000","last_active":"2024-04-20T10:30:23.449348+0000","last_peered":"2024-04-20T10:30:23.449348+0000","last_clean":"2024-04-20T10:30:23.449348+0000","last_became_active":"2024-04-20T10:30:21.295427+0000","last_became_peered":"2024-04-20T10:30:21.295427+0000","last_unstale":"2024-04-20T10:30:23.449348+0000","last_undegraded":"2024-04-20T10:30:23.449348+0000","last_fullsized":"2024-04-20T10:30:23.449348+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:42:52.984671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482761+0000","last_change":"2024-04-20T10:30:22.329140+0000","last_active":"2024-04-20T10:30:23.482761+0000","last_peered":"2024-04-20T10:30:23.482761+0000","last_clean":"2024-04-20T10:30:23.482761+0000","last_became_active":"2024-04-20T10:30:22.328862+0000","last_became_peered":"2024-04-20T10:30:22.328862+0000","last_unstale":"2024-04-20T10:30:23.482761+0000","last_undegraded":"2024-04-20T10:30:23.482761+0000","last_fullsized":"2024-04-20T10:30:23.482761+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:53.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.298813+0000","last_change":"2024-04-20T10:30:21.290598+0000","last_active":"2024-04-20T10:30:23.298813+0000","last_peered":"2024-04-20T10:30:23.298813+0000","last_clean":"2024-04-20T10:30:23.298813+0000","last_became_active":"2024-04-20T10:30:21.290202+0000","last_became_peered":"2024-04-20T10:30:21.290202+0000","last_unstale":"2024-04-20T10:30:23.298813+0000","last_undegraded":"2024-04-20T10:30:23.298813+0000","last_fullsized":"2024-04-20T10:30:23.298813+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:15:41.804702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.305034+0000","last_change":"2024-04-20T10:30:22.333023+0000","last_active":"2024-04-20T10:30:23.305034+0000","last_peered":"2024-04-20T10:30:23.305034+0000","last_clean":"2024-04-20T10:30:23.305034+0000","last_became_active":"2024-04-20T10:30:22.332711+0000","last_became_peered":"2024-04-20T10:30:22.332711+0000","last_unstale":"2024-04-20T10:30:23.305034+0000","last_undegraded":"2024-04-20T10:30:23.305034+0000","last_fullsized":"2024-04-20T10:30:23.305034+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:38:31.579201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482019+0000","last_change":"2024-04-20T10:30:21.305877+0000","last_active":"2024-04-20T10:30:23.482019+0000","last_peered":"2024-04-20T10:30:23.482019+0000","last_clean":"2024-04-20T10:30:23.482019+0000","last_became_active":"2024-04-20T10:30:21.305421+0000","last_became_peered":"2024-04-20T10:30:21.305421+0000","last_unstale":"2024-04-20T10:30:23.482019+0000","last_undegraded":"2024-04-20T10:30:23.482019+0000","last_fullsized":"2024-04-20T10:30:23.482019+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:46:34.127797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340018+0000","last_change":"2024-04-20T10:30:22.326109+0000","last_active":"2024-04-20T10:30:23.340018+0000","last_peered":"2024-04-20T10:30:23.340018+0000","last_clean":"2024-04-20T10:30:23.340018+0000","last_became_active":"2024-04-20T10:30:22.325559+0000","last_became_peered":"2024-04-20T10:30:22.325559+0000","last_unstale":"2024-04-20T10:30:23.340018+0000","last_undegraded":"2024-04-20T10:30:23.340018+0000","last_fullsized":"2024-04-20T10:30:23.340018+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T16:20:54.410647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482942+0000","last_change":"2024-04-20T10:30:21.306532+0000","last_active":"2024-04-20T10:30:23.482942+0000","last_peered":"2024-04-20T10:30:23.482942+0000","last_clean":"2024-04-20T10:30:23.482942+0000","last_became_active":"2024-04-20T10:30:21.305377+0000","last_became_peered":"2024-04-20T10:30:21.305377+0000","last_unstale":"2024-04-20T10:30:23.482942+0000","last_undegraded":"2024-04-20T10:30:23.482942+0000","last_fullsized":"2024-04-20T10:30:23.482942+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:55:25.206126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.301691+0000","last_change":"2024-04-20T10:30:22.319088+0000","last_active":"2024-04-20T10:30:23.301691+0000","last_peered":"2024-04-20T10:30:23.301691+0000","last_clean":"2024-04-20T10:30:23.301691+0000","last_became_active":"2024-04-20T10:30:22.318684+0000","last_became_peered":"2024-04-20T10:30:22.318684+0000","last_unstale":"2024-04-20T10:30:23.301691+0000","last_undegraded":"2024-04-20T10:30:23.301691+0000","last_fullsized":"2024-04-20T10:30:23.301691+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:26.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482736+0000","last_change":"2024-04-20T10:30:21.302031+0000","last_active":"2024-04-20T10:30:23.482736+0000","last_peered":"2024-04-20T10:30:23.482736+0000","last_clean":"2024-04-20T10:30:23.482736+0000","last_became_active":"2024-04-20T10:30:21.301752+0000","last_became_peered":"2024-04-20T10:30:21.301752+0000","last_unstale":"2024-04-20T10:30:23.482736+0000","last_undegraded":"2024-04-20T10:30:23.482736+0000","last_fullsized":"2024-04-20T10:30:23.482736+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:48.262531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299149+0000","last_change":"2024-04-20T10:30:22.330549+0000","last_active":"2024-04-20T10:30:23.299149+0000","last_peered":"2024-04-20T10:30:23.299149+0000","last_clean":"2024-04-20T10:30:23.299149+0000","last_became_active":"2024-04-20T10:30:22.330271+0000","last_became_peered":"2024-04-20T10:30:22.330271+0000","last_unstale":"2024-04-20T10:30:23.299149+0000","last_undegraded":"2024-04-20T10:30:23.299149+0000","last_fullsized":"2024-04-20T10:30:23.299149+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:00.274563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449680+0000","last_change":"2024-04-20T10:30:21.301918+0000","last_active":"2024-04-20T10:30:23.449680+0000","last_peered":"2024-04-20T10:30:23.449680+0000","last_clean":"2024-04-20T10:30:23.449680+0000","last_became_active":"2024-04-20T10:30:21.301595+0000","last_became_peered":"2024-04-20T10:30:21.301595+0000","last_unstale":"2024-04-20T10:30:23.449680+0000","last_undegraded":"2024-04-20T10:30:23.449680+0000","last_fullsized":"2024-04-20T10:30:23.449680+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:36:48.262531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.481783+0000","last_change":"2024-04-20T10:30:22.334190+0000","last_active":"2024-04-20T10:30:23.481783+0000","last_peered":"2024-04-20T10:30:23.481783+0000","last_clean":"2024-04-20T10:30:23.481783+0000","last_became_active":"2024-04-20T10:30:22.333836+0000","last_became_peered":"2024-04-20T10:30:22.333836+0000","last_unstale":"2024-04-20T10:30:23.481783+0000","last_undegraded":"2024-04-20T10:30:23.481783+0000","last_fullsized":"2024-04-20T10:30:23.481783+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:16.425728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.300042+0000","last_change":"2024-04-20T10:30:21.302296+0000","last_active":"2024-04-20T10:30:23.300042+0000","last_peered":"2024-04-20T10:30:23.300042+0000","last_clean":"2024-04-20T10:30:23.300042+0000","last_became_active":"2024-04-20T10:30:21.301965+0000","last_became_peered":"2024-04-20T10:30:21.301965+0000","last_unstale":"2024-04-20T10:30:23.300042+0000","last_undegraded":"2024-04-20T10:30:23.300042+0000","last_fullsized":"2024-04-20T10:30:23.300042+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:26:01.953689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299899+0000","last_change":"2024-04-20T10:30:22.327551+0000","last_active":"2024-04-20T10:30:23.299899+0000","last_peered":"2024-04-20T10:30:23.299899+0000","last_clean":"2024-04-20T10:30:23.299899+0000","last_became_active":"2024-04-20T10:30:22.326605+0000","last_became_peered":"2024-04-20T10:30:22.326605+0000","last_unstale":"2024-04-20T10:30:23.299899+0000","last_undegraded":"2024-04-20T10:30:23.299899+0000","last_fullsized":"2024-04-20T10:30:23.299899+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:03.855614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.302231+0000","last_change":"2024-04-20T10:30:21.290127+0000","last_active":"2024-04-20T10:30:23.302231+0000","last_peered":"2024-04-20T10:30:23.302231+0000","last_clean":"2024-04-20T10:30:23.302231+0000","last_became_active":"2024-04-20T10:30:21.289602+0000","last_became_peered":"2024-04-20T10:30:21.289602+0000","last_unstale":"2024-04-20T10:30:23.302231+0000","last_undegraded":"2024-04-20T10:30:23.302231+0000","last_fullsized":"2024-04-20T10:30:23.302231+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:59.254736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449674+0000","last_change":"2024-04-20T10:30:22.332438+0000","last_active":"2024-04-20T10:30:23.449674+0000","last_peered":"2024-04-20T10:30:23.449674+0000","last_clean":"2024-04-20T10:30:23.449674+0000","last_became_active":"2024-04-20T10:30:22.331431+0000","last_became_peered":"2024-04-20T10:30:22.331431+0000","last_unstale":"2024-04-20T10:30:23.449674+0000","last_undegraded":"2024-04-20T10:30:23.449674+0000","last_fullsized":"2024-04-20T10:30:23.449674+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:56:53.848853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303554+0000","last_change":"2024-04-20T10:30:21.299403+0000","last_active":"2024-04-20T10:30:23.303554+0000","last_peered":"2024-04-20T10:30:23.303554+0000","last_clean":"2024-04-20T10:30:23.303554+0000","last_became_active":"2024-04-20T10:30:21.298263+0000","last_became_peered":"2024-04-20T10:30:21.298263+0000","last_unstale":"2024-04-20T10:30:23.303554+0000","last_undegraded":"2024-04-20T10:30:23.303554+0000","last_fullsized":"2024-04-20T10:30:23.303554+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:48:40.408177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.301994+0000","last_change":"2024-04-20T10:30:22.314238+0000","last_active":"2024-04-20T10:30:23.301994+0000","last_peered":"2024-04-20T10:30:23.301994+0000","last_clean":"2024-04-20T10:30:23.301994+0000","last_became_active":"2024-04-20T10:30:22.313673+0000","last_became_peered":"2024-04-20T10:30:22.313673+0000","last_unstale":"2024-04-20T10:30:23.301994+0000","last_undegraded":"2024-04-20T10:30:23.301994+0000","last_fullsized":"2024-04-20T10:30:23.301994+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:53:17.743013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339315+0000","last_change":"2024-04-20T10:30:21.306163+0000","last_active":"2024-04-20T10:30:23.339315+0000","last_peered":"2024-04-20T10:30:23.339315+0000","last_clean":"2024-04-20T10:30:23.339315+0000","last_became_active":"2024-04-20T10:30:21.305841+0000","last_became_peered":"2024-04-20T10:30:21.305841+0000","last_unstale":"2024-04-20T10:30:23.339315+0000","last_undegraded":"2024-04-20T10:30:23.339315+0000","last_fullsized":"2024-04-20T10:30:23.339315+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:10:33.118760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482985+0000","last_change":"2024-04-20T10:30:22.337266+0000","last_active":"2024-04-20T10:30:23.482985+0000","last_peered":"2024-04-20T10:30:23.482985+0000","last_clean":"2024-04-20T10:30:23.482985+0000","last_became_active":"2024-04-20T10:30:22.336913+0000","last_became_peered":"2024-04-20T10:30:22.336913+0000","last_unstale":"2024-04-20T10:30:23.482985+0000","last_undegraded":"2024-04-20T10:30:23.482985+0000","last_fullsized":"2024-04-20T10:30:23.482985+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:55:25.615670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339487+0000","last_change":"2024-04-20T10:30:21.298825+0000","last_active":"2024-04-20T10:30:23.339487+0000","last_peered":"2024-04-20T10:30:23.339487+0000","last_clean":"2024-04-20T10:30:23.339487+0000","last_became_active":"2024-04-20T10:30:21.298517+0000","last_became_peered":"2024-04-20T10:30:21.298517+0000","last_unstale":"2024-04-20T10:30:23.339487+0000","last_undegraded":"2024-04-20T10:30:23.339487+0000","last_fullsized":"2024-04-20T10:30:23.339487+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:28:17.381867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449087+0000","last_change":"2024-04-20T10:30:22.329795+0000","last_active":"2024-04-20T10:30:23.449087+0000","last_peered":"2024-04-20T10:30:23.449087+0000","last_clean":"2024-04-20T10:30:23.449087+0000","last_became_active":"2024-04-20T10:30:22.328182+0000","last_became_peered":"2024-04-20T10:30:22.328182+0000","last_unstale":"2024-04-20T10:30:23.449087+0000","last_undegraded":"2024-04-20T10:30:23.449087+0000","last_fullsized":"2024-04-20T10:30:23.449087+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:16.425728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480223+0000","last_change":"2024-04-20T10:30:21.292488+0000","last_active":"2024-04-20T10:30:23.480223+0000","last_peered":"2024-04-20T10:30:23.480223+0000","last_clean":"2024-04-20T10:30:23.480223+0000","last_became_active":"2024-04-20T10:30:21.292190+0000","last_became_peered":"2024-04-20T10:30:21.292190+0000","last_unstale":"2024-04-20T10:30:23.480223+0000","last_undegraded":"2024-04-20T10:30:23.480223+0000","last_fullsized":"2024-04-20T10:30:23.480223+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:32:36.003651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339394+0000","last_change":"2024-04-20T10:30:22.324315+0000","last_active":"2024-04-20T10:30:23.339394+0000","last_peered":"2024-04-20T10:30:23.339394+0000","last_clean":"2024-04-20T10:30:23.339394+0000","last_became_active":"2024-04-20T10:30:22.324038+0000","last_became_peered":"2024-04-20T10:30:22.324038+0000","last_unstale":"2024-04-20T10:30:23.339394+0000","last_undegraded":"2024-04-20T10:30:23.339394+0000","last_fullsized":"2024-04-20T10:30:23.339394+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:14:26.388474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.480469+0000","last_change":"2024-04-20T10:30:21.299878+0000","last_active":"2024-04-20T10:30:23.480469+0000","last_peered":"2024-04-20T10:30:23.480469+0000","last_clean":"2024-04-20T10:30:23.480469+0000","last_became_active":"2024-04-20T10:30:21.299346+0000","last_became_peered":"2024-04-20T10:30:21.299346+0000","last_unstale":"2024-04-20T10:30:23.480469+0000","last_undegraded":"2024-04-20T10:30:23.480469+0000","last_fullsized":"2024-04-20T10:30:23.480469+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T19:32:15.353709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304212+0000","last_change":"2024-04-20T10:30:22.319692+0000","last_active":"2024-04-20T10:30:23.304212+0000","last_peered":"2024-04-20T10:30:23.304212+0000","last_clean":"2024-04-20T10:30:23.304212+0000","last_became_active":"2024-04-20T10:30:22.319379+0000","last_became_peered":"2024-04-20T10:30:22.319379+0000","last_unstale":"2024-04-20T10:30:23.304212+0000","last_undegraded":"2024-04-20T10:30:23.304212+0000","last_fullsized":"2024-04-20T10:30:23.304212+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T17:18:27.294334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.304094+0000","last_change":"2024-04-20T10:30:21.299031+0000","last_active":"2024-04-20T10:30:23.304094+0000","last_peered":"2024-04-20T10:30:23.304094+0000","last_clean":"2024-04-20T10:30:23.304094+0000","last_became_active":"2024-04-20T10:30:21.298071+0000","last_became_peered":"2024-04-20T10:30:21.298071+0000","last_unstale":"2024-04-20T10:30:23.304094+0000","last_undegraded":"2024-04-20T10:30:23.304094+0000","last_fullsized":"2024-04-20T10:30:23.304094+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:25:35.344994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.339998+0000","last_change":"2024-04-20T10:30:22.330573+0000","last_active":"2024-04-20T10:30:23.339998+0000","last_peered":"2024-04-20T10:30:23.339998+0000","last_clean":"2024-04-20T10:30:23.339998+0000","last_became_active":"2024-04-20T10:30:22.330217+0000","last_became_peered":"2024-04-20T10:30:22.330217+0000","last_unstale":"2024-04-20T10:30:23.339998+0000","last_undegraded":"2024-04-20T10:30:23.339998+0000","last_fullsized":"2024-04-20T10:30:23.339998+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T18:22:58.877802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340110+0000","last_change":"2024-04-20T10:30:21.302806+0000","last_active":"2024-04-20T10:30:23.340110+0000","last_peered":"2024-04-20T10:30:23.340110+0000","last_clean":"2024-04-20T10:30:23.340110+0000","last_became_active":"2024-04-20T10:30:21.302383+0000","last_became_peered":"2024-04-20T10:30:21.302383+0000","last_unstale":"2024-04-20T10:30:23.340110+0000","last_undegraded":"2024-04-20T10:30:23.340110+0000","last_fullsized":"2024-04-20T10:30:23.340110+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T11:34:15.411531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299611+0000","last_change":"2024-04-20T10:30:22.322414+0000","last_active":"2024-04-20T10:30:23.299611+0000","last_peered":"2024-04-20T10:30:23.299611+0000","last_clean":"2024-04-20T10:30:23.299611+0000","last_became_active":"2024-04-20T10:30:22.322136+0000","last_became_peered":"2024-04-20T10:30:22.322136+0000","last_unstale":"2024-04-20T10:30:23.299611+0000","last_undegraded":"2024-04-20T10:30:23.299611+0000","last_fullsized":"2024-04-20T10:30:23.299611+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T12:30:00.604634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.303861+0000","last_change":"2024-04-20T10:30:21.298661+0000","last_active":"2024-04-20T10:30:23.303861+0000","last_peered":"2024-04-20T10:30:23.303861+0000","last_clean":"2024-04-20T10:30:23.303861+0000","last_became_active":"2024-04-20T10:30:21.298297+0000","last_became_peered":"2024-04-20T10:30:21.298297+0000","last_unstale":"2024-04-20T10:30:23.303861+0000","last_undegraded":"2024-04-20T10:30:23.303861+0000","last_fullsized":"2024-04-20T10:30:23.303861+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:59:51.803957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.340600+0000","last_change":"2024-04-20T10:30:22.329243+0000","last_active":"2024-04-20T10:30:23.340600+0000","last_peered":"2024-04-20T10:30:23.340600+0000","last_clean":"2024-04-20T10:30:23.340600+0000","last_became_active":"2024-04-20T10:30:22.327549+0000","last_became_peered":"2024-04-20T10:30:22.327549+0000","last_unstale":"2024-04-20T10:30:23.340600+0000","last_undegraded":"2024-04-20T10:30:23.340600+0000","last_fullsized":"2024-04-20T10:30:23.340600+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:42:25.740328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.299089+0000","last_change":"2024-04-20T10:30:21.294433+0000","last_active":"2024-04-20T10:30:23.299089+0000","last_peered":"2024-04-20T10:30:23.299089+0000","last_clean":"2024-04-20T10:30:23.299089+0000","last_became_active":"2024-04-20T10:30:21.294166+0000","last_became_peered":"2024-04-20T10:30:21.294166+0000","last_unstale":"2024-04-20T10:30:23.299089+0000","last_undegraded":"2024-04-20T10:30:23.299089+0000","last_fullsized":"2024-04-20T10:30:23.299089+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:22:18.491101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.482730+0000","last_change":"2024-04-20T10:30:22.334102+0000","last_active":"2024-04-20T10:30:23.482730+0000","last_peered":"2024-04-20T10:30:23.482730+0000","last_clean":"2024-04-20T10:30:23.482730+0000","last_became_active":"2024-04-20T10:30:22.333754+0000","last_became_peered":"2024-04-20T10:30:22.333754+0000","last_unstale":"2024-04-20T10:30:23.482730+0000","last_undegraded":"2024-04-20T10:30:23.482730+0000","last_fullsized":"2024-04-20T10:30:23.482730+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:03:00.274563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449069+0000","last_change":"2024-04-20T10:30:21.298073+0000","last_active":"2024-04-20T10:30:23.449069+0000","last_peered":"2024-04-20T10:30:23.449069+0000","last_clean":"2024-04-20T10:30:23.449069+0000","last_became_active":"2024-04-20T10:30:21.295751+0000","last_became_peered":"2024-04-20T10:30:21.295751+0000","last_unstale":"2024-04-20T10:30:23.449069+0000","last_undegraded":"2024-04-20T10:30:23.449069+0000","last_fullsized":"2024-04-20T10:30:23.449069+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T15:55:25.206126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.345800+0000","last_change":"2024-04-20T10:30:22.326587+0000","last_active":"2024-04-20T10:30:23.345800+0000","last_peered":"2024-04-20T10:30:23.345800+0000","last_clean":"2024-04-20T10:30:23.345800+0000","last_became_active":"2024-04-20T10:30:22.326233+0000","last_became_peered":"2024-04-20T10:30:22.326233+0000","last_unstale":"2024-04-20T10:30:23.345800+0000","last_undegraded":"2024-04-20T10:30:23.345800+0000","last_fullsized":"2024-04-20T10:30:23.345800+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:21.254575+0000","last_clean_scrub_stamp":"2024-04-20T10:30:21.254575+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T20:37:03.855614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.344638+0000","last_change":"2024-04-20T10:30:21.297078+0000","last_active":"2024-04-20T10:30:23.344638+0000","last_peered":"2024-04-20T10:30:23.344638+0000","last_clean":"2024-04-20T10:30:23.344638+0000","last_became_active":"2024-04-20T10:30:21.296174+0000","last_became_peered":"2024-04-20T10:30:21.296174+0000","last_unstale":"2024-04-20T10:30:23.344638+0000","last_undegraded":"2024-04-20T10:30:23.344638+0000","last_fullsized":"2024-04-20T10:30:23.344638+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:30:20.240378+0000","last_clean_scrub_stamp":"2024-04-20T10:30:20.240378+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T21:09:59.254736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":230,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-20T10:30:23.449119+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:30:23.449119+0000","last_peered":"2024-04-20T10:30:23.449119+0000","last_clean":"2024-04-20T10:30:23.449119+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:30:23.449119+0000","last_undegraded":"2024-04-20T10:30:23.449119+0000","last_fullsized":"2024-04-20T10:30:23.449119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509480,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27656,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344184,"statfs":{"total":96636764160,"available":96608444416,"internally_reserved":0,"allocated":643072,"data_stored":279733,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509480,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27648,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344192,"statfs":{"total":96636764160,"available":96608452608,"internally_reserved":0,"allocated":638976,"data_stored":280259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509480,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28236,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343604,"statfs":{"total":96636764160,"available":96607850496,"internally_reserved":0,"allocated":1245184,"data_stored":871747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509480,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":279767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509480,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":279711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509480,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32384,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339456,"statfs":{"total":96636764160,"available":96603602944,"internally_reserved":0,"allocated":1232896,"data_stored":870115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509480,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31812,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340028,"statfs":{"total":96636764160,"available":96604188672,"internally_reserved":0,"allocated":638976,"data_stored":280259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542184,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":655360,"data_stored":281381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:30:27.281 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-20T10:30:27.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-20T10:30:27.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.540+0000 7f8163e00640 1 Processor -- start 2024-04-20T10:30:27.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.540+0000 7f8163e00640 1 -- start start 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.541+0000 7f8163e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 0x7f81640d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.541+0000 7f8163e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f81640b3330 0x7f81640b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.541+0000 7f8163e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f816406c190 con 0x7f816406c6d0 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.541+0000 7f8163e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8164162130 con 0x7f81640b3330 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.541+0000 7f8163e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f816406c470 con 0x7f8164161d50 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.542+0000 7f8162000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f81640b3330 0x7f81640b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.542+0000 7f8162000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f81640b3330 0x7f81640b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41192/0 (socket says 172.21.15.47:41192) 2024-04-20T10:30:27.542 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.542+0000 7f8162000640 1 -- 172.21.15.47:0/2116458148 learned_addr learned my addr 172.21.15.47:0/2116458148 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.542+0000 7f8162000640 1 -- 172.21.15.47:0/2116458148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 msgr2=0x7f81640d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.542+0000 7f8162000640 1 --2- 172.21.15.47:0/2116458148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 0x7f81640d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.542+0000 7f8162000640 1 -- 172.21.15.47:0/2116458148 >> v1:172.21.15.175:6789/0 conn(0x7f816406c6d0 legacy=0x7f81640cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.542+0000 7f8162000640 1 -- 172.21.15.47:0/2116458148 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81640d2e00 con 0x7f81640b3330 2024-04-20T10:30:27.543 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.544+0000 7f8162000640 1 --2- 172.21.15.47:0/2116458148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81640b3330 0x7f81640b3880 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8150009870 tx=0x7f815002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8f5e9d42dc0d460 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.544+0000 7f8161600640 1 -- 172.21.15.47:0/2116458148 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150035020 con 0x7f81640b3330 2024-04-20T10:30:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.544+0000 7f8161600640 1 -- 172.21.15.47:0/2116458148 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8150031640 con 0x7f81640b3330 2024-04-20T10:30:27.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.544+0000 7f8161600640 1 -- 172.21.15.47:0/2116458148 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f815002d040 con 0x7f81640b3330 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.544+0000 7f8163e00640 1 -- 172.21.15.47:0/2116458148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81640b3330 msgr2=0x7f81640b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.544+0000 7f8163e00640 1 --2- 172.21.15.47:0/2116458148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81640b3330 0x7f81640b3880 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8150009870 tx=0x7f815002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.545+0000 7f8163e00640 1 -- 172.21.15.47:0/2116458148 shutdown_connections 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.545+0000 7f8163e00640 1 --2- 172.21.15.47:0/2116458148 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f81640b3330 0x7f81640b3880 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.545+0000 7f8163e00640 1 --2- 172.21.15.47:0/2116458148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 0x7f81640d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.545+0000 7f8163e00640 1 -- 172.21.15.47:0/2116458148 >> 172.21.15.47:0/2116458148 conn(0x7f81640d4450 msgr2=0x7f81640d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.545+0000 7f8163e00640 1 -- 172.21.15.47:0/2116458148 shutdown_connections 2024-04-20T10:30:27.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.545+0000 7f8163e00640 1 -- 172.21.15.47:0/2116458148 wait complete. 2024-04-20T10:30:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.546+0000 7f8163e00640 1 Processor -- start 2024-04-20T10:30:27.546 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.546+0000 7f8163e00640 1 -- start start 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.547+0000 7f8163e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f816406c6d0 0x7f816415bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.547+0000 7f8163e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81640b3330 0x7f81641e8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.547+0000 7f8163e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 0x7f81640b20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.547+0000 7f8163e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f816406cb60 con 0x7f816406c6d0 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.547+0000 7f8163e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81640d33c0 con 0x7f81640b3330 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.547+0000 7f8163e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f816406cce0 con 0x7f8164161d50 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f816406c6d0 0x7f816415bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f816406c6d0 0x7f816415bb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41194/0 (socket says 172.21.15.47:41194) 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 -- 172.21.15.47:0/3216422604 learned_addr learned my addr 172.21.15.47:0/3216422604 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:27.548 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 -- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 msgr2=0x7f81640b20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 0x7f81640b20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 -- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81640b3330 msgr2=0x7f81641e8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81640b3330 0x7f81641e8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 -- 172.21.15.47:0/3216422604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81641e6fd0 con 0x7f816406c6d0 2024-04-20T10:30:27.549 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.548+0000 7f8162a00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f816406c6d0 0x7f816415bb80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f815400ba50 tx=0x7f815400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.549+0000 7f815aa00640 1 -- 172.21.15.47:0/3216422604 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154019070 con 0x7f816406c6d0 2024-04-20T10:30:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.549+0000 7f815aa00640 1 -- 172.21.15.47:0/3216422604 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f815400cc00 con 0x7f816406c6d0 2024-04-20T10:30:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.549+0000 7f815aa00640 1 -- 172.21.15.47:0/3216422604 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8154014440 con 0x7f816406c6d0 2024-04-20T10:30:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.549+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81641e7270 con 0x7f816406c6d0 2024-04-20T10:30:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.549+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81641e7830 con 0x7f816406c6d0 2024-04-20T10:30:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.550+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8164075b60 con 0x7f816406c6d0 2024-04-20T10:30:27.550 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.551+0000 7f815aa00640 1 -- 172.21.15.47:0/3216422604 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f815401c070 con 0x7f816406c6d0 2024-04-20T10:30:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.552+0000 7f815aa00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f814406c920 0x7f814406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.552+0000 7f815aa00640 1 -- 172.21.15.47:0/3216422604 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f815408bad0 con 0x7f816406c6d0 2024-04-20T10:30:27.552 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.552+0000 7f8162000640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f814406c920 0x7f814406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:27.553 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.554+0000 7f8162000640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f814406c920 0x7f814406ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8150036000 tx=0x7f8150030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:27.556 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.556+0000 7f815aa00640 1 -- 172.21.15.47:0/3216422604 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8154059cd0 con 0x7f816406c6d0 2024-04-20T10:30:27.838 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.838+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f81640cb2e0 con 0x7f816406c6d0 2024-04-20T10:30:27.842 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.842+0000 7f815aa00640 1 -- 172.21.15.47:0/3216422604 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7f81540595d0 con 0x7f816406c6d0 2024-04-20T10:30:27.842 INFO:teuthology.orchestra.run.smithi047.stdout:[client.0] 2024-04-20T10:30:27.843 INFO:teuthology.orchestra.run.smithi047.stdout: key = AQBXmCNmrHlNFRAAY90gnx6m90huEzUUDh9rtw== 2024-04-20T10:30:27.843 INFO:teuthology.orchestra.run.smithi047.stdout: caps mds = "allow" 2024-04-20T10:30:27.843 INFO:teuthology.orchestra.run.smithi047.stdout: caps mon = "allow r" 2024-04-20T10:30:27.843 INFO:teuthology.orchestra.run.smithi047.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-20T10:30:27.843 INFO:teuthology.orchestra.run.smithi047.stderr:updated caps for client.0 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.845+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f814406c920 msgr2=0x7f814406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.845+0000 7f8163e00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f814406c920 0x7f814406ede0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8150036000 tx=0x7f8150030000 comp rx=0 tx=0).stop 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.845+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f816406c6d0 msgr2=0x7f816415bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.845+0000 7f8163e00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f816406c6d0 0x7f816415bb80 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f815400ba50 tx=0x7f815400bf10 comp rx=0 tx=0).stop 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 shutdown_connections 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8164161d50 0x7f81640b20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81640b3330 0x7f81641e8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f814406c920 0x7f814406ede0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 --2- 172.21.15.47:0/3216422604 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f816406c6d0 0x7f816415bb80 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:27.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 >> 172.21.15.47:0/3216422604 conn(0x7f81640d4450 msgr2=0x7f81640b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:27.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 shutdown_connections 2024-04-20T10:30:27.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:27.846+0000 7f8163e00640 1 -- 172.21.15.47:0/3216422604 wait complete. 2024-04-20T10:30:27.863 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.078+0000 7f6e16400640 1 Processor -- start 2024-04-20T10:30:28.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.078+0000 7f6e16400640 1 -- start start 2024-04-20T10:30:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.079+0000 7f6e16400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6e10156c20 0x7f6e1006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.079+0000 7f6e16400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e10141c60 con 0x7f6e1015ca40 2024-04-20T10:30:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.079+0000 7f6e16400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e100adab0 con 0x7f6e100d0360 2024-04-20T10:30:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.079+0000 7f6e16400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6e1015cf90 con 0x7f6e10156c20 2024-04-20T10:30:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.080+0000 7f6e0ea00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6e1015ca40 0x7f6e100ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:53474/0 (socket says 172.21.15.47:53474) 2024-04-20T10:30:28.080 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.080+0000 7f6e0ea00640 1 -- 172.21.15.47:0/2890234715 learned_addr learned my addr 172.21.15.47:0/2890234715 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:28.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.080+0000 7f6e0f400640 1 --2- 172.21.15.47:0/2890234715 >> v2:172.21.15.175:3300/0 conn(0x7f6e10156c20 0x7f6e1006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.080+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 716012532 0 0) 0x7f6e10141c60 con 0x7f6e1015ca40 2024-04-20T10:30:28.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.080+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6df4003520 con 0x7f6e1015ca40 2024-04-20T10:30:28.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 65474966 0 0) 0x7f6df4003520 con 0x7f6e1015ca40 2024-04-20T10:30:28.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 >> v2:172.21.15.175:3300/0 conn(0x7f6e10156c20 msgr2=0x7f6e1006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 --2- 172.21.15.47:0/2890234715 >> v2:172.21.15.175:3300/0 conn(0x7f6e10156c20 0x7f6e1006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 >> v1:172.21.15.175:6789/0 conn(0x7f6e100d0360 legacy=0x7f6e10142140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e1015d270 con 0x7f6e1015ca40 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f6df8001670 con 0x7f6e1015ca40 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 >> v1:172.21.15.47:6789/0 conn(0x7f6e1015ca40 legacy=0x7f6e100ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6df4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4004800 0x7f6df4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40051c0 0x7f6df400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1015cf90 con 0x7f6df40039b0 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6df400c060 con 0x7f6df40051c0 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6df400c260 con 0x7f6df4004800 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.081+0000 7f6e0f400640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6df4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0f400640 1 -- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4004800 msgr2=0x7f6df4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0ea00640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4004800 0x7f6df4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0fe00640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40051c0 0x7f6df400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0f400640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6df4004800 0x7f6df4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0f400640 1 -- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40051c0 msgr2=0x7f6df400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0f400640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6df40051c0 0x7f6df400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0f400640 1 -- 172.21.15.47:0/2890234715 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6df4003520 con 0x7f6df40039b0 2024-04-20T10:30:28.084 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.082+0000 7f6e0fe00640 1 -- 172.21.15.47:0/2890234715 reap_dead start 2024-04-20T10:30:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.083+0000 7f6e0f400640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6df4003dd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009ac0 tx=0x7f6df4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934df21f77707e3f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.083+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00039070 con 0x7f6df40039b0 2024-04-20T10:30:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.083+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0002c6b0 con 0x7f6df40039b0 2024-04-20T10:30:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.083+0000 7f6e0e000640 1 -- 172.21.15.47:0/2890234715 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00009690 con 0x7f6df40039b0 2024-04-20T10:30:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.084+0000 7f6e16400640 1 -- 172.21.15.47:0/2890234715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 msgr2=0x7f6df4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.084+0000 7f6e16400640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6df4003dd0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009ac0 tx=0x7f6df4005830 comp rx=0 tx=0).stop 2024-04-20T10:30:28.085 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.084+0000 7f6e16400640 1 -- 172.21.15.47:0/2890234715 shutdown_connections 2024-04-20T10:30:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.084+0000 7f6e16400640 1 --2- 172.21.15.47:0/2890234715 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6df4003dd0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.084+0000 7f6e16400640 1 -- 172.21.15.47:0/2890234715 >> 172.21.15.47:0/2890234715 conn(0x7f6e100b3350 msgr2=0x7f6e100b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.084+0000 7f6e16400640 1 -- 172.21.15.47:0/2890234715 shutdown_connections 2024-04-20T10:30:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.084+0000 7f6e16400640 1 -- 172.21.15.47:0/2890234715 wait complete. 2024-04-20T10:30:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.086+0000 7f6e16400640 1 Processor -- start 2024-04-20T10:30:28.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.086+0000 7f6e16400640 1 -- start start 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e16400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6e10151f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e16400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100ccb30 0x7f6e100ccfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e16400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e10072ba0 0x7f6e10073020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e16400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6e100d07d0 con 0x7f6df40039b0 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e16400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e1015d830 con 0x7f6e10072ba0 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e16400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e100d0950 con 0x7f6e100ccb30 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e0f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6e10151f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.088 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e0f400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6e10151f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41216/0 (socket says 172.21.15.47:41216) 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.087+0000 7f6e0f400640 1 -- 172.21.15.47:0/1150266094 learned_addr learned my addr 172.21.15.47:0/1150266094 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0f400640 1 -- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100ccb30 msgr2=0x7f6e100ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0ea00640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100ccb30 0x7f6e100ccfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0fe00640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e10072ba0 0x7f6e10073020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0f400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100ccb30 0x7f6e100ccfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0f400640 1 -- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e10072ba0 msgr2=0x7f6e10073020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0f400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e10072ba0 0x7f6e10073020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0f400640 1 -- 172.21.15.47:0/1150266094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e100cd5f0 con 0x7f6df40039b0 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0fe00640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e10072ba0 0x7f6e10073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:28.089 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e0f400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6e10151f80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009be0 tx=0x7f6e00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.088+0000 7f6e07400640 1 -- 172.21.15.47:0/1150266094 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00042020 con 0x7f6df40039b0 2024-04-20T10:30:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.089+0000 7f6e07400640 1 -- 172.21.15.47:0/1150266094 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e0003a040 con 0x7f6df40039b0 2024-04-20T10:30:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.089+0000 7f6e07400640 1 -- 172.21.15.47:0/1150266094 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e00039070 con 0x7f6df40039b0 2024-04-20T10:30:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.089+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e10152dd0 con 0x7f6df40039b0 2024-04-20T10:30:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.089+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e100cd8d0 con 0x7f6df40039b0 2024-04-20T10:30:28.090 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.090+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e10074010 con 0x7f6df40039b0 2024-04-20T10:30:28.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.091+0000 7f6e07400640 1 -- 172.21.15.47:0/1150266094 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6e00045020 con 0x7f6df40039b0 2024-04-20T10:30:28.091 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.092+0000 7f6e07400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6dec06c8f0 0x7f6dec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.092+0000 7f6e07400640 1 -- 172.21.15.47:0/1150266094 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f6e000ad150 con 0x7f6df40039b0 2024-04-20T10:30:28.092 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.092+0000 7f6e0ea00640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6dec06c8f0 0x7f6dec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.094+0000 7f6e0ea00640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6dec06c8f0 0x7f6dec06edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6df8001200 tx=0x7f6df8009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:28.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.097+0000 7f6e07400640 1 -- 172.21.15.47:0/1150266094 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6e0004c070 con 0x7f6df40039b0 2024-04-20T10:30:28.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.340+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e10068980 con 0x7f6df40039b0 2024-04-20T10:30:28.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.341+0000 7f6e07400640 1 -- 172.21.15.47:0/1150266094 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5202 (secure 0 0 0) 0x7f6e0007b350 con 0x7f6df40039b0 2024-04-20T10:30:28.341 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:28.341 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3182036779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3182036779},{"type":"v1","addr":"172.21.15.175:6835","nonce":3182036779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5398,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1207907337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1207907337},{"type":"v1","addr":"172.21.15.175:6837","nonce":1207907337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/793057653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":793057653},{"type":"v1","addr":"172.21.15.47:6833","nonce":793057653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:24.860182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6057":{"gid":6057,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3814505199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3814505199},{"type":"v1","addr":"172.21.15.47:6835","nonce":3814505199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6057,"qdb_cluster":[6057]},"id":3}]} 2024-04-20T10:30:28.342 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6dec06c8f0 msgr2=0x7f6dec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6dec06c8f0 0x7f6dec06edb0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f6df8001200 tx=0x7f6df8009040 comp rx=0 tx=0).stop 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 msgr2=0x7f6e10151f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6e10151f80 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f6e00009be0 tx=0x7f6e00004730 comp rx=0 tx=0).stop 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 shutdown_connections 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e10072ba0 0x7f6e10073020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e100ccb30 0x7f6e100ccfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6dec06c8f0 0x7f6dec06edb0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 --2- 172.21.15.47:0/1150266094 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6df40039b0 0x7f6e10151f80 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 >> 172.21.15.47:0/1150266094 conn(0x7f6e100b3350 msgr2=0x7f6e1015db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 shutdown_connections 2024-04-20T10:30:28.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.344+0000 7f6e16400640 1 -- 172.21.15.47:0/1150266094 wait complete. 2024-04-20T10:30:28.739 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:28.748 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:28.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 32, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:30:21.745506+0000', 'modified': '2024-04-20T10:30:24.860182+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6057}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6057': {'gid': 6057, 'name': 'a', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.47:6835/3814505199', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6834', 'nonce': 3814505199}, {'type': 'v1', 'addr': '172.21.15.47:6835', 'nonce': 3814505199}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6057, 'qdb_cluster': [6057]} 2024-04-20T10:30:28.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:30:28.766 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T10:30:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.977+0000 7fedaa800640 1 Processor -- start 2024-04-20T10:30:28.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.977+0000 7fedaa800640 1 -- start start 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.978+0000 7fedaa800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7feda406da90 0x7feda406def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.978+0000 7fedaa800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feda406c960 0x7feda406cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.978+0000 7fedaa800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7feda406b900 con 0x7feda415a7b0 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.978+0000 7fedaa800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7feda40b50a0 con 0x7feda406c960 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.978+0000 7fedaa800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7feda40ae670 con 0x7feda406da90 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.979+0000 7feda9e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feda406c960 0x7feda406cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.979+0000 7feda9e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7feda406c960 0x7feda406cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41222/0 (socket says 172.21.15.47:41222) 2024-04-20T10:30:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.979+0000 7feda9e00640 1 -- 172.21.15.47:0/778261345 learned_addr learned my addr 172.21.15.47:0/778261345 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.979+0000 7feda8a00640 1 --2- 172.21.15.47:0/778261345 >> v2:172.21.15.175:3300/0 conn(0x7feda406da90 0x7feda406def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.979+0000 7feda9e00640 1 -- 172.21.15.47:0/778261345 >> v2:172.21.15.175:3300/0 conn(0x7feda406da90 msgr2=0x7feda406def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.979+0000 7feda9e00640 1 --2- 172.21.15.47:0/778261345 >> v2:172.21.15.175:3300/0 conn(0x7feda406da90 0x7feda406def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.980+0000 7feda9e00640 1 -- 172.21.15.47:0/778261345 >> v1:172.21.15.175:6789/0 conn(0x7feda415a7b0 legacy=0x7feda40adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.980+0000 7feda9e00640 1 -- 172.21.15.47:0/778261345 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda406bba0 con 0x7feda406c960 2024-04-20T10:30:28.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.981+0000 7feda9e00640 1 --2- 172.21.15.47:0/778261345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406c960 0x7feda406cda0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7feda4075420 tx=0x7fed9c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6812d4edbb3dfe server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:28.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.981+0000 7feda3e00640 1 -- 172.21.15.47:0/778261345 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c02ca80 con 0x7feda406c960 2024-04-20T10:30:28.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.981+0000 7feda3e00640 1 -- 172.21.15.47:0/778261345 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c02cc20 con 0x7feda406c960 2024-04-20T10:30:28.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.981+0000 7feda3e00640 1 -- 172.21.15.47:0/778261345 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c0343f0 con 0x7feda406c960 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 -- 172.21.15.47:0/778261345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406c960 msgr2=0x7feda406cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 --2- 172.21.15.47:0/778261345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406c960 0x7feda406cda0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7feda4075420 tx=0x7fed9c02bee0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 -- 172.21.15.47:0/778261345 shutdown_connections 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 --2- 172.21.15.47:0/778261345 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406c960 0x7feda406cda0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 --2- 172.21.15.47:0/778261345 >> v2:172.21.15.175:3300/0 conn(0x7feda406da90 0x7feda406def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 -- 172.21.15.47:0/778261345 >> 172.21.15.47:0/778261345 conn(0x7feda40d0040 msgr2=0x7feda40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 -- 172.21.15.47:0/778261345 shutdown_connections 2024-04-20T10:30:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.982+0000 7fedaa800640 1 -- 172.21.15.47:0/778261345 wait complete. 2024-04-20T10:30:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.984+0000 7fedaa800640 1 Processor -- start 2024-04-20T10:30:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.984+0000 7fedaa800640 1 -- start start 2024-04-20T10:30:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7fedaa800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feda406c960 0x7feda40b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7fedaa800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406da90 0x7feda40af930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7fedaa800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feda415a7b0 0x7feda41e65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7fedaa800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feda406c440 con 0x7feda406da90 2024-04-20T10:30:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7fedaa800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feda406c160 con 0x7feda415a7b0 2024-04-20T10:30:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7fedaa800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feda415ab90 con 0x7feda406c960 2024-04-20T10:30:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406da90 0x7feda40af930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406da90 0x7feda40af930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41230/0 (socket says 172.21.15.47:41230) 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda8a00640 1 -- 172.21.15.47:0/1684940776 learned_addr learned my addr 172.21.15.47:0/1684940776 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda9400640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feda406c960 0x7feda40b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda8a00640 1 -- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feda406c960 msgr2=0x7feda40b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda8a00640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feda406c960 0x7feda40b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda8a00640 1 -- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feda415a7b0 msgr2=0x7feda41e65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda9e00640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feda415a7b0 0x7feda41e65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.985+0000 7feda8a00640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feda415a7b0 0x7feda41e65f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7feda8a00640 1 -- 172.21.15.47:0/1684940776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feda40b0160 con 0x7feda406da90 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7feda9400640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feda406c960 0x7feda40b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7feda8a00640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406da90 0x7feda40af930 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fed9000d830 tx=0x7fed9000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7feda1600640 1 -- 172.21.15.47:0/1684940776 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9000b6c0 con 0x7feda406da90 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7feda1600640 1 -- 172.21.15.47:0/1684940776 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed90019070 con 0x7feda406da90 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7feda1600640 1 -- 172.21.15.47:0/1684940776 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9000ba80 con 0x7feda406da90 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feda40b0440 con 0x7feda406da90 2024-04-20T10:30:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.986+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feda40b0a00 con 0x7feda406da90 2024-04-20T10:30:28.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.988+0000 7feda1600640 1 -- 172.21.15.47:0/1684940776 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fed9001e460 con 0x7feda406da90 2024-04-20T10:30:28.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.990+0000 7feda1600640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8406c8f0 0x7fed8406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.990+0000 7feda1600640 1 -- 172.21.15.47:0/1684940776 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fed9008c460 con 0x7feda406da90 2024-04-20T10:30:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.991+0000 7feda9400640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8406c8f0 0x7fed8406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.991+0000 7fedaec11740 1 Processor -- start 2024-04-20T10:30:28.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.991+0000 7fedaec11740 1 -- start start 2024-04-20T10:30:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7feda9400640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8406c8f0 0x7fed8406edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed9400a830 tx=0x7fed94008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 0x5557ce123ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce1241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7fedaec11740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce11eb90 0x5557ce14e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7fedaec11740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5557ce08a9d0 con 0x5557ce11eb90 2024-04-20T10:30:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5557ce031270 con 0x5557ce1226f0 2024-04-20T10:30:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5557ce086990 con 0x5557ce16c670 2024-04-20T10:30:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7feda9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce11eb90 0x5557ce14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7feda9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce11eb90 0x5557ce14e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41238/0 (socket says 172.21.15.47:41238) 2024-04-20T10:30:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.992+0000 7feda9e00640 1 -- 172.21.15.47:0/4274948842 learned_addr learned my addr 172.21.15.47:0/4274948842 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda8a00640 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce1241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda9400640 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 0x5557ce123ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda9e00640 1 -- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 msgr2=0x5557ce123ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda9e00640 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 0x5557ce123ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda9e00640 1 -- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 msgr2=0x5557ce1241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda9e00640 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce1241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda9e00640 1 -- 172.21.15.47:0/4274948842 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557cdd9d2d0 con 0x5557ce11eb90 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda8a00640 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce1241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7feda9e00640 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce11eb90 0x5557ce14e980 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009f40 tx=0x7fed9c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4297c8f4ed1f4f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7fed99c00640 1 -- 172.21.15.47:0/4274948842 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c045070 con 0x5557ce11eb90 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7fed99c00640 1 -- 172.21.15.47:0/4274948842 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c02ca80 con 0x5557ce11eb90 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.993+0000 7fed99c00640 1 -- 172.21.15.47:0/4274948842 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c009510 con 0x5557ce11eb90 2024-04-20T10:30:28.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 -- 172.21.15.47:0/4274948842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce11eb90 msgr2=0x5557ce14e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce11eb90 0x5557ce14e980 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fed9c009f40 tx=0x7fed9c02e000 comp rx=0 tx=0).stop 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 -- 172.21.15.47:0/4274948842 shutdown_connections 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce11eb90 0x5557ce14e980 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce1241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 --2- 172.21.15.47:0/4274948842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 0x5557ce123ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 -- 172.21.15.47:0/4274948842 >> 172.21.15.47:0/4274948842 conn(0x5557ce120e00 msgr2=0x5557ce169980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 -- 172.21.15.47:0/4274948842 shutdown_connections 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.994+0000 7fedaec11740 1 -- 172.21.15.47:0/4274948842 wait complete. 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.995+0000 7fedaec11740 1 -- start start 2024-04-20T10:30:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.995+0000 7fedaec11740 1 Processor -- start 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce138fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 0x5557ce170710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7fedaec11740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce14d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7fedaec11740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5557ce122c00 con 0x5557ce14c860 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5557ce0445e0 con 0x5557ce1226f0 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5557ce122ee0 con 0x5557ce16c670 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7fedaec11740 10 client.0 _ll_register_callbacks cb 0x5557cdd9cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7feda9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce14d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7feda9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce14d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41244/0 (socket says 172.21.15.47:41244) 2024-04-20T10:30:28.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.996+0000 7feda9e00640 1 -- 172.21.15.47:0/3945908728 learned_addr learned my addr 172.21.15.47:0/3945908728 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7feda9e00640 1 -- 172.21.15.47:0/3945908728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 msgr2=0x5557ce170710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7feda9e00640 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 0x5557ce170710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7feda9e00640 1 -- 172.21.15.47:0/3945908728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 msgr2=0x5557ce138fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7feda9e00640 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce138fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7feda9e00640 1 -- 172.21.15.47:0/3945908728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557cde676b0 con 0x5557ce14c860 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7feda9e00640 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce14d530 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fed9c043040 tx=0x7fed9c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9806e8e2fb2de21a server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7fed8e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7fed8e000640 1 -- 172.21.15.47:0/3945908728 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c045070 con 0x5557ce14c860 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7fed8e000640 1 -- 172.21.15.47:0/3945908728 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fed9c04cc80 con 0x5557ce14c860 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9c02ca80 con 0x5557ce14c860 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.997+0000 7fed8e000640 1 -- 172.21.15.47:0/3945908728 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed9c02cc30 con 0x5557ce14c860 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 10 client.6201 fetch_fsmap learned FSMap version 32 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5557ce16f150 con 0x5557ce14c860 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fed8e000640 1 -- 172.21.15.47:0/3945908728 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9c02ca80 con 0x5557ce14c860 2024-04-20T10:30:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fed8e000640 1 -- 172.21.15.47:0/3945908728 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7fed9c008040 con 0x5557ce14c860 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 10 client.6201 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 10 client.6201 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 20 client.6201 populate_metadata read hostname 'smithi047' 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce1714e0 0x5557ce1718c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 4 client.6201 mds_command: new command op to 6057 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 --> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557ce0983f0 con 0x5557ce1714e0 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.998+0000 7feda9400640 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce1714e0 0x5557ce1718c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.999+0000 7feda9400640 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce1714e0 0x5557ce1718c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d6a1fefcf0671cb server_cookie=57bfc301fe39c52f in_seq=0 out_seq=0 2024-04-20T10:30:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:28.999+0000 7fed8e000640 10 client.6201 ms_handle_connect on v2:172.21.15.47:6834/3814505199 2024-04-20T10:30:29.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fed8e000640 1 -- 172.21.15.47:0/3945908728 <== mds.0 v2:172.21.15.47:6834/3814505199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557ce0983f0 con 0x5557ce1714e0 2024-04-20T10:30:29.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fed8e000640 10 client.6201 handle_command_reply: tid=0 2024-04-20T10:30:29.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 client.6201 shutdown 2024-04-20T10:30:29.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 msgr2=0x5557ce14d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.002 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce14d530 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fed9c043040 tx=0x7fed9c004ab0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 shutdown_connections 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce1714e0 0x5557ce1718c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce14d530 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce16c670 0x5557ce170710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 --2- 172.21.15.47:0/3945908728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce1226f0 0x5557ce138fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 >> 172.21.15.47:0/3945908728 conn(0x5557ce11eb90 msgr2=0x5557ce16cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fed8e000640 0 client.6201 ms_handle_reset on v2:172.21.15.47:6834/3814505199 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 shutdown_connections 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 1 -- 172.21.15.47:0/3945908728 wait complete. 2024-04-20T10:30:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.000+0000 7fedaec11740 20 client.6201 trim_cache size 0 max 16384 2024-04-20T10:30:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.020+0000 7fedaec11740 1 Processor -- start 2024-04-20T10:30:29.020 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.020+0000 7fedaec11740 1 -- start start 2024-04-20T10:30:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce1249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7fedaec11740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce16c670 0x5557ce1455f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7fedaec11740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7feda4200020 con 0x5557ce14c860 2024-04-20T10:30:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feda41e5d00 con 0x5557ce16c670 2024-04-20T10:30:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feda41ffc10 con 0x5557ce1226f0 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7feda8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7feda8a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41246/0 (socket says 172.21.15.47:41246) 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.021+0000 7feda8a00640 1 -- 172.21.15.47:0/1086235375 learned_addr learned my addr 172.21.15.47:0/1086235375 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda9400640 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce1249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda9e00640 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce16c670 0x5557ce1455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda8a00640 1 -- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 msgr2=0x5557ce1249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda8a00640 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce1249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda8a00640 1 -- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce16c670 msgr2=0x5557ce1455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda8a00640 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce16c670 0x5557ce1455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda8a00640 1 -- 172.21.15.47:0/1086235375 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557ce11ab20 con 0x5557ce14c860 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda9e00640 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce16c670 0x5557ce1455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7feda8a00640 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce16df60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fed900563c0 tx=0x7fed90059bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b358015ae024ba75 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7fed99200640 1 -- 172.21.15.47:0/1086235375 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90072020 con 0x5557ce14c860 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7fed99200640 1 -- 172.21.15.47:0/1086235375 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9005e310 con 0x5557ce14c860 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.022+0000 7fed99200640 1 -- 172.21.15.47:0/1086235375 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed90062b40 con 0x5557ce14c860 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 -- 172.21.15.47:0/1086235375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 msgr2=0x5557ce16df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce16df60 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fed900563c0 tx=0x7fed90059bd0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 -- 172.21.15.47:0/1086235375 shutdown_connections 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce16c670 0x5557ce1455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 --2- 172.21.15.47:0/1086235375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce1249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 -- 172.21.15.47:0/1086235375 >> 172.21.15.47:0/1086235375 conn(0x5557ce11eb90 msgr2=0x5557ce122e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 -- 172.21.15.47:0/1086235375 shutdown_connections 2024-04-20T10:30:29.024 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.023+0000 7fedaec11740 1 -- 172.21.15.47:0/1086235375 wait complete. 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.024+0000 7fedaec11740 1 -- start start 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.024+0000 7fedaec11740 1 Processor -- start 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce16abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7fedaec11740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7fedaec11740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce16c670 0x5557ce16e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7fedaec11740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x5557ce17b680 con 0x5557ce16c670 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5557ce122c00 con 0x5557ce14c860 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7fedaec11740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5557ce17af30 con 0x5557ce1226f0 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7fedaec11740 10 client.0 _ll_register_callbacks cb 0x7fed9c043070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7feda9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce16c670 0x5557ce16e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7feda9e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce16c670 0x5557ce16e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41252/0 (socket says 172.21.15.47:41252) 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.025+0000 7feda9e00640 1 -- 172.21.15.47:0/3760055797 learned_addr learned my addr 172.21.15.47:0/3760055797 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda8a00640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9400640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce16abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9e00640 1 -- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 msgr2=0x5557ce16abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9e00640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce16abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9e00640 1 -- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce14c860 msgr2=0x5557ce16df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9e00640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9e00640 1 -- 172.21.15.47:0/3760055797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557ce16b8b0 con 0x5557ce16c670 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda8a00640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:29.030 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9400640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce16abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7feda9e00640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce16c670 0x5557ce16e4a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0042e0 tx=0x7fed9c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346ae43159ac03e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7fed8e000640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7fed8e000640 1 -- 172.21.15.47:0/3760055797 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed9c059020 con 0x5557ce16c670 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7fed8e000640 1 -- 172.21.15.47:0/3760055797 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fed9c040670 con 0x5557ce16c670 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed9005e310 con 0x5557ce16c670 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7fed8e000640 1 -- 172.21.15.47:0/3760055797 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fed9c040870 con 0x5557ce16c670 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7fedaec11740 10 client.6207 fetch_fsmap learned FSMap version 32 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.026+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=2+}) -- 0x5557ce11fd90 con 0x5557ce16c670 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fed8e000640 1 -- 172.21.15.47:0/3760055797 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed9005e310 con 0x5557ce16c670 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fed8e000640 1 -- 172.21.15.47:0/3760055797 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 32) ==== 3495+0+0 (secure 0 0 0) 0x7fed9c008040 con 0x5557ce16c670 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fedaec11740 10 client.6207 fetch_fsmap finished waiting for FSMap version 32 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fedaec11740 10 client.6207 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:30:29.031 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fedaec11740 20 client.6207 populate_metadata read hostname 'smithi047' 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fedaec11740 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce16d2b0 0x5557ce11db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fedaec11740 4 client.6207 mds_command: new command op to 6057 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 --> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] -- command(tid 0: {"prefix": "status"}) -- 0x5557ce092260 con 0x5557ce16d2b0 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.027+0000 7feda9400640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce16d2b0 0x5557ce11db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7feda9400640 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce16d2b0 0x5557ce11db60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98ff4ff1dfb85a6f server_cookie=a2bcbb499e6773dd in_seq=0 out_seq=0 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fed8e000640 10 client.6207 ms_handle_connect on v2:172.21.15.47:6834/3814505199 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fed8e000640 1 -- 172.21.15.47:0/3760055797 <== mds.0 v2:172.21.15.47:6834/3814505199 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x5557ce092260 con 0x5557ce16d2b0 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fed8e000640 10 client.6207 handle_command_reply: tid=0 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 client.6207 shutdown 2024-04-20T10:30:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce16c670 msgr2=0x5557ce16e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce16c670 0x5557ce16e4a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fed9c0042e0 tx=0x7fed9c030000 comp rx=0 tx=0).stop 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 shutdown_connections 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:6834/3814505199,v1:172.21.15.47:6835/3814505199] conn(0x5557ce16d2b0 0x5557ce11db60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x5557ce16c670 0x5557ce16e4a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5557ce14c860 0x5557ce16df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 --2- 172.21.15.47:0/3760055797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5557ce1226f0 0x5557ce16abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fed8e000640 0 client.6207 ms_handle_reset on v2:172.21.15.47:6834/3814505199 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 >> 172.21.15.47:0/3760055797 conn(0x5557ce11eb90 msgr2=0x5557ce1249a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 shutdown_connections 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 1 -- 172.21.15.47:0/3760055797 wait complete. 2024-04-20T10:30:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.028+0000 7fedaec11740 20 client.6207 trim_cache size 0 max 16384 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "775946e6-93c9-4843-aaf4-db758801eec1", 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6057, 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-20T10:30:29.034 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 6.7152233519999998, 2024-04-20T10:30:29.035 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 32, 2024-04-20T10:30:29.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 54, 2024-04-20T10:30:29.035 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 54, 2024-04-20T10:30:29.035 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 12.114523438999999 2024-04-20T10:30:29.035 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:30:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8406c8f0 msgr2=0x7fed8406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8406c8f0 0x7fed8406edb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fed9400a830 tx=0x7fed94008040 comp rx=0 tx=0).stop 2024-04-20T10:30:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406da90 msgr2=0x7feda40af930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406da90 0x7feda40af930 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fed9000d830 tx=0x7fed9000dcf0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 shutdown_connections 2024-04-20T10:30:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feda415a7b0 0x7feda41e65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fed8406c8f0 0x7fed8406edb0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7feda406da90 0x7feda40af930 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 --2- 172.21.15.47:0/1684940776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feda406c960 0x7feda40b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 >> 172.21.15.47:0/1684940776 conn(0x7feda40d0040 msgr2=0x7feda415f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 shutdown_connections 2024-04-20T10:30:29.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.029+0000 7fedaa800640 1 -- 172.21.15.47:0/1684940776 wait complete. 2024-04-20T10:30:29.045 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-20T10:30:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.247+0000 7f4a69c00640 1 Processor -- start 2024-04-20T10:30:29.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.247+0000 7f4a69c00640 1 -- start start 2024-04-20T10:30:29.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.248+0000 7f4a69c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a64159e10 0x7f4a64071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.248+0000 7f4a69c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4a6415a1f0 0x7f4a64071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.248+0000 7f4a69c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6406da90 0x7f4a64160030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.248+0000 7f4a69c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4a64072900 con 0x7f4a64159e10 2024-04-20T10:30:29.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.248+0000 7f4a69c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4a6415ca40 con 0x7f4a6415a1f0 2024-04-20T10:30:29.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.248+0000 7f4a69c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6415cc90 con 0x7f4a6406da90 2024-04-20T10:30:29.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a64159e10 0x7f4a64071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4a64159e10 0x7f4a64071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41262/0 (socket says 172.21.15.47:41262) 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 -- 172.21.15.47:0/2576054585 learned_addr learned my addr 172.21.15.47:0/2576054585 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63400640 1 --2- 172.21.15.47:0/2576054585 >> v2:172.21.15.175:3300/0 conn(0x7f4a6415a1f0 0x7f4a64071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a69200640 1 --2- 172.21.15.47:0/2576054585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6406da90 0x7f4a64160030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 -- 172.21.15.47:0/2576054585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6406da90 msgr2=0x7f4a64160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 --2- 172.21.15.47:0/2576054585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6406da90 0x7f4a64160030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 -- 172.21.15.47:0/2576054585 >> v2:172.21.15.175:3300/0 conn(0x7f4a6415a1f0 msgr2=0x7f4a64071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 --2- 172.21.15.47:0/2576054585 >> v2:172.21.15.175:3300/0 conn(0x7f4a6415a1f0 0x7f4a64071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a63e00640 1 -- 172.21.15.47:0/2576054585 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6415cf70 con 0x7f4a64159e10 2024-04-20T10:30:29.250 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.249+0000 7f4a69200640 1 --2- 172.21.15.47:0/2576054585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6406da90 0x7f4a64160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.250+0000 7f4a63e00640 1 --2- 172.21.15.47:0/2576054585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a64159e10 0x7f4a64071a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4a64050400 tx=0x7f4a5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867df3bd54248121 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a62a00640 1 -- 172.21.15.47:0/2576054585 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54035020 con 0x7f4a64159e10 2024-04-20T10:30:29.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a62a00640 1 -- 172.21.15.47:0/2576054585 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a54031870 con 0x7f4a64159e10 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a62a00640 1 -- 172.21.15.47:0/2576054585 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54031c40 con 0x7f4a64159e10 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 -- 172.21.15.47:0/2576054585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a64159e10 msgr2=0x7f4a64071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2576054585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a64159e10 0x7f4a64071a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f4a64050400 tx=0x7f4a5402b910 comp rx=0 tx=0).stop 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 -- 172.21.15.47:0/2576054585 shutdown_connections 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2576054585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6406da90 0x7f4a64160030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2576054585 >> v2:172.21.15.175:3300/0 conn(0x7f4a6415a1f0 0x7f4a64071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2576054585 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a64159e10 0x7f4a64071a80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 -- 172.21.15.47:0/2576054585 >> 172.21.15.47:0/2576054585 conn(0x7f4a640d0040 msgr2=0x7f4a640d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 -- 172.21.15.47:0/2576054585 shutdown_connections 2024-04-20T10:30:29.252 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.251+0000 7f4a69c00640 1 -- 172.21.15.47:0/2576054585 wait complete. 2024-04-20T10:30:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.253+0000 7f4a69c00640 1 Processor -- start 2024-04-20T10:30:29.253 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.253+0000 7f4a69c00640 1 -- start start 2024-04-20T10:30:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a69c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a6406da90 0x7f4a640b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.255 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a69c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a64159e10 0x7f4a64151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a69c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6415a1f0 0x7f4a64152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a69c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6415d810 con 0x7f4a6406da90 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a69c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6415d530 con 0x7f4a64159e10 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a69c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a6406b900 con 0x7f4a6415a1f0 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a6406da90 0x7f4a640b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a63e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a6406da90 0x7f4a640b3570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:41268/0 (socket says 172.21.15.47:41268) 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.254+0000 7f4a63e00640 1 -- 172.21.15.47:0/2533401429 learned_addr learned my addr 172.21.15.47:0/2533401429 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a69200640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6415a1f0 0x7f4a64152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a63e00640 1 -- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6415a1f0 msgr2=0x7f4a64152c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a63e00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6415a1f0 0x7f4a64152c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a63e00640 1 -- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a64159e10 msgr2=0x7f4a64151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a63e00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a64159e10 0x7f4a64151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a63e00640 1 -- 172.21.15.47:0/2533401429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a640b3ce0 con 0x7f4a6406da90 2024-04-20T10:30:29.256 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a69200640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6415a1f0 0x7f4a64152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a63e00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a6406da90 0x7f4a640b3570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4a5402b950 tx=0x7f4a540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a5be00640 1 -- 172.21.15.47:0/2533401429 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54040070 con 0x7f4a6406da90 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a5be00640 1 -- 172.21.15.47:0/2533401429 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a54030d00 con 0x7f4a6406da90 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a5be00640 1 -- 172.21.15.47:0/2533401429 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54035040 con 0x7f4a6406da90 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.255+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a640b3fc0 con 0x7f4a6406da90 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.256+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a640b4580 con 0x7f4a6406da90 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.257+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a64075b60 con 0x7f4a6406da90 2024-04-20T10:30:29.257 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.258+0000 7f4a5be00640 1 -- 172.21.15.47:0/2533401429 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4a5404b060 con 0x7f4a6406da90 2024-04-20T10:30:29.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.259+0000 7f4a5be00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a4406c920 0x7f4a4406ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:29.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.259+0000 7f4a5be00640 1 -- 172.21.15.47:0/2533401429 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f4a540adb00 con 0x7f4a6406da90 2024-04-20T10:30:29.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.259+0000 7f4a63400640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a4406c920 0x7f4a4406ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:29.260 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.260+0000 7f4a63400640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a4406c920 0x7f4a4406ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4a50006fd0 tx=0x7f4a50008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:29.263 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.263+0000 7f4a5be00640 1 -- 172.21.15.47:0/2533401429 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4a5407bd90 con 0x7f4a6406da90 2024-04-20T10:30:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.512+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4a640b2e30 con 0x7f4a6406da90 2024-04-20T10:30:29.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.513+0000 7f4a5be00640 1 -- 172.21.15.47:0/2533401429 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v32) ==== 53+0+73 (secure 0 0 0) 0x7f4a540ade20 con 0x7f4a6406da90 2024-04-20T10:30:29.513 INFO:teuthology.orchestra.run.smithi047.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-20T10:30:29.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a4406c920 msgr2=0x7f4a4406ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a4406c920 0x7f4a4406ede0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4a50006fd0 tx=0x7f4a50008040 comp rx=0 tx=0).stop 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a6406da90 msgr2=0x7f4a640b3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a6406da90 0x7f4a640b3570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4a5402b950 tx=0x7f4a540046c0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 shutdown_connections 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a6415a1f0 0x7f4a64152c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a64159e10 0x7f4a64151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4a4406c920 0x7f4a4406ede0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 --2- 172.21.15.47:0/2533401429 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4a6406da90 0x7f4a640b3570 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 >> 172.21.15.47:0/2533401429 conn(0x7f4a640d0040 msgr2=0x7f4a641607a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.515+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 shutdown_connections 2024-04-20T10:30:29.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29.516+0000 7f4a69c00640 1 -- 172.21.15.47:0/2533401429 wait complete. 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi047.front.sepia.ceph.com 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-20T10:30:29.537 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:30:29.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:29.538 DEBUG:teuthology.orchestra.run.smithi047:> ip addr 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout: inet 127.0.0.1/8 scope host lo 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 ::1/128 scope host 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:30:29.595 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:78 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f0 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:6b:ff:79 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp7s0f1 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8e brd ff:ff:ff:ff:ff:ff 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f0 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: inet 172.21.15.47/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft 42653sec preferred_lft 42653sec 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::ec4:7aff:febd:148e/64 scope link noprefixroute 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 0c:c4:7a:bd:14:8f brd ff:ff:ff:ff:ff:ff 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout: altname enp1s0f1 2024-04-20T10:30:29.596 INFO:teuthology.orchestra.run.smithi047.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::50dc:baff:fe1b:7c56/64 scope link 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether 82:14:63:d5:b5:99 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::8014:63ff:fed5:b599/64 scope link 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: link/ether aa:c8:12:38:db:8e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: inet6 fe80::a8c8:12ff:fe38:db8e/64 scope link 2024-04-20T10:30:29.597 INFO:teuthology.orchestra.run.smithi047.stdout: valid_lft forever preferred_lft forever 2024-04-20T10:30:29.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:29.599 DEBUG:teuthology.orchestra.run.smithi047:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-20T10:30:29.681 INFO:teuthology.orchestra.run.smithi047.stdout:1 2024-04-20T10:30:29.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:29.685 DEBUG:teuthology.orchestra.run.smithi047:> route 2024-04-20T10:30:29.741 INFO:teuthology.orchestra.run.smithi047.stdout:Kernel IP routing table 2024-04-20T10:30:29.741 INFO:teuthology.orchestra.run.smithi047.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-20T10:30:29.741 INFO:teuthology.orchestra.run.smithi047.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-04-20T10:30:29.741 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-04-20T10:30:29.741 INFO:teuthology.orchestra.run.smithi047.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-20T10:30:29.742 DEBUG:teuthology.orchestra.run.smithi047:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-20T10:30:29.742 DEBUG:teuthology.orchestra.run.smithi047:> set -e 2024-04-20T10:30:29.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-04-20T10:30:29.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-04-20T10:30:29.742 DEBUG:teuthology.orchestra.run.smithi047:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-04-20T10:30:29.742 DEBUG:teuthology.orchestra.run.smithi047:> ') 2024-04-20T10:30:29.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-20T10:30:29.954 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:30:29.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:29.965 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:30:30.014 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:30:30.014 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T10:30:30.015 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:30:30.015 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:30.015 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link set brx.0 up 2024-04-20T10:30:30.104 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-04-20T10:30:30.104 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:30:30.104 DEBUG:teuthology.orchestra.run.smithi047:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-20T10:30:30.163 INFO:teuthology.orchestra.run.smithi047.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-20T10:30:30.163 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-20T10:30:30.164 DEBUG:teuthology.orchestra.run.smithi047:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:30:30.216 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:30:30.308 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:30:30.370 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:30:30.371 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:30:30.372 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:30:30.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:30.373 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:30:30.426 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-20T10:30:30.427 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-04-20T10:30:30.432 DEBUG:teuthology.orchestra.run.smithi047:> sudo modprobe fuse 2024-04-20T10:30:30.502 DEBUG:teuthology.orchestra.run.smithi047:> cat /proc/self/mounts | awk '{print $2}' 2024-04-20T10:30:30.544 INFO:teuthology.orchestra.run.smithi047.stdout:/proc 2024-04-20T10:30:30.544 INFO:teuthology.orchestra.run.smithi047.stdout:/sys 2024-04-20T10:30:30.544 INFO:teuthology.orchestra.run.smithi047.stdout:/dev 2024-04-20T10:30:30.544 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/security 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/shm 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/pts 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/run 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/cgroup 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/pstore 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/bpf 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/ 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/selinux 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/proc/sys/fs/binfmt_misc 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/hugepages 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/dev/mqueue 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/debug 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/tracing 2024-04-20T10:30:30.545 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/fs/fuse/connections 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/sys/kernel/config 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-sysctl.service 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/nfs/rpc_pipefs 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/run/user/1000 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-0 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-1 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-2 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/var/lib/ceph/osd/ceph-3 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns 2024-04-20T10:30:30.546 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:30:30.547 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:30:30.547 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:30:30.547 INFO:teuthology.orchestra.run.smithi047.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:30:30.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:30.548 DEBUG:teuthology.orchestra.run.smithi047:> ls /sys/fs/fuse/connections 2024-04-20T10:30:30.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:2024-04-20T10:30:30.603+0000 7f63ad649480 -1 init, newargv = 0x555708ea4f50 newargc=15 2024-04-20T10:30:30.603 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[41728]: starting ceph client 2024-04-20T10:30:30.614 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[41728]: starting fuse 2024-04-20T10:30:30.632 INFO:teuthology.orchestra.run.smithi047.stdout:42 2024-04-20T10:30:30.632 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [42] 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> sudo stdin-killer -- python3 -c ' 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> import glob 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> import re 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> import subprocess 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> def _find_admin_socket(client_name): 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> files = glob.glob(asok_path) 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> # Given a non-glob path, it better be there 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> if "*" not in asok_path: 2024-04-20T10:30:30.633 DEBUG:teuthology.orchestra.run.smithi047:> assert(len(files) == 1) 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> return files[0] 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> for f in files: 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> contents = proc_f.read() 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> if mountpoint in contents: 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> return f 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> print(_find_admin_socket("client.0")) 2024-04-20T10:30:30.634 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:30:30.834 INFO:teuthology.orchestra.run.smithi047.stdout:/var/run/ceph/ceph-client.0.41728.asok 2024-04-20T10:30:30.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:30:30.852 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.41728.asok 2024-04-20T10:30:30.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:30.852 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.41728.asok status 2024-04-20T10:30:31.081 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:30:31.081 INFO:teuthology.orchestra.run.smithi047.stdout: "metadata": { 2024-04-20T10:30:31.081 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_sha1": "a8c84f5997950fd7d008204839253291d52e6eda", 2024-04-20T10:30:31.081 INFO:teuthology.orchestra.run.smithi047.stdout: "ceph_version": "ceph version 19.0.0-3133-ga8c84f59 (a8c84f5997950fd7d008204839253291d52e6eda) squid (dev)", 2024-04-20T10:30:31.081 INFO:teuthology.orchestra.run.smithi047.stdout: "entity_id": "0", 2024-04-20T10:30:31.081 INFO:teuthology.orchestra.run.smithi047.stdout: "hostname": "smithi047", 2024-04-20T10:30:31.081 INFO:teuthology.orchestra.run.smithi047.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "pid": "41728", 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "root": "/" 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_count": 0, 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "dentry_pinned_count": 0, 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6219, 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "inst": { 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "name": { 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "client", 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "num": 6219 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:30:31.082 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3540034946 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: } 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": { 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "type": "v1", 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addr": "192.168.0.1:0", 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "nonce": 3540034946 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: }, 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "inst_str": "client.6219 192.168.0.1:0/3540034946", 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "addr_str": "192.168.0.1:0/3540034946", 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "inode_count": 1, 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "mds_epoch": 32, 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch": 54, 2024-04-20T10:30:31.083 INFO:teuthology.orchestra.run.smithi047.stdout: "osd_epoch_barrier": 0, 2024-04-20T10:30:31.084 INFO:teuthology.orchestra.run.smithi047.stdout: "blocklisted": false, 2024-04-20T10:30:31.084 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs" 2024-04-20T10:30:31.084 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:30:31.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:31.097 DEBUG:teuthology.orchestra.run.smithi047:> stat --file-system '--printf=%T 2024-04-20T10:30:31.097 DEBUG:teuthology.orchestra.run.smithi047:> ' -- /home/ubuntu/cephtest/mnt.0 2024-04-20T10:30:31.153 INFO:teuthology.orchestra.run.smithi047.stdout:fuseblk 2024-04-20T10:30:31.154 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-04-20T10:30:31.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:31.154 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-20T10:30:31.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:31.244 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file1) 2024-04-20T10:30:31.314 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:30:31.314 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> stdin-killer -- python3 -c ' 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> import fcntl 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> import os 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> import sys 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> import time 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> fcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK) 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> fd = os.open("/home/ubuntu/cephtest/mnt.0/background_file", os.O_RDWR | os.O_CREAT, 0o644) 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> try: 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> while True: 2024-04-20T10:30:31.374 DEBUG:teuthology.orchestra.run.smithi047:> print("write_background: writing", file=sys.stderr) 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> os.write(fd, b'"'"'content'"'"') 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> if not True: 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> break 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> try: 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> if os.read(0, 4096) == b"": 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> break 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> except BlockingIOError: 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> pass 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> time.sleep(2) 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> except IOError as e: 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> pass 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> os.close(fd) 2024-04-20T10:30:31.375 DEBUG:teuthology.orchestra.run.smithi047:> ' 2024-04-20T10:30:31.530 INFO:teuthology.orchestra.run.smithi047.stderr:write_background: writing 2024-04-20T10:30:33.540 INFO:teuthology.orchestra.run.smithi047.stderr:write_background: writing 2024-04-20T10:30:35.543 INFO:teuthology.orchestra.run.smithi047.stderr:write_background: writing 2024-04-20T10:30:37.546 INFO:teuthology.orchestra.run.smithi047.stderr:write_background: writing 2024-04-20T10:30:39.549 INFO:teuthology.orchestra.run.smithi047.stderr:write_background: writing 2024-04-20T10:30:41.482 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:41.552 INFO:teuthology.orchestra.run.smithi047.stderr:write_background: writing 2024-04-20T10:30:41.696 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.696+0000 7f871b200640 1 Processor -- start 2024-04-20T10:30:41.697 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.697+0000 7f871b200640 1 -- start start 2024-04-20T10:30:41.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.698+0000 7f871b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.698+0000 7f871b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f871c15eec0 0x7f871c15f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.698+0000 7f871b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f871c06ccc0 con 0x7f871c15f840 2024-04-20T10:30:41.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.698+0000 7f871b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f871c06d280 con 0x7f871c15eec0 2024-04-20T10:30:41.698 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.698+0000 7f871b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f871c06cfa0 con 0x7f871c15b400 2024-04-20T10:30:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.699+0000 7f8711400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f871c15eec0 0x7f871c15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.699+0000 7f8711400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f871c15eec0 0x7f871c15f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:40798/0 (socket says 172.21.15.47:40798) 2024-04-20T10:30:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.699+0000 7f8711400640 1 -- 172.21.15.47:0/2501104105 learned_addr learned my addr 172.21.15.47:0/2501104105 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:41.699 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.699+0000 7f8719e00640 1 --2- 172.21.15.47:0/2501104105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.700+0000 7f8719e00640 1 -- 172.21.15.47:0/2501104105 >> v2:172.21.15.175:3300/0 conn(0x7f871c15eec0 msgr2=0x7f871c15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.700+0000 7f8719e00640 1 --2- 172.21.15.47:0/2501104105 >> v2:172.21.15.175:3300/0 conn(0x7f871c15eec0 0x7f871c15f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.700+0000 7f8719e00640 1 -- 172.21.15.47:0/2501104105 >> v1:172.21.15.175:6789/0 conn(0x7f871c15f840 legacy=0x7f871c15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.700+0000 7f8719e00640 1 -- 172.21.15.47:0/2501104105 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c06e440 con 0x7f871c15b400 2024-04-20T10:30:41.700 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.700+0000 7f8711400640 1 --2- 172.21.15.47:0/2501104105 >> v2:172.21.15.175:3300/0 conn(0x7f871c15eec0 0x7f871c15f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:41.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.701+0000 7f8719e00640 1 --2- 172.21.15.47:0/2501104105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c072d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f870c0097a0 tx=0x7f870c02dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3f69ea9187ed8d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:41.701 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.701+0000 7f8719400640 1 -- 172.21.15.47:0/2501104105 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c02f8a0 con 0x7f871c15b400 2024-04-20T10:30:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f8719400640 1 -- 172.21.15.47:0/2501104105 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870c02fa40 con 0x7f871c15b400 2024-04-20T10:30:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f8719400640 1 -- 172.21.15.47:0/2501104105 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c02fe10 con 0x7f871c15b400 2024-04-20T10:30:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 -- 172.21.15.47:0/2501104105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 msgr2=0x7f871c072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.702 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 --2- 172.21.15.47:0/2501104105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c072d00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f870c0097a0 tx=0x7f870c02dae0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 -- 172.21.15.47:0/2501104105 shutdown_connections 2024-04-20T10:30:41.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 --2- 172.21.15.47:0/2501104105 >> v2:172.21.15.175:3300/0 conn(0x7f871c15eec0 0x7f871c15f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 --2- 172.21.15.47:0/2501104105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c072d00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 -- 172.21.15.47:0/2501104105 >> 172.21.15.47:0/2501104105 conn(0x7f871c0d0040 msgr2=0x7f871c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:41.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 -- 172.21.15.47:0/2501104105 shutdown_connections 2024-04-20T10:30:41.703 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.702+0000 7f871b200640 1 -- 172.21.15.47:0/2501104105 wait complete. 2024-04-20T10:30:41.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.704+0000 7f871b200640 1 Processor -- start 2024-04-20T10:30:41.704 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.704+0000 7f871b200640 1 -- start start 2024-04-20T10:30:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.705+0000 7f871b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c14e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.705+0000 7f871b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f871c15eec0 0x7f871c1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.705+0000 7f871b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c15f840 0x7f871c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.705+0000 7f871b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f871c073240 con 0x7f871c15eec0 2024-04-20T10:30:41.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.705+0000 7f871b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f871c15fc90 con 0x7f871c15f840 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.705+0000 7f871b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f871c06fc50 con 0x7f871c15b400 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f871c15eec0 0x7f871c1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f871c15eec0 0x7f871c1567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38346/0 (socket says 172.21.15.47:38346) 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 -- 172.21.15.47:0/3241792906 learned_addr learned my addr 172.21.15.47:0/3241792906 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8719e00640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c14e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f871a800640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c15f840 0x7f871c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 -- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 msgr2=0x7f871c14e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c14e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 -- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c15f840 msgr2=0x7f871c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c15f840 0x7f871c0cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 -- 172.21.15.47:0/3241792906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c0cbc20 con 0x7f871c15eec0 2024-04-20T10:30:41.707 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.706+0000 7f8711400640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f871c15eec0 0x7f871c1567c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f870400e9d0 tx=0x7f870400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:41.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.707+0000 7f8712a00640 1 -- 172.21.15.47:0/3241792906 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704017070 con 0x7f871c15eec0 2024-04-20T10:30:41.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.707+0000 7f8712a00640 1 -- 172.21.15.47:0/3241792906 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f870400ca60 con 0x7f871c15eec0 2024-04-20T10:30:41.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.707+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c0cd2b0 con 0x7f871c15eec0 2024-04-20T10:30:41.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.707+0000 7f8712a00640 1 -- 172.21.15.47:0/3241792906 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870400ce10 con 0x7f871c15eec0 2024-04-20T10:30:41.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.707+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c0cd870 con 0x7f871c15eec0 2024-04-20T10:30:41.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.708+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c0b1a90 con 0x7f871c15eec0 2024-04-20T10:30:41.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.709+0000 7f8712a00640 1 -- 172.21.15.47:0/3241792906 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f870401a070 con 0x7f871c15eec0 2024-04-20T10:30:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.710+0000 7f8712a00640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f86f806c8f0 0x7f86f806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:41.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.711+0000 7f8712a00640 1 -- 172.21.15.47:0/3241792906 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f870408bff0 con 0x7f871c15eec0 2024-04-20T10:30:41.711 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.711+0000 7f8719e00640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f86f806c8f0 0x7f86f806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:41.712 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.712+0000 7f8719e00640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f86f806c8f0 0x7f86f806edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f870c038000 tx=0x7f870c008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:41.715 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.716+0000 7f8712a00640 1 -- 172.21.15.47:0/3241792906 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f871c0b1a90 con 0x7f871c15eec0 2024-04-20T10:30:41.957 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.957+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f871c0b5220 con 0x7f871c15eec0 2024-04-20T10:30:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.958+0000 7f8712a00640 1 -- 172.21.15.47:0/3241792906 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+5202 (secure 0 0 0) 0x7f870405a2c0 con 0x7f871c15eec0 2024-04-20T10:30:41.959 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:41.959 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5392,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3182036779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3182036779},{"type":"v1","addr":"172.21.15.175:6835","nonce":3182036779}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":5398,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1207907337","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1207907337},{"type":"v1","addr":"172.21.15.175:6837","nonce":1207907337}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/793057653","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":793057653},{"type":"v1","addr":"172.21.15.47:6833","nonce":793057653}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:24.860182+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6057},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6057":{"gid":6057,"name":"a","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.47:6835/3814505199","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3814505199},{"type":"v1","addr":"172.21.15.47:6835","nonce":3814505199}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6057,"qdb_cluster":[6057]},"id":3}]} 2024-04-20T10:30:41.959 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 32 2024-04-20T10:30:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f86f806c8f0 msgr2=0x7f86f806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f86f806c8f0 0x7f86f806edb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f870c038000 tx=0x7f870c008030 comp rx=0 tx=0).stop 2024-04-20T10:30:41.961 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f871c15eec0 msgr2=0x7f871c1567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f871c15eec0 0x7f871c1567c0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f870400e9d0 tx=0x7f870400ee90 comp rx=0 tx=0).stop 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 shutdown_connections 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f871c15f840 0x7f871c0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f86f806c8f0 0x7f86f806edb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f871c15eec0 0x7f871c1567c0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 --2- 172.21.15.47:0/3241792906 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f871c15b400 0x7f871c14e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 >> 172.21.15.47:0/3241792906 conn(0x7f871c0d0040 msgr2=0x7f871c0b1380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 shutdown_connections 2024-04-20T10:30:41.962 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:41.961+0000 7f871b200640 1 -- 172.21.15.47:0/3241792906 wait complete. 2024-04-20T10:30:41.980 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json force_readonly 2024-04-20T10:30:42.205 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2024-04-20T10:30:43.554 INFO:teuthology.orchestra.run.smithi047.stderr:write_background: writing 2024-04-20T10:30:43.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-20T10:30:52.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:30:52.206 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch test_file1) 2024-04-20T10:30:52.272 INFO:teuthology.orchestra.run.smithi047.stderr:touch: cannot touch 'test_file1': Read-only file system 2024-04-20T10:30:52.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:30:52.274 INFO:tasks.ceph_test_case:waiting 30s for health warning matching MDS_READ_ONLY 2024-04-20T10:30:52.274 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:30:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.487+0000 7fcedb200640 1 Processor -- start 2024-04-20T10:30:52.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.487+0000 7fcedb200640 1 -- start start 2024-04-20T10:30:52.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.488+0000 7fcedb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 0x7fcedc0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.488+0000 7fcedb200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcedc0adf60 0x7fcedc15f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.488+0000 7fcedb200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcedc071a80 con 0x7fcedc15faf0 2024-04-20T10:30:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.488+0000 7fcedb200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcedc0ae3c0 con 0x7fcedc0adf60 2024-04-20T10:30:52.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.488+0000 7fcedb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc071cf0 con 0x7fcedc0b2f70 2024-04-20T10:30:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.488+0000 7fceda800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fcedc15faf0 0x7fcedc06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59824/0 (socket says 172.21.15.47:59824) 2024-04-20T10:30:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.488+0000 7fceda800640 1 -- 172.21.15.47:0/1184377467 learned_addr learned my addr 172.21.15.47:0/1184377467 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9e00640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 0x7fcedc0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced3e00640 1 --2- 172.21.15.47:0/1184377467 >> v2:172.21.15.175:3300/0 conn(0x7fcedc0adf60 0x7fcedc15f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1377815039 0 0) 0x7fcedc071a80 con 0x7fcedc15faf0 2024-04-20T10:30:52.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fcec0003520 con 0x7fcedc15faf0 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1043918599 0 0) 0x7fcec0003520 con 0x7fcedc15faf0 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 msgr2=0x7fcedc0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9400640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 0x7fcedc0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 >> v2:172.21.15.175:3300/0 conn(0x7fcedc0adf60 msgr2=0x7fcedc15f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.489+0000 7fced9400640 1 --2- 172.21.15.47:0/1184377467 >> v2:172.21.15.175:3300/0 conn(0x7fcedc0adf60 0x7fcedc15f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc071fd0 con 0x7fcedc15faf0 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fced40019f0 con 0x7fcedc15faf0 2024-04-20T10:30:52.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced3e00640 1 --2- 172.21.15.47:0/1184377467 >> v2:172.21.15.175:3300/0 conn(0x7fcedc0adf60 0x7fcedc15f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:52.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 >> v1:172.21.15.47:6789/0 conn(0x7fcedc15faf0 legacy=0x7fcedc06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcec00039b0 0x7fcec0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 0x7fcec0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 0x7fcec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0ae3c0 con 0x7fcec00039b0 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc071cf0 con 0x7fcec0004800 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcec000b9c0 con 0x7fcedc0b2f70 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9e00640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcec00039b0 0x7fcec0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced3e00640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 0x7fcec0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fced9e00640 1 -- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 msgr2=0x7fcec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.490+0000 7fceda800640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 0x7fcec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.491+0000 7fced9e00640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 0x7fcec00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.491+0000 7fced9e00640 1 -- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 msgr2=0x7fcec0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.491+0000 7fced9e00640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 0x7fcec0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.491+0000 7fced9e00640 1 -- 172.21.15.47:0/1184377467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcec0003520 con 0x7fcec00039b0 2024-04-20T10:30:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.491+0000 7fced3e00640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcec0004800 0x7fcec0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.491+0000 7fceda800640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcedc0b2f70 0x7fcec00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:52.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.492+0000 7fced9e00640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcec00039b0 0x7fcec0003dd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009ac0 tx=0x7fcec0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d14c389180fe35bc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.492+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc039070 con 0x7fcec00039b0 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.492+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcecc02c6b0 con 0x7fcec00039b0 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.492+0000 7fced9400640 1 -- 172.21.15.47:0/1184377467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcecc009690 con 0x7fcec00039b0 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.492+0000 7fcedb200640 1 -- 172.21.15.47:0/1184377467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcec00039b0 msgr2=0x7fcec0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.492+0000 7fcedb200640 1 --2- 172.21.15.47:0/1184377467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcec00039b0 0x7fcec0003dd0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009ac0 tx=0x7fcec0005830 comp rx=0 tx=0).stop 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.493+0000 7fceda800640 1 -- 172.21.15.47:0/1184377467 reap_dead start 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.493+0000 7fcedb200640 1 -- 172.21.15.47:0/1184377467 shutdown_connections 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.493+0000 7fcedb200640 1 -- 172.21.15.47:0/1184377467 >> 172.21.15.47:0/1184377467 conn(0x7fcedc0d0040 msgr2=0x7fcedc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:52.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.493+0000 7fcedb200640 1 -- 172.21.15.47:0/1184377467 shutdown_connections 2024-04-20T10:30:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.493+0000 7fcedb200640 1 -- 172.21.15.47:0/1184377467 wait complete. 2024-04-20T10:30:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.495+0000 7fcedb200640 1 Processor -- start 2024-04-20T10:30:52.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.495+0000 7fcedb200640 1 -- start start 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fcedb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec00039b0 0x7fcedc15d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fcedb200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc15c030 0x7fcedc15c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fcedb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc0b0bf0 0x7fcedc0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fcedb200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc06c010 con 0x7fcedc15c030 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fcedb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcedc0736c0 con 0x7fcedc0b0bf0 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fcedb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcedc06c190 con 0x7fcec00039b0 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc15c030 0x7fcedc15c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc15c030 0x7fcedc15c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59968/0 (socket says 172.21.15.47:59968) 2024-04-20T10:30:52.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 -- 172.21.15.47:0/3906573926 learned_addr learned my addr 172.21.15.47:0/3906573926 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced9e00640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec00039b0 0x7fcedc15d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 -- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec00039b0 msgr2=0x7fcedc15d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec00039b0 0x7fcedc15d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 -- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc0b0bf0 msgr2=0x7fcedc0b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc0b0bf0 0x7fcedc0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.496+0000 7fced3e00640 1 -- 172.21.15.47:0/3906573926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcedc157720 con 0x7fcedc15c030 2024-04-20T10:30:52.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.497+0000 7fced3e00640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc15c030 0x7fcedc15c4b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcec400f600 tx=0x7fcec400fac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.497+0000 7fced2000640 1 -- 172.21.15.47:0/3906573926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4018070 con 0x7fcedc15c030 2024-04-20T10:30:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.497+0000 7fced2000640 1 -- 172.21.15.47:0/3906573926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcec4014890 con 0x7fcedc15c030 2024-04-20T10:30:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.497+0000 7fced2000640 1 -- 172.21.15.47:0/3906573926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcec4014c80 con 0x7fcedc15c030 2024-04-20T10:30:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.497+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcedc157a00 con 0x7fcedc15c030 2024-04-20T10:30:52.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.497+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcedc1584e0 con 0x7fcedc15c030 2024-04-20T10:30:52.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.498+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcedc0b15b0 con 0x7fcedc15c030 2024-04-20T10:30:52.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.499+0000 7fced2000640 1 -- 172.21.15.47:0/3906573926 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcec401b070 con 0x7fcedc15c030 2024-04-20T10:30:52.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.500+0000 7fced2000640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fceb806c920 0x7fceb806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:52.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.500+0000 7fced2000640 1 -- 172.21.15.47:0/3906573926 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fcec408bf70 con 0x7fcedc15c030 2024-04-20T10:30:52.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.501+0000 7fced9e00640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fceb806c920 0x7fceb806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:52.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.502+0000 7fced9e00640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fceb806c920 0x7fceb806ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcecc036000 tx=0x7fcecc030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:52.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.505+0000 7fced2000640 1 -- 172.21.15.47:0/3906573926 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcec405a200 con 0x7fcedc15c030 2024-04-20T10:30:52.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.751+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fcedc06d220 con 0x7fcedc15c030 2024-04-20T10:30:52.752 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.752+0000 7fced2000640 1 -- 172.21.15.47:0/3906573926 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+159 (secure 0 0 0) 0x7fcec4014e20 con 0x7fcedc15c030 2024-04-20T10:30:52.752 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:52.752 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_WARN","checks":{"MDS_READ_ONLY":{"severity":"HEALTH_WARN","summary":{"message":"1 MDSs are read only","count":1},"muted":false}},"mutes":[]} 2024-04-20T10:30:52.758 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.756+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fceb806c920 msgr2=0x7fceb806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.756+0000 7fcedb200640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fceb806c920 0x7fceb806ede0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcecc036000 tx=0x7fcecc030000 comp rx=0 tx=0).stop 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.756+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc15c030 msgr2=0x7fcedc15c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.756+0000 7fcedb200640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc15c030 0x7fcedc15c4b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fcec400f600 tx=0x7fcec400fac0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 shutdown_connections 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcedc0b0bf0 0x7fcedc0b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fceb806c920 0x7fceb806ede0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcedc15c030 0x7fcedc15c4b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 --2- 172.21.15.47:0/3906573926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcec00039b0 0x7fcedc15d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 >> 172.21.15.47:0/3906573926 conn(0x7fcedc0d0040 msgr2=0x7fcedc06f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 shutdown_connections 2024-04-20T10:30:52.759 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:52.757+0000 7fcedb200640 1 -- 172.21.15.47:0/3906573926 wait complete. 2024-04-20T10:30:52.776 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T10:30:52.776 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T10:30:52.776 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:52.777 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T10:30:52.777 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:52.777 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T10:30:52.777 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:52.777 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T10:30:52.778 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:30:52.800 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T10:30:52.800 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-20T10:30:52.801 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T10:30:52.801 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-20T10:30:52.802 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T10:30:52.802 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-20T10:30:52.806 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T10:30:52.806 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-20T10:30:53.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.033+0000 7f7151a00640 1 Processor -- start 2024-04-20T10:30:53.033 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.033+0000 7f7151a00640 1 -- start start 2024-04-20T10:30:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.034+0000 7f57a0c00640 1 Processor -- start 2024-04-20T10:30:53.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.034+0000 7f57a0c00640 1 -- start start 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.035+0000 7f7151a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f714c06cde0 0x7f714c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.035+0000 7f7151a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714c06c190 con 0x7f714c0b3330 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.035+0000 7f7151a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f714c06c470 con 0x7f714c161d50 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.035+0000 7f7151a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f714c162130 con 0x7f714c06cde0 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.035+0000 7f7151000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f714c0b3330 0x7f714c0b7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59826/0 (socket says 172.21.15.47:59826) 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.035+0000 7f7151000640 1 -- 172.21.15.47:0/2484504644 learned_addr learned my addr 172.21.15.47:0/2484504644 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.035+0000 7f714b400640 1 --2- 172.21.15.47:0/2484504644 >> v2:172.21.15.175:3300/0 conn(0x7f714c06cde0 0x7f714c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f714aa00640 1 -- 172.21.15.47:0/2484504644 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 734883317 0 0) 0x7f714c06c190 con 0x7f714c0b3330 2024-04-20T10:30:53.036 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f714aa00640 1 -- 172.21.15.47:0/2484504644 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f7130003520 con 0x7f714c0b3330 2024-04-20T10:30:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f57a0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c161d50 0x7f579c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f57a0c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f579c06cde0 0x7f579c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f57a0c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f579c06c190 con 0x7f579c0b3330 2024-04-20T10:30:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f57a0c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f579c162130 con 0x7f579c06cde0 2024-04-20T10:30:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f57a0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f579c06c470 con 0x7f579c161d50 2024-04-20T10:30:53.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f714b400640 1 -- 172.21.15.47:0/2484504644 >> v1:172.21.15.175:6789/0 conn(0x7f714c161d50 legacy=0x7f714c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f714b400640 1 -- 172.21.15.47:0/2484504644 >> v1:172.21.15.47:6789/0 conn(0x7f714c0b3330 legacy=0x7f714c0b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f714b400640 1 -- 172.21.15.47:0/2484504644 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c0d2e00 con 0x7f714c06cde0 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f579aa00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f579c0b3330 0x7f579c06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59838/0 (socket says 172.21.15.47:59838) 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f579aa00640 1 -- 172.21.15.47:0/3155686521 learned_addr learned my addr 172.21.15.47:0/3155686521 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.036+0000 7f579be00640 1 --2- 172.21.15.47:0/3155686521 >> v2:172.21.15.47:3300/0 conn(0x7f579c06cde0 0x7f579c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.037+0000 7f579b400640 1 --2- 172.21.15.47:0/3155686521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c161d50 0x7f579c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.037+0000 7f579a000640 1 -- 172.21.15.47:0/3155686521 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 777409126 0 0) 0x7f579c06c190 con 0x7f579c0b3330 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.037+0000 7f579a000640 1 -- 172.21.15.47:0/3155686521 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f5780003520 con 0x7f579c0b3330 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.037+0000 7f579be00640 1 -- 172.21.15.47:0/3155686521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c161d50 msgr2=0x7f579c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.037+0000 7f579be00640 1 --2- 172.21.15.47:0/3155686521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c161d50 0x7f579c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.037+0000 7f579be00640 1 -- 172.21.15.47:0/3155686521 >> v1:172.21.15.47:6789/0 conn(0x7f579c0b3330 legacy=0x7f579c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.037+0000 7f579be00640 1 -- 172.21.15.47:0/3155686521 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c0d2e00 con 0x7f579c06cde0 2024-04-20T10:30:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.038+0000 7f714b400640 1 --2- 172.21.15.47:0/2484504644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c06cde0 0x7f714c0d38e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f71380097a0 tx=0x7f713802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ab659f3a8be49ff server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.039+0000 7f714aa00640 1 -- 172.21.15.47:0/2484504644 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138034070 con 0x7f714c06cde0 2024-04-20T10:30:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.039+0000 7f714aa00640 1 -- 172.21.15.47:0/2484504644 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7138030840 con 0x7f714c06cde0 2024-04-20T10:30:53.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.039+0000 7f714aa00640 1 -- 172.21.15.47:0/2484504644 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138030c10 con 0x7f714c06cde0 2024-04-20T10:30:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.039+0000 7f579be00640 1 --2- 172.21.15.47:0/3155686521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c06cde0 0x7f579c0b3880 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5794009d90 tx=0x7f579402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b438aad997d1756c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.039+0000 7f579a000640 1 -- 172.21.15.47:0/3155686521 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794039070 con 0x7f579c06cde0 2024-04-20T10:30:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f579a000640 1 -- 172.21.15.47:0/3155686521 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f579402cc60 con 0x7f579c06cde0 2024-04-20T10:30:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f579a000640 1 -- 172.21.15.47:0/3155686521 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794034660 con 0x7f579c06cde0 2024-04-20T10:30:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f7151a00640 1 -- 172.21.15.47:0/2484504644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c06cde0 msgr2=0x7f714c0d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f7151a00640 1 --2- 172.21.15.47:0/2484504644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c06cde0 0x7f714c0d38e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f71380097a0 tx=0x7f713802b980 comp rx=0 tx=0).stop 2024-04-20T10:30:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f7151a00640 1 -- 172.21.15.47:0/2484504644 shutdown_connections 2024-04-20T10:30:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f7151a00640 1 --2- 172.21.15.47:0/2484504644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c06cde0 0x7f714c0d38e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f7151a00640 1 -- 172.21.15.47:0/2484504644 >> 172.21.15.47:0/2484504644 conn(0x7f714c0d4450 msgr2=0x7f714c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f7151a00640 1 -- 172.21.15.47:0/2484504644 shutdown_connections 2024-04-20T10:30:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f57a0c00640 1 -- 172.21.15.47:0/3155686521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c06cde0 msgr2=0x7f579c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f57a0c00640 1 --2- 172.21.15.47:0/3155686521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c06cde0 0x7f579c0b3880 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f5794009d90 tx=0x7f579402beb0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.042 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.040+0000 7f7151a00640 1 -- 172.21.15.47:0/2484504644 wait complete. 2024-04-20T10:30:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.041+0000 7f57a0c00640 1 -- 172.21.15.47:0/3155686521 shutdown_connections 2024-04-20T10:30:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.041+0000 7f57a0c00640 1 --2- 172.21.15.47:0/3155686521 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c06cde0 0x7f579c0b3880 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.041+0000 7f57a0c00640 1 --2- 172.21.15.47:0/3155686521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c161d50 0x7f579c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.041+0000 7f57a0c00640 1 -- 172.21.15.47:0/3155686521 >> 172.21.15.47:0/3155686521 conn(0x7f579c0d4450 msgr2=0x7f579c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.041+0000 7f57a0c00640 1 -- 172.21.15.47:0/3155686521 shutdown_connections 2024-04-20T10:30:53.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.041+0000 7f57a0c00640 1 -- 172.21.15.47:0/3155686521 wait complete. 2024-04-20T10:30:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.042+0000 7fe722800640 1 Processor -- start 2024-04-20T10:30:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.042+0000 7fe722800640 1 -- start start 2024-04-20T10:30:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.042+0000 7f7151a00640 1 Processor -- start 2024-04-20T10:30:53.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7f57a0c00640 1 Processor -- start 2024-04-20T10:30:53.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7f7151a00640 1 -- start start 2024-04-20T10:30:53.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7f57a0c00640 1 -- start start 2024-04-20T10:30:53.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7fe722800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c161d50 0x7fe71c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7fe722800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe71c06cde0 0x7fe71c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7fe722800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe71c06c190 con 0x7fe71c0b3330 2024-04-20T10:30:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7fc83ae00640 1 Processor -- start 2024-04-20T10:30:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7fe722800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe71c162130 con 0x7fe71c06cde0 2024-04-20T10:30:53.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.043+0000 7fe722800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c06c470 con 0x7fe71c161d50 2024-04-20T10:30:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7fc83ae00640 1 -- start start 2024-04-20T10:30:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f7151a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f714c06cde0 0x7f714c15b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f7151a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f714c0b3330 0x7f714c0c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f57a0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f579c06cde0 0x7f579c0ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f57a0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c0b3330 0x7f579c0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f57a0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c161d50 0x7f579c0cd050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f57a0c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f579c0b7660 con 0x7f579c161d50 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f57a0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f579c0d33c0 con 0x7f579c06cde0 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f57a0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f579c0b77e0 con 0x7f579c0b3330 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f7151a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c161d50 0x7f714c15d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f7151a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0b3880 con 0x7f714c0b3330 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f7151a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f714c0d33c0 con 0x7f714c161d50 2024-04-20T10:30:53.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f7151a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f714c0b3a00 con 0x7f714c06cde0 2024-04-20T10:30:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7fe720a00640 1 --1- >> v1:172.21.15.175:6789/0 conn(0x7fe71c0b3330 0x7fe71c06c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.175:6789/0 says I am v1:172.21.15.47:57168/0 (socket says 172.21.15.47:57168) 2024-04-20T10:30:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7fe720a00640 1 -- 172.21.15.47:0/4034850537 learned_addr learned my addr 172.21.15.47:0/4034850537 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7fe721400640 1 --2- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c161d50 0x7fe71c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7fe721e00640 1 --2- 172.21.15.47:0/4034850537 >> v2:172.21.15.175:3300/0 conn(0x7fe71c06cde0 0x7fe71c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.044+0000 7f714b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f714c0b3330 0x7f714c0c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f714c0b3330 0x7f714c0c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:59994/0 (socket says 172.21.15.47:59994) 2024-04-20T10:30:53.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714b400640 1 -- 172.21.15.47:0/3285814080 learned_addr learned my addr 172.21.15.47:0/3285814080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:53.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f579be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c161d50 0x7f579c0cd050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f579be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c161d50 0x7f579c0cd050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60008/0 (socket says 172.21.15.47:60008) 2024-04-20T10:30:53.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f579be00640 1 -- 172.21.15.47:0/4060194223 learned_addr learned my addr 172.21.15.47:0/4060194223 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:53.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714be00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f714c06cde0 0x7f714c15b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f579b400640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f579c06cde0 0x7f579c0ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f7151000640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c161d50 0x7f714c15d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.051 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f579aa00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c0b3330 0x7f579c0ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714b400640 1 -- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f714c06cde0 msgr2=0x7f714c15b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714b400640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f714c06cde0 0x7f714c15b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7fe71be00640 1 -- 172.21.15.47:0/4034850537 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 642098384 0 0) 0x7fe71c06c190 con 0x7fe71c0b3330 2024-04-20T10:30:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7fe71be00640 1 -- 172.21.15.47:0/4034850537 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe704003520 con 0x7fe71c0b3330 2024-04-20T10:30:53.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f579be00640 1 -- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c0b3330 msgr2=0x7f579c0ccb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714b400640 1 -- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c161d50 msgr2=0x7f714c15d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714b400640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c161d50 0x7f714c15d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714b400640 1 -- 172.21.15.47:0/3285814080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f714c0c7f60 con 0x7f714c0b3330 2024-04-20T10:30:53.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.045+0000 7f714be00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f714c06cde0 0x7f714c15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:53.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7f579be00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c0b3330 0x7f579c0ccb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7f579be00640 1 -- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f579c06cde0 msgr2=0x7f579c0ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7f579be00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f579c06cde0 0x7f579c0ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7f579be00640 1 -- 172.21.15.47:0/4060194223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579c14fea0 con 0x7f579c161d50 2024-04-20T10:30:53.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fe721e00640 1 -- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c161d50 msgr2=0x7fe71c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:53.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fe721e00640 1 --2- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c161d50 0x7fe71c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fe721e00640 1 -- 172.21.15.47:0/4034850537 >> v1:172.21.15.175:6789/0 conn(0x7fe71c0b3330 legacy=0x7fe71c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fe721e00640 1 -- 172.21.15.47:0/4034850537 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c0d2e00 con 0x7fe71c06cde0 2024-04-20T10:30:53.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7f579aa00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c0b3330 0x7f579c0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:30:53.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fc83ae00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fc8340d1a00 0x7fc8340b1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fc83ae00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8340d1e90 con 0x7fc8340b17d0 2024-04-20T10:30:53.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fc83ae00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8340d3820 con 0x7fc8340d4270 2024-04-20T10:30:53.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fc83ae00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fc8340d40c0 con 0x7fc8340d1a00 2024-04-20T10:30:53.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7f714b400640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f714c0b3330 0x7f714c0c9d00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f713802bf90 tx=0x7f71380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7f579be00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c161d50 0x7f579c0cd050 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5794009d90 tx=0x7f5794004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fc838e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc8340b17d0 0x7fc8340d0380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59840/0 (socket says 172.21.15.47:59840) 2024-04-20T10:30:53.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.046+0000 7fc838e00640 1 -- 172.21.15.47:0/2637145390 learned_addr learned my addr 172.21.15.47:0/2637145390 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:53.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7fc833400640 1 --2- 172.21.15.47:0/2637145390 >> v2:172.21.15.175:3300/0 conn(0x7fc8340d1a00 0x7fc8340b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f7143e00640 1 -- 172.21.15.47:0/3285814080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138034040 con 0x7f714c0b3330 2024-04-20T10:30:53.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f7143e00640 1 -- 172.21.15.47:0/3285814080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7138030640 con 0x7f714c0b3330 2024-04-20T10:30:53.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f7143e00640 1 -- 172.21.15.47:0/3285814080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7138006e10 con 0x7f714c0b3330 2024-04-20T10:30:53.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f5793400640 1 -- 172.21.15.47:0/4060194223 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794039050 con 0x7f579c161d50 2024-04-20T10:30:53.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f5793400640 1 -- 172.21.15.47:0/4060194223 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5794045070 con 0x7f579c161d50 2024-04-20T10:30:53.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f5793400640 1 -- 172.21.15.47:0/4060194223 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794009450 con 0x7f579c161d50 2024-04-20T10:30:53.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579c14fbc0 con 0x7f579c161d50 2024-04-20T10:30:53.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f714c0c8240 con 0x7f714c0b3330 2024-04-20T10:30:53.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f714c0c8800 con 0x7f714c0b3330 2024-04-20T10:30:53.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7fc832a00640 1 -- 172.21.15.47:0/2637145390 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 703667410 0 0) 0x7fc82c001490 con 0x7fc8340b17d0 2024-04-20T10:30:53.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fe721e00640 1 --2- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c06cde0 0x7fe71c0b3880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe714009d90 tx=0x7fe71402beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c01b81bd9b5da10 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.047+0000 7fc832a00640 1 -- 172.21.15.47:0/2637145390 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc818003520 con 0x7fc8340b17d0 2024-04-20T10:30:53.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fc832a00640 1 -- 172.21.15.47:0/2637145390 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3767916425 0 0) 0x7fc8340d3820 con 0x7fc8340d4270 2024-04-20T10:30:53.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fc832a00640 1 -- 172.21.15.47:0/2637145390 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc82c001490 con 0x7fc8340d4270 2024-04-20T10:30:53.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fc833400640 1 -- 172.21.15.47:0/2637145390 >> v1:172.21.15.175:6789/0 conn(0x7fc8340d4270 legacy=0x7fc8340d4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fc833400640 1 -- 172.21.15.47:0/2637145390 >> v1:172.21.15.47:6789/0 conn(0x7fc8340b17d0 legacy=0x7fc8340d0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fc833400640 1 -- 172.21.15.47:0/2637145390 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8340d3b00 con 0x7fc8340d1a00 2024-04-20T10:30:53.059 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fe71be00640 1 -- 172.21.15.47:0/4034850537 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714039070 con 0x7fe71c06cde0 2024-04-20T10:30:53.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f579c0b0410 con 0x7f579c161d50 2024-04-20T10:30:53.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fe71be00640 1 -- 172.21.15.47:0/4034850537 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe71402cc60 con 0x7fe71c06cde0 2024-04-20T10:30:53.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.048+0000 7fe71be00640 1 -- 172.21.15.47:0/4034850537 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe714034640 con 0x7fe71c06cde0 2024-04-20T10:30:53.060 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.049+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f714c075b60 con 0x7f714c0b3330 2024-04-20T10:30:53.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 -- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c06cde0 msgr2=0x7fe71c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 --2- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c06cde0 0x7fe71c0b3880 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe714009d90 tx=0x7fe71402beb0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f579c075b60 con 0x7f579c161d50 2024-04-20T10:30:53.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 -- 172.21.15.47:0/4034850537 shutdown_connections 2024-04-20T10:30:53.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 --2- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c06cde0 0x7fe71c0b3880 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.061 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 --2- 172.21.15.47:0/4034850537 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c161d50 0x7fe71c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 -- 172.21.15.47:0/4034850537 >> 172.21.15.47:0/4034850537 conn(0x7fe71c0d4450 msgr2=0x7fe71c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 -- 172.21.15.47:0/4034850537 shutdown_connections 2024-04-20T10:30:53.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fc833400640 1 --2- 172.21.15.47:0/2637145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340b1290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc8200097a0 tx=0x7fc82002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a2d1460a275e38b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7f7143e00640 1 -- 172.21.15.47:0/3285814080 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f7138037070 con 0x7f714c0b3330 2024-04-20T10:30:53.062 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.050+0000 7fe722800640 1 -- 172.21.15.47:0/4034850537 wait complete. 2024-04-20T10:30:53.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.051+0000 7f5793400640 1 -- 172.21.15.47:0/4060194223 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f579403e850 con 0x7f579c161d50 2024-04-20T10:30:53.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.051+0000 7fc832a00640 1 -- 172.21.15.47:0/2637145390 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820034070 con 0x7fc8340d1a00 2024-04-20T10:30:53.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.051+0000 7fc832a00640 1 -- 172.21.15.47:0/2637145390 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc820030610 con 0x7fc8340d1a00 2024-04-20T10:30:53.063 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.051+0000 7fc832a00640 1 -- 172.21.15.47:0/2637145390 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200309e0 con 0x7fc8340d1a00 2024-04-20T10:30:53.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.051+0000 7f7143e00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f712c06c940 0x7f712c06ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7f7143e00640 1 -- 172.21.15.47:0/3285814080 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f71380ae9c0 con 0x7f714c0b3330 2024-04-20T10:30:53.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7f5793400640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f577806c970 0x7f577806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7fc83ae00640 1 -- 172.21.15.47:0/2637145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 msgr2=0x7fc8340b1290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.064 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7fc83ae00640 1 --2- 172.21.15.47:0/2637145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340b1290 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc8200097a0 tx=0x7fc82002b960 comp rx=0 tx=0).stop 2024-04-20T10:30:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7fc83ae00640 1 -- 172.21.15.47:0/2637145390 shutdown_connections 2024-04-20T10:30:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7fc83ae00640 1 --2- 172.21.15.47:0/2637145390 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340b1290 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7fc83ae00640 1 -- 172.21.15.47:0/2637145390 >> 172.21.15.47:0/2637145390 conn(0x7fc8340b99c0 msgr2=0x7fc8340b7530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.052+0000 7f714be00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f712c06c940 0x7f712c06ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.053+0000 7f579b400640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f577806c970 0x7f577806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.053+0000 7fc83ae00640 1 -- 172.21.15.47:0/2637145390 shutdown_connections 2024-04-20T10:30:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.053+0000 7fc83ae00640 1 -- 172.21.15.47:0/2637145390 wait complete. 2024-04-20T10:30:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.053+0000 7fe722800640 1 Processor -- start 2024-04-20T10:30:53.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.054+0000 7f714be00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f712c06c940 0x7f712c06ee00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f713c001660 tx=0x7f713c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.054+0000 7fe722800640 1 -- start start 2024-04-20T10:30:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.054+0000 7f579b400640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f577806c970 0x7f577806ee30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5784009aa0 tx=0x7f5784009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fc83ae00640 1 Processor -- start 2024-04-20T10:30:53.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fc83ae00640 1 -- start start 2024-04-20T10:30:53.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fe722800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c06cde0 0x7fe71c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fe722800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c0b3330 0x7fe71c15cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.068 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fe722800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c161d50 0x7fe71c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fe722800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0b7660 con 0x7fe71c161d50 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fe722800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe71c0d33c0 con 0x7fe71c0b3330 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.055+0000 7fe722800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe71c0b77e0 con 0x7fe71c06cde0 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fe721400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c06cde0 0x7fe71c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fe721400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c06cde0 0x7fe71c154960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.47:49656/0 (socket says 172.21.15.47:49656) 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fe721400640 1 -- 172.21.15.47:0/1470184504 learned_addr learned my addr 172.21.15.47:0/1470184504 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fe720a00640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c0b3330 0x7fe71c15cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fe721e00640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c161d50 0x7fe71c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fc83ae00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340b17d0 0x7fc83406ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fc83ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fc83ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8340d4270 0x7fc8341f79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fc83ae00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc8340d53b0 con 0x7fc8340b17d0 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fc83ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8340cfe60 con 0x7fc8340d1a00 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.056+0000 7fc83ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc834161f70 con 0x7fc8340d4270 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fc833e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340b17d0 0x7fc83406ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fc833e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340b17d0 0x7fc83406ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:60024/0 (socket says 172.21.15.47:60024) 2024-04-20T10:30:53.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fc833e00640 1 -- 172.21.15.47:0/3730174973 learned_addr learned my addr 172.21.15.47:0/3730174973 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:53.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fe720a00640 1 -- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c06cde0 msgr2=0x7fe71c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fe720a00640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c06cde0 0x7fe71c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fe720a00640 1 -- 172.21.15.47:0/1470184504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c161d50 msgr2=0x7fe71c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fe720a00640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c161d50 0x7fe71c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fe720a00640 1 -- 172.21.15.47:0/1470184504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe71c15b1c0 con 0x7fe71c0b3330 2024-04-20T10:30:53.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fe721400640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c06cde0 0x7fe71c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:53.071 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fe721e00640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c161d50 0x7fe71c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-20T10:30:53.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fc833400640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fc838e00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8340d4270 0x7fc8341f79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.057+0000 7fc833e00640 1 -- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8340d4270 msgr2=0x7fc8341f79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fc833e00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8340d4270 0x7fc8341f79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fc833e00640 1 -- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 msgr2=0x7fc8340cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.072 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fc833e00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fc833e00640 1 -- 172.21.15.47:0/3730174973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83406d490 con 0x7fc8340b17d0 2024-04-20T10:30:53.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fc833400640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:53.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fc838e00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8340d4270 0x7fc8341f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:53.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fe720a00640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c0b3330 0x7fe71c15cf70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0ba2b0 tx=0x7fe70800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fc833e00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340b17d0 0x7fc83406ce40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c430 tx=0x7fc82400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fe719600640 1 -- 172.21.15.47:0/1470184504 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708017070 con 0x7fe71c0b3330 2024-04-20T10:30:53.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fe719600640 1 -- 172.21.15.47:0/1470184504 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe708005ce0 con 0x7fe71c0b3330 2024-04-20T10:30:53.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7f5793400640 1 -- 172.21.15.47:0/4060194223 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7f57940ac5b0 con 0x7f579c161d50 2024-04-20T10:30:53.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.058+0000 7fe719600640 1 -- 172.21.15.47:0/1470184504 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708007310 con 0x7fe71c0b3330 2024-04-20T10:30:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.059+0000 7fc82be00640 1 -- 172.21.15.47:0/3730174973 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824017070 con 0x7fc8340b17d0 2024-04-20T10:30:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.059+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83406d770 con 0x7fc8340b17d0 2024-04-20T10:30:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.059+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc83406dd30 con 0x7fc8340b17d0 2024-04-20T10:30:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.059+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe71c15b4a0 con 0x7fe71c0b3330 2024-04-20T10:30:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.059+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe71c15ba60 con 0x7fe71c0b3330 2024-04-20T10:30:53.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.059+0000 7f7143e00640 1 -- 172.21.15.47:0/3285814080 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f713807ccd0 con 0x7f714c0b3330 2024-04-20T10:30:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.060+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc834068980 con 0x7fc8340b17d0 2024-04-20T10:30:53.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.061+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe71c075b60 con 0x7fe71c0b3330 2024-04-20T10:30:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.063+0000 7fe719600640 1 -- 172.21.15.47:0/1470184504 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe70801a070 con 0x7fe71c0b3330 2024-04-20T10:30:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.064+0000 7fe719600640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.064+0000 7fe719600640 1 -- 172.21.15.47:0/1470184504 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fe70808afb0 con 0x7fe71c0b3330 2024-04-20T10:30:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.065+0000 7fe721400640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.066+0000 7fc82be00640 1 -- 172.21.15.47:0/3730174973 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8240052d0 con 0x7fc8340b17d0 2024-04-20T10:30:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.066+0000 7fc82be00640 1 -- 172.21.15.47:0/3730174973 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc824021060 con 0x7fc8340b17d0 2024-04-20T10:30:53.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.066+0000 7f5793400640 1 -- 172.21.15.47:0/4060194223 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f579404e020 con 0x7f579c161d50 2024-04-20T10:30:53.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.067+0000 7fe721400640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe710009f50 tx=0x7fe710009190 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.078 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.068+0000 7fc82be00640 1 -- 172.21.15.47:0/3730174973 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc82401a070 con 0x7fc8340b17d0 2024-04-20T10:30:53.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.069+0000 7fc82be00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:53.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.069+0000 7fc82be00640 1 -- 172.21.15.47:0/3730174973 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 6101+0+0 (secure 0 0 0) 0x7fc82408b520 con 0x7fc8340b17d0 2024-04-20T10:30:53.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.069+0000 7fc833400640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:53.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.071+0000 7fc82be00640 1 -- 172.21.15.47:0/3730174973 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc824059150 con 0x7fc8340b17d0 2024-04-20T10:30:53.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.071+0000 7fc833400640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc81406c8f0 0x7fc81406edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc820036000 tx=0x7fc82003b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:53.079 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.079+0000 7fe719600640 1 -- 172.21.15.47:0/1470184504 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe708058c70 con 0x7fe71c0b3330 2024-04-20T10:30:53.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.351+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) -- 0x7fc8340ce9a0 con 0x7fc8340b17d0 2024-04-20T10:30:53.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.353+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) -- 0x7f579c0cb2e0 con 0x7f579c161d50 2024-04-20T10:30:53.362 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.362+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) -- 0x7f714c0cfb40 con 0x7f714c0b3330 2024-04-20T10:30:53.382 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.382+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) -- 0x7fe71c0d48d0 con 0x7fe71c0b3330 2024-04-20T10:30:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.458+0000 7fc82be00640 1 -- 172.21.15.47:0/3730174973 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 5392 v34) ==== 95+0+0 (secure 0 0 0) 0x7fc82405d450 con 0x7fc8340b17d0 2024-04-20T10:30:53.458 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 5392 2024-04-20T10:30:53.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc81406c8f0 msgr2=0x7fc81406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc81406c8f0 0x7fc81406edb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fc820036000 tx=0x7fc82003b040 comp rx=0 tx=0).stop 2024-04-20T10:30:53.461 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340b17d0 msgr2=0x7fc83406ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340b17d0 0x7fc83406ce40 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fc82400c430 tx=0x7fc82400c8f0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 shutdown_connections 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8340d4270 0x7fc8341f79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8340d1a00 0x7fc8340cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc81406c8f0 0x7fc81406edb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 --2- 172.21.15.47:0/3730174973 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc8340b17d0 0x7fc83406ce40 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 >> 172.21.15.47:0/3730174973 conn(0x7fc8340b99c0 msgr2=0x7fc8340b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 shutdown_connections 2024-04-20T10:30:53.462 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.461+0000 7fc83ae00640 1 -- 172.21.15.47:0/3730174973 wait complete. 2024-04-20T10:30:53.464 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-20T10:30:53.464+0000 7f06dde00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:30:53.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.467+0000 7f5793400640 1 -- 172.21.15.47:0/4060194223 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6057 v35) ==== 95+0+0 (secure 0 0 0) 0x7f579407c430 con 0x7f579c161d50 2024-04-20T10:30:53.468 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 6057 2024-04-20T10:30:53.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.470+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f577806c970 msgr2=0x7f577806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.470+0000 7f57a0c00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f577806c970 0x7f577806ee30 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5784009aa0 tx=0x7f5784009190 comp rx=0 tx=0).stop 2024-04-20T10:30:53.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.470+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c161d50 msgr2=0x7f579c0cd050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.470 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.470+0000 7f57a0c00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c161d50 0x7f579c0cd050 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f5794009d90 tx=0x7f5794004a40 comp rx=0 tx=0).stop 2024-04-20T10:30:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.471+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 shutdown_connections 2024-04-20T10:30:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.471+0000 7f57a0c00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f577806c970 0x7f577806ee30 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.471+0000 7f57a0c00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f579c161d50 0x7f579c0cd050 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.471+0000 7f57a0c00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f579c0b3330 0x7f579c0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.471+0000 7f57a0c00640 1 --2- 172.21.15.47:0/4060194223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f579c06cde0 0x7f579c0ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.471+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 >> 172.21.15.47:0/4060194223 conn(0x7f579c0d4450 msgr2=0x7f579c0cfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.471+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 shutdown_connections 2024-04-20T10:30:53.472 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.472+0000 7f57a0c00640 1 -- 172.21.15.47:0/4060194223 wait complete. 2024-04-20T10:30:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.477+0000 7f7143e00640 1 -- 172.21.15.47:0/3285814080 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 5398 v36) ==== 95+0+0 (secure 0 0 0) 0x7f713807c5d0 con 0x7f714c0b3330 2024-04-20T10:30:53.477 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 5398 2024-04-20T10:30:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f712c06c940 msgr2=0x7f712c06ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f712c06c940 0x7f712c06ee00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f713c001660 tx=0x7f713c009040 comp rx=0 tx=0).stop 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f714c0b3330 msgr2=0x7f714c0c9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f714c0b3330 0x7f714c0c9d00 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f713802bf90 tx=0x7f71380046c0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 shutdown_connections 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f714c161d50 0x7f714c15d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f712c06c940 0x7f712c06ee00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f714c0b3330 0x7f714c0c9d00 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 --2- 172.21.15.47:0/3285814080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f714c06cde0 0x7f714c15b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 >> 172.21.15.47:0/3285814080 conn(0x7f714c0d4450 msgr2=0x7f714c0d2350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.480+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 shutdown_connections 2024-04-20T10:30:53.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:53.481+0000 7f7151a00640 1 -- 172.21.15.47:0/3285814080 wait complete. 2024-04-20T10:30:53.482 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-20T10:30:53.482 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-20T10:30:53.492 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-20T10:30:53.492 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-20T10:30:53.493 INFO:tasks.ceph.mds.b:Started 2024-04-20T10:30:53.493 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:53.500 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-20T10:30:53.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-20T10:30:53.508 INFO:tasks.ceph.mds.d:Started 2024-04-20T10:30:53.508 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:53.544 INFO:tasks.ceph.mds.a:Started 2024-04-20T10:30:53.544 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:53.647 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-04-20T10:30:53.657 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-04-20T10:30:53.709 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-20T10:30:54.470 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-20T10:30:54.470+0000 7f06dde00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:30:54.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.482+0000 7fe719600640 1 -- 172.21.15.47:0/1470184504 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6051 v37) ==== 95+0+0 (secure 0 0 0) 0x7fe70805cf70 con 0x7fe71c0b3330 2024-04-20T10:30:54.483 INFO:teuthology.orchestra.run.smithi047.stderr:failed mds gid 6051 2024-04-20T10:30:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe6fc06c8f0 msgr2=0x7fe6fc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe710009f50 tx=0x7fe710009190 comp rx=0 tx=0).stop 2024-04-20T10:30:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c0b3330 msgr2=0x7fe71c15cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c0b3330 0x7fe71c15cf70 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fe71c0ba2b0 tx=0x7fe70800c890 comp rx=0 tx=0).stop 2024-04-20T10:30:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 shutdown_connections 2024-04-20T10:30:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe6fc06c8f0 0x7fe6fc06edb0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe71c161d50 0x7fe71c0cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe71c0b3330 0x7fe71c15cf70 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 --2- 172.21.15.47:0/1470184504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe71c06cde0 0x7fe71c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 >> 172.21.15.47:0/1470184504 conn(0x7fe71c0d4450 msgr2=0x7fe71c0b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:54.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.487+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 shutdown_connections 2024-04-20T10:30:54.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.488+0000 7fe722800640 1 -- 172.21.15.47:0/1470184504 wait complete. 2024-04-20T10:30:54.509 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-20T10:30:54.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-20T10:30:54.558 INFO:tasks.ceph.mds.c:Started 2024-04-20T10:30:54.558 DEBUG:teuthology.parallel:result is None 2024-04-20T10:30:54.558 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:54.711 INFO:tasks.ceph.mds.c.smithi047.stdout:starting mds.c at 2024-04-20T10:30:54.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.735+0000 7f209f200640 1 Processor -- start 2024-04-20T10:30:54.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.735+0000 7f209f200640 1 -- start start 2024-04-20T10:30:54.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.736+0000 7f209f200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20a0161d50 0x7f20a00d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:54.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.736+0000 7f209f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a00b3330 0x7f20a00b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.736+0000 7f209f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f20a006c190 con 0x7f20a006c6d0 2024-04-20T10:30:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.736+0000 7f209f200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f20a0162130 con 0x7f20a0161d50 2024-04-20T10:30:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.736+0000 7f209f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20a006c470 con 0x7f20a00b3330 2024-04-20T10:30:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209de00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20a0161d50 0x7f20a00d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209de00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f20a0161d50 0x7f20a00d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33754/0 (socket says 172.21.15.47:33754) 2024-04-20T10:30:54.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209de00640 1 -- 172.21.15.47:0/3774697924 learned_addr learned my addr 172.21.15.47:0/3774697924 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209d400640 1 --2- 172.21.15.47:0/3774697924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a00b3330 0x7f20a00b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209de00640 1 -- 172.21.15.47:0/3774697924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a00b3330 msgr2=0x7f20a00b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209de00640 1 --2- 172.21.15.47:0/3774697924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a00b3330 0x7f20a00b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209de00640 1 -- 172.21.15.47:0/3774697924 >> v1:172.21.15.175:6789/0 conn(0x7f20a006c6d0 legacy=0x7f20a00cfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:54.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.737+0000 7f209de00640 1 -- 172.21.15.47:0/3774697924 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a00d2e00 con 0x7f20a0161d50 2024-04-20T10:30:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.738+0000 7f209de00640 1 --2- 172.21.15.47:0/3774697924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0161d50 0x7f20a00d38e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2090009870 tx=0x7f209002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5de216efd54e2f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209ca00640 1 -- 172.21.15.47:0/3774697924 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2090035020 con 0x7f20a0161d50 2024-04-20T10:30:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209ca00640 1 -- 172.21.15.47:0/3774697924 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2090031640 con 0x7f20a0161d50 2024-04-20T10:30:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209ca00640 1 -- 172.21.15.47:0/3774697924 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209002d040 con 0x7f20a0161d50 2024-04-20T10:30:54.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 -- 172.21.15.47:0/3774697924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0161d50 msgr2=0x7f20a00d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 --2- 172.21.15.47:0/3774697924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0161d50 0x7f20a00d38e0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f2090009870 tx=0x7f209002b8f0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 -- 172.21.15.47:0/3774697924 shutdown_connections 2024-04-20T10:30:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 --2- 172.21.15.47:0/3774697924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a00b3330 0x7f20a00b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 --2- 172.21.15.47:0/3774697924 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a0161d50 0x7f20a00d38e0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 -- 172.21.15.47:0/3774697924 >> 172.21.15.47:0/3774697924 conn(0x7f20a00d4450 msgr2=0x7f20a00d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 -- 172.21.15.47:0/3774697924 shutdown_connections 2024-04-20T10:30:54.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.739+0000 7f209f200640 1 -- 172.21.15.47:0/3774697924 wait complete. 2024-04-20T10:30:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.741+0000 7f209f200640 1 Processor -- start 2024-04-20T10:30:54.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.741+0000 7f209f200640 1 -- start start 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a006c6d0 0x7f20a0152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00b3330 0x7f20a015d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a0161d50 0x7f20a015dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f20a006cb60 con 0x7f20a00b3330 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20a00d33c0 con 0x7f20a006c6d0 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20a006cce0 con 0x7f20a0161d50 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00b3330 0x7f20a015d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209d400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00b3330 0x7f20a015d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33766/0 (socket says 172.21.15.47:33766) 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.742+0000 7f209d400640 1 -- 172.21.15.47:0/4216064230 learned_addr learned my addr 172.21.15.47:0/4216064230 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:54.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209de00640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a006c6d0 0x7f20a0152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209e800640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a0161d50 0x7f20a015dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209d400640 1 -- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a0161d50 msgr2=0x7f20a015dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209d400640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a0161d50 0x7f20a015dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209d400640 1 -- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a006c6d0 msgr2=0x7f20a0152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209d400640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a006c6d0 0x7f20a0152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209d400640 1 -- 172.21.15.47:0/4216064230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20a00cb770 con 0x7f20a00b3330 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209e800640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a0161d50 0x7f20a015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209de00640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a006c6d0 0x7f20a0152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.743+0000 7f209d400640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00b3330 0x7f20a015d800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7f0 tx=0x7f208c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.744+0000 7f2096000640 1 -- 172.21.15.47:0/4216064230 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c015070 con 0x7f20a00b3330 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.744+0000 7f2096000640 1 -- 172.21.15.47:0/4216064230 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f208c004240 con 0x7f20a00b3330 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.744+0000 7f2096000640 1 -- 172.21.15.47:0/4216064230 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c007620 con 0x7f20a00b3330 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.744+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20a00cba50 con 0x7f20a00b3330 2024-04-20T10:30:54.745 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.744+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20a00cc010 con 0x7f20a00b3330 2024-04-20T10:30:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.745+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20a0075b60 con 0x7f20a00b3330 2024-04-20T10:30:54.746 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.746+0000 7f2096000640 1 -- 172.21.15.47:0/4216064230 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f208c0079d0 con 0x7f20a00b3330 2024-04-20T10:30:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.747+0000 7f2096000640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f208006c8f0 0x7f208006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.747+0000 7f2096000640 1 -- 172.21.15.47:0/4216064230 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f208c08c230 con 0x7f20a00b3330 2024-04-20T10:30:54.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.747+0000 7f209de00640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f208006c8f0 0x7f208006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:54.748 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.749+0000 7f209de00640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f208006c8f0 0x7f208006edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2090036000 tx=0x7f2090008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:54.751 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.751+0000 7f2096000640 1 -- 172.21.15.47:0/4216064230 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f208c05a410 con 0x7f20a00b3330 2024-04-20T10:30:54.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.992+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20a015fbf0 con 0x7f20a00b3330 2024-04-20T10:30:54.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.994+0000 7f2096000640 1 -- 172.21.15.47:0/4216064230 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+4371 (secure 0 0 0) 0x7f208c059d10 con 0x7f20a00b3330 2024-04-20T10:30:54.994 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:54.994 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":38,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:54.482281+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"a","rank":0,"incarnation":38,"state":"up:replay","state_seq":1,"addr":"172.21.15.47:6833/566678338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":566678338},{"type":"v1","addr":"172.21.15.47:6833","nonce":566678338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-20T10:30:54.994 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 38 2024-04-20T10:30:54.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f208006c8f0 msgr2=0x7f208006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f208006c8f0 0x7f208006edb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2090036000 tx=0x7f2090008030 comp rx=0 tx=0).stop 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00b3330 msgr2=0x7f20a015d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00b3330 0x7f20a015d800 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f208c00c7f0 tx=0x7f208c00ccb0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 shutdown_connections 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20a0161d50 0x7f20a015dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f208006c8f0 0x7f208006edb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f20a00b3330 0x7f20a015d800 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 --2- 172.21.15.47:0/4216064230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20a006c6d0 0x7f20a0152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 >> 172.21.15.47:0/4216064230 conn(0x7f20a00d4450 msgr2=0x7f20a00677c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 shutdown_connections 2024-04-20T10:30:54.997 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:54.997+0000 7f209f200640 1 -- 172.21.15.47:0/4216064230 wait complete. 2024-04-20T10:30:55.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:30:21.745506+0000', 'modified': '2024-04-20T10:30:54.482281+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 58, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6264': {'gid': 6264, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.47:6833/566678338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 566678338}, {'type': 'v1', 'addr': '172.21.15.47:6833', 'nonce': 566678338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T10:30:55.418 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6264:up:replay 2024-04-20T10:30:56.419 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:56.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.644+0000 7f71f6e00640 1 Processor -- start 2024-04-20T10:30:56.645 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.645+0000 7f71f6e00640 1 -- start start 2024-04-20T10:30:56.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.646+0000 7f71f6e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f71f0159e10 0x7f71f0071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.646+0000 7f71f6e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71f015a1f0 0x7f71f0071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.646+0000 7f71f6e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f71f015cc90 con 0x7f71f006da90 2024-04-20T10:30:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.646+0000 7f71f6e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f71f0072900 con 0x7f71f015a1f0 2024-04-20T10:30:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.646+0000 7f71f6e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f71f015ca40 con 0x7f71f0159e10 2024-04-20T10:30:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71f015a1f0 0x7f71f0071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f71f015a1f0 0x7f71f0071fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33776/0 (socket says 172.21.15.47:33776) 2024-04-20T10:30:56.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5000640 1 -- 172.21.15.47:0/3884834491 learned_addr learned my addr 172.21.15.47:0/3884834491 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5a00640 1 --2- 172.21.15.47:0/3884834491 >> v2:172.21.15.175:3300/0 conn(0x7f71f0159e10 0x7f71f0071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5000640 1 -- 172.21.15.47:0/3884834491 >> v2:172.21.15.175:3300/0 conn(0x7f71f0159e10 msgr2=0x7f71f0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5000640 1 --2- 172.21.15.47:0/3884834491 >> v2:172.21.15.175:3300/0 conn(0x7f71f0159e10 0x7f71f0071a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5000640 1 -- 172.21.15.47:0/3884834491 >> v1:172.21.15.175:6789/0 conn(0x7f71f006da90 legacy=0x7f71f0160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5000640 1 -- 172.21.15.47:0/3884834491 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f015cf70 con 0x7f71f015a1f0 2024-04-20T10:30:56.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.647+0000 7f71f5a00640 1 --2- 172.21.15.47:0/3884834491 >> v2:172.21.15.175:3300/0 conn(0x7f71f0159e10 0x7f71f0071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.648+0000 7f71f5000640 1 --2- 172.21.15.47:0/3884834491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f0071fc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f71f0073f30 tx=0x7f71dc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16646696d9854545 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.649+0000 7f71efe00640 1 -- 172.21.15.47:0/3884834491 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc035020 con 0x7f71f015a1f0 2024-04-20T10:30:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.649+0000 7f71efe00640 1 -- 172.21.15.47:0/3884834491 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71dc031870 con 0x7f71f015a1f0 2024-04-20T10:30:56.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.649+0000 7f71efe00640 1 -- 172.21.15.47:0/3884834491 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71dc031c40 con 0x7f71f015a1f0 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.649+0000 7f71f6e00640 1 -- 172.21.15.47:0/3884834491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 msgr2=0x7f71f0071fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.649+0000 7f71f6e00640 1 --2- 172.21.15.47:0/3884834491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f0071fc0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f71f0073f30 tx=0x7f71dc02b910 comp rx=0 tx=0).stop 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.650+0000 7f71f6e00640 1 -- 172.21.15.47:0/3884834491 shutdown_connections 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.650+0000 7f71f6e00640 1 --2- 172.21.15.47:0/3884834491 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f0071fc0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.650+0000 7f71f6e00640 1 --2- 172.21.15.47:0/3884834491 >> v2:172.21.15.175:3300/0 conn(0x7f71f0159e10 0x7f71f0071a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.650+0000 7f71f6e00640 1 -- 172.21.15.47:0/3884834491 >> 172.21.15.47:0/3884834491 conn(0x7f71f00d0040 msgr2=0x7f71f00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.650+0000 7f71f6e00640 1 -- 172.21.15.47:0/3884834491 shutdown_connections 2024-04-20T10:30:56.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.650+0000 7f71f6e00640 1 -- 172.21.15.47:0/3884834491 wait complete. 2024-04-20T10:30:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.651+0000 7f71f6e00640 1 Processor -- start 2024-04-20T10:30:56.651 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.652+0000 7f71f6e00640 1 -- start start 2024-04-20T10:30:56.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.652+0000 7f71f6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71f006da90 0x7f71f009c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.652+0000 7f71f6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f0159e10 0x7f71f006bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f009a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f71f015d810 con 0x7f71f015a1f0 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71f015d530 con 0x7f71f0159e10 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71f006b900 con 0x7f71f006da90 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f009a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f009a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33786/0 (socket says 172.21.15.47:33786) 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 -- 172.21.15.47:0/2103711368 learned_addr learned my addr 172.21.15.47:0/2103711368 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f5a00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71f006da90 0x7f71f009c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 -- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71f006da90 msgr2=0x7f71f009c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71f006da90 0x7f71f009c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 -- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f0159e10 msgr2=0x7f71f006bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f5000640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f0159e10 0x7f71f006bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f0159e10 0x7f71f006bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f6400640 1 -- 172.21.15.47:0/2103711368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71f01575e0 con 0x7f71f015a1f0 2024-04-20T10:30:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.653+0000 7f71f5a00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71f006da90 0x7f71f009c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:56.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.654+0000 7f71f6400640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f009a9f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f71e800c780 tx=0x7f71e800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.654+0000 7f71ed600640 1 -- 172.21.15.47:0/2103711368 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e8004b60 con 0x7f71f015a1f0 2024-04-20T10:30:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.654+0000 7f71ed600640 1 -- 172.21.15.47:0/2103711368 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71e8004d00 con 0x7f71f015a1f0 2024-04-20T10:30:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.654+0000 7f71ed600640 1 -- 172.21.15.47:0/2103711368 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71e800aa90 con 0x7f71f015a1f0 2024-04-20T10:30:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.654+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71f01578c0 con 0x7f71f015a1f0 2024-04-20T10:30:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.654+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71f0157e80 con 0x7f71f015a1f0 2024-04-20T10:30:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.655+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71f0073d80 con 0x7f71f015a1f0 2024-04-20T10:30:56.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.656+0000 7f71ed600640 1 -- 172.21.15.47:0/2103711368 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f71e800ac30 con 0x7f71f015a1f0 2024-04-20T10:30:56.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.657+0000 7f71ed600640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f71d006c8f0 0x7f71d006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:56.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.657+0000 7f71ed600640 1 -- 172.21.15.47:0/2103711368 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f71e808c460 con 0x7f71f015a1f0 2024-04-20T10:30:56.658 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.658+0000 7f71f5a00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f71d006c8f0 0x7f71d006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:56.659 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.659+0000 7f71f5a00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f71d006c8f0 0x7f71d006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f71e4006fd0 tx=0x7f71e4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:56.662 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.663+0000 7f71ed600640 1 -- 172.21.15.47:0/2103711368 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f71f0073d80 con 0x7f71f015a1f0 2024-04-20T10:30:56.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.904+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f71f0075b60 con 0x7f71f015a1f0 2024-04-20T10:30:56.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.905+0000 7f71ed600640 1 -- 172.21.15.47:0/2103711368 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+5196 (secure 0 0 0) 0x7f71e805a520 con 0x7f71f015a1f0 2024-04-20T10:30:56.905 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:56.906 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":40,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3774977814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3774977814},{"type":"v1","addr":"172.21.15.47:6835","nonce":3774977814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:55.500848+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"a","rank":0,"incarnation":38,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.47:6833/566678338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":566678338},{"type":"v1","addr":"172.21.15.47:6833","nonce":566678338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":3}]} 2024-04-20T10:30:56.906 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 40 2024-04-20T10:30:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f71d006c8f0 msgr2=0x7f71d006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f71d006c8f0 0x7f71d006edb0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f71e4006fd0 tx=0x7f71e4008040 comp rx=0 tx=0).stop 2024-04-20T10:30:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 msgr2=0x7f71f009a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f009a9f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f71e800c780 tx=0x7f71e800cc40 comp rx=0 tx=0).stop 2024-04-20T10:30:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 shutdown_connections 2024-04-20T10:30:56.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f71d006c8f0 0x7f71d006edb0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f71f015a1f0 0x7f71f009a9f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71f0159e10 0x7f71f006bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 --2- 172.21.15.47:0/2103711368 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71f006da90 0x7f71f009c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 >> 172.21.15.47:0/2103711368 conn(0x7f71f00d0040 msgr2=0x7f71f0073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 shutdown_connections 2024-04-20T10:30:56.909 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:56.908+0000 7f71f6e00640 1 -- 172.21.15.47:0/2103711368 wait complete. 2024-04-20T10:30:57.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:30:21.745506+0000', 'modified': '2024-04-20T10:30:55.500848+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 58, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6264': {'gid': 6264, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.47:6833/566678338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 566678338}, {'type': 'v1', 'addr': '172.21.15.47:6833', 'nonce': 566678338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-04-20T10:30:57.332 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6264:up:rejoin 2024-04-20T10:30:58.333 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:30:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.557+0000 7f8aa6000640 1 Processor -- start 2024-04-20T10:30:58.557 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.557+0000 7f8aa6000640 1 -- start start 2024-04-20T10:30:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.559+0000 7f8aa6000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8aa0161d50 0x7f8aa00d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.559+0000 7f8aa6000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa006c190 con 0x7f8aa00b3330 2024-04-20T10:30:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.559+0000 7f8aa6000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8aa006c470 con 0x7f8aa006cde0 2024-04-20T10:30:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.559+0000 7f8aa6000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8aa0162130 con 0x7f8aa0161d50 2024-04-20T10:30:58.559 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.559+0000 7f8a97e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8aa00b3330 0x7f8aa006c6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57880/0 (socket says 172.21.15.47:57880) 2024-04-20T10:30:58.560 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.559+0000 7f8a97e00640 1 -- 172.21.15.47:0/655335224 learned_addr learned my addr 172.21.15.47:0/655335224 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:58.561 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8aa4c00640 1 --2- 172.21.15.47:0/655335224 >> v2:172.21.15.175:3300/0 conn(0x7f8aa0161d50 0x7f8aa00d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1268299124 0 0) 0x7f8aa006c190 con 0x7f8aa00b3330 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8a80003520 con 0x7f8aa00b3330 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1326927593 0 0) 0x7f8a80003520 con 0x7f8aa00b3330 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 >> v2:172.21.15.175:3300/0 conn(0x7f8aa0161d50 msgr2=0x7f8aa00d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/655335224 >> v2:172.21.15.175:3300/0 conn(0x7f8aa0161d50 0x7f8aa00d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 >> v1:172.21.15.175:6789/0 conn(0x7f8aa006cde0 legacy=0x7f8aa00b7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa00d2e00 con 0x7f8aa00b3330 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.560+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f8a90001670 con 0x7f8aa00b3330 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 >> v1:172.21.15.47:6789/0 conn(0x7f8aa00b3330 legacy=0x7f8aa006c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a80003a00 0x7f8a80003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a9fe00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a800051c0 0x7f8a8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8000c090 con 0x7f8a80004800 2024-04-20T10:30:58.562 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8000c240 con 0x7f8a800051c0 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8000c440 con 0x7f8a80003a00 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8a97e00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8aa5600640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a800051c0 0x7f8a8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.561+0000 7f8aa4c00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a80003a00 0x7f8a80003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.562+0000 7f8a97e00640 1 -- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a80003a00 msgr2=0x7f8a80003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.562+0000 7f8a97e00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8a80003a00 0x7f8a80003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.562+0000 7f8a97e00640 1 -- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a800051c0 msgr2=0x7f8a8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.562+0000 7f8a97e00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8a800051c0 0x7f8a8000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.562+0000 7f8a97e00640 1 -- 172.21.15.47:0/655335224 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a80003520 con 0x7f8a80004800 2024-04-20T10:30:58.563 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.562+0000 7f8aa5600640 1 -- 172.21.15.47:0/655335224 reap_dead start 2024-04-20T10:30:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.563+0000 7f8a97e00640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c0e0 tx=0x7f8a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73ca4e877eabc8af server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.563+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90007c20 con 0x7f8a80004800 2024-04-20T10:30:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.563+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a90030070 con 0x7f8a80004800 2024-04-20T10:30:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.563+0000 7f8a9fe00640 1 -- 172.21.15.47:0/655335224 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90005d70 con 0x7f8a80004800 2024-04-20T10:30:58.564 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.564+0000 7f8aa6000640 1 -- 172.21.15.47:0/655335224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 msgr2=0x7f8a80004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.564+0000 7f8aa6000640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f8a9000c0e0 tx=0x7f8a80005830 comp rx=0 tx=0).stop 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.564+0000 7f8aa6000640 1 -- 172.21.15.47:0/655335224 shutdown_connections 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.564+0000 7f8aa6000640 1 --2- 172.21.15.47:0/655335224 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.564+0000 7f8aa6000640 1 -- 172.21.15.47:0/655335224 >> 172.21.15.47:0/655335224 conn(0x7f8aa00d4450 msgr2=0x7f8aa00d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.564+0000 7f8aa6000640 1 -- 172.21.15.47:0/655335224 shutdown_connections 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.564+0000 7f8aa6000640 1 -- 172.21.15.47:0/655335224 wait complete. 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.566+0000 7f8aa6000640 1 Processor -- start 2024-04-20T10:30:58.565 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.566+0000 7f8aa6000640 1 -- start start 2024-04-20T10:30:58.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8aa00c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aa01f4040 0x7f8aa00b8d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aa00ca1c0 0x7f8aa00ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa6000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa00d38e0 con 0x7f8a80004800 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa6000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa00d33c0 con 0x7f8aa00ca1c0 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa6000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8aa00d3a60 con 0x7f8aa01f4040 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8aa00c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa4c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8aa00c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33800/0 (socket says 172.21.15.47:33800) 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8aa4c00640 1 -- 172.21.15.47:0/3360948552 learned_addr learned my addr 172.21.15.47:0/3360948552 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.567+0000 7f8a97e00640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aa01f4040 0x7f8aa00b8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa5600640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aa00ca1c0 0x7f8aa00ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa4c00640 1 -- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aa01f4040 msgr2=0x7f8aa00b8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa4c00640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aa01f4040 0x7f8aa00b8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa4c00640 1 -- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aa00ca1c0 msgr2=0x7f8aa00ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa4c00640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aa00ca1c0 0x7f8aa00ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa4c00640 1 -- 172.21.15.47:0/3360948552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa00c8790 con 0x7f8a80004800 2024-04-20T10:30:58.568 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8a97e00640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aa01f4040 0x7f8aa00b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa5600640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aa00ca1c0 0x7f8aa00ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa4c00640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8aa00c7f60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8a88011540 tx=0x7f8a88011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8a9d600640 1 -- 172.21.15.47:0/3360948552 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88015870 con 0x7f8a80004800 2024-04-20T10:30:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8a9d600640 1 -- 172.21.15.47:0/3360948552 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8a88015a10 con 0x7f8a80004800 2024-04-20T10:30:58.569 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8a9d600640 1 -- 172.21.15.47:0/3360948552 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88015dc0 con 0x7f8a80004800 2024-04-20T10:30:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.568+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa00c8a70 con 0x7f8a80004800 2024-04-20T10:30:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.569+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa00c9030 con 0x7f8a80004800 2024-04-20T10:30:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.570+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8aa0075b60 con 0x7f8a80004800 2024-04-20T10:30:58.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.571+0000 7f8a9d600640 1 -- 172.21.15.47:0/3360948552 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8a8800f460 con 0x7f8a80004800 2024-04-20T10:30:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.572+0000 7f8a9d600640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:58.572 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.572+0000 7f8a9d600640 1 -- 172.21.15.47:0/3360948552 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f8a8808c350 con 0x7f8a80004800 2024-04-20T10:30:58.573 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.573+0000 7f8a97e00640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:58.574 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.574+0000 7f8a97e00640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8a90034000 tx=0x7f8a90039040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:58.578 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.578+0000 7f8a9d600640 1 -- 172.21.15.47:0/3360948552 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8a88019030 con 0x7f8a80004800 2024-04-20T10:30:58.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.821+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8aa00cb2e0 con 0x7f8a80004800 2024-04-20T10:30:58.821 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.821+0000 7f8a9d600640 1 -- 172.21.15.47:0/3360948552 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5203 (secure 0 0 0) 0x7f8a8805a4a0 con 0x7f8a80004800 2024-04-20T10:30:58.822 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:58.822 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3774977814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3774977814},{"type":"v1","addr":"172.21.15.47:6835","nonce":3774977814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:57.508717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6833/566678338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":566678338},{"type":"v1","addr":"172.21.15.47:6833","nonce":566678338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6264,"qdb_cluster":[6264]},"id":3}]} 2024-04-20T10:30:58.822 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-04-20T10:30:58.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 msgr2=0x7f8a7c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f8a90034000 tx=0x7f8a90039040 comp rx=0 tx=0).stop 2024-04-20T10:30:58.824 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 msgr2=0x7f8aa00c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8aa00c7f60 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f8a88011540 tx=0x7f8a88011a00 comp rx=0 tx=0).stop 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 shutdown_connections 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8aa00ca1c0 0x7f8aa00ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8aa01f4040 0x7f8aa00b8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8a7c06c8f0 0x7f8a7c06edb0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 --2- 172.21.15.47:0/3360948552 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8a80004800 0x7f8aa00c7f60 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 >> 172.21.15.47:0/3360948552 conn(0x7f8aa00d4450 msgr2=0x7f8aa015f200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 shutdown_connections 2024-04-20T10:30:58.825 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:58.824+0000 7f8aa6000640 1 -- 172.21.15.47:0/3360948552 wait complete. 2024-04-20T10:30:58.869 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:58.876 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:30:59.246 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 41, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-20T10:30:21.745506+0000', 'modified': '2024-04-20T10:30:57.508717+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 58, 'snap_rstat': True, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6264}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6264': {'gid': 6264, 'name': 'a', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.47:6833/566678338', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.47:6832', 'nonce': 566678338}, {'type': 'v1', 'addr': '172.21.15.47:6833', 'nonce': 566678338}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6264, 'qdb_cluster': [6264]} 2024-04-20T10:30:59.246 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-20T10:30:59.246 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-20T10:30:59.468 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.468+0000 7f1d5ac00640 1 Processor -- start 2024-04-20T10:30:59.469 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.469+0000 7f1d5ac00640 1 -- start start 2024-04-20T10:30:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.470+0000 7f1d5ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 0x7f1d540adf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d5ac00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d54071a80 con 0x7f1d540b2f70 2024-04-20T10:30:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d5ac00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1d54071cf0 con 0x7f1d5406b900 2024-04-20T10:30:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d5ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d54071fd0 con 0x7f1d540b4320 2024-04-20T10:30:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d59800640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1d540b2f70 0x7f1d5415f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57886/0 (socket says 172.21.15.47:57886) 2024-04-20T10:30:59.471 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d59800640 1 -- 172.21.15.47:0/3053248745 learned_addr learned my addr 172.21.15.47:0/3053248745 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d58e00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 0x7f1d540adf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3275053641 0 0) 0x7f1d54071a80 con 0x7f1d540b2f70 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.471+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1d3c003520 con 0x7f1d540b2f70 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3389377251 0 0) 0x7f1d3c003520 con 0x7f1d540b2f70 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 msgr2=0x7f1d540adf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 0x7f1d540adf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 >> v1:172.21.15.175:6789/0 conn(0x7f1d5406b900 legacy=0x7f1d5406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.473 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d540739c0 con 0x7f1d540b2f70 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f1d48001670 con 0x7f1d540b2f70 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 >> v1:172.21.15.47:6789/0 conn(0x7f1d540b2f70 legacy=0x7f1d5415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d3c0039e0 0x7f1d3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 0x7f1d3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1d54071fd0 con 0x7f1d3c004d40 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3c00bbd0 con 0x7f1d3c0039e0 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d3c00bdd0 con 0x7f1d540b4320 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.472+0000 7f1d5a200640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d59800640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d3c0039e0 0x7f1d3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d58e00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 0x7f1d3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d5a200640 1 -- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 msgr2=0x7f1d3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d5a200640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 0x7f1d3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d5a200640 1 -- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d3c0039e0 msgr2=0x7f1d3c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.474 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d5a200640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d3c0039e0 0x7f1d3c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d5a200640 1 -- 172.21.15.47:0/3053248745 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3c003520 con 0x7f1d3c004d40 2024-04-20T10:30:59.475 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.473+0000 7f1d58e00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d540b4320 0x7f1d3c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.474+0000 7f1d5a200640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d3c0051c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c001e50 tx=0x7f1d3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca98658e8c2bc8e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.474+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c001a70 con 0x7f1d3c004d40 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.474+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d4c003b70 con 0x7f1d3c004d40 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.474+0000 7f1d53e00640 1 -- 172.21.15.47:0/3053248745 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c0050d0 con 0x7f1d3c004d40 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.475+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3053248745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 msgr2=0x7f1d3c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.475+0000 7f1d5ac00640 1 --2- 172.21.15.47:0/3053248745 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d3c0051c0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c001e50 tx=0x7f1d3c005830 comp rx=0 tx=0).stop 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.475+0000 7f1d5a200640 1 -- 172.21.15.47:0/3053248745 reap_dead start 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.475+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3053248745 shutdown_connections 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.475+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3053248745 >> 172.21.15.47:0/3053248745 conn(0x7f1d540d0040 msgr2=0x7f1d540d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.475+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3053248745 shutdown_connections 2024-04-20T10:30:59.476 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.475+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3053248745 wait complete. 2024-04-20T10:30:59.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.477+0000 7f1d5ac00640 1 Processor -- start 2024-04-20T10:30:59.477 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.477+0000 7f1d5ac00640 1 -- start start 2024-04-20T10:30:59.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.478+0000 7f1d5ac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d54146670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.478+0000 7f1d5ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d54146e90 0x7f1d54158bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.478+0000 7f1d5ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d54159110 0x7f1d541562a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.478 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.478+0000 7f1d5ac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5415fe40 con 0x7f1d3c004d40 2024-04-20T10:30:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.478+0000 7f1d5ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5415fcc0 con 0x7f1d54159110 2024-04-20T10:30:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.478+0000 7f1d5ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d540adf60 con 0x7f1d54146e90 2024-04-20T10:30:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.478+0000 7f1d59800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d54146670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d54146670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33812/0 (socket says 172.21.15.47:33812) 2024-04-20T10:30:59.479 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 -- 172.21.15.47:0/3093428468 learned_addr learned my addr 172.21.15.47:0/3093428468 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d58e00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d54146e90 0x7f1d54158bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d5a200640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d54159110 0x7f1d541562a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 -- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d54146e90 msgr2=0x7f1d54158bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d54146e90 0x7f1d54158bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 -- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d54159110 msgr2=0x7f1d541562a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.480 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d54159110 0x7f1d541562a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 -- 172.21.15.47:0/3093428468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d54146bb0 con 0x7f1d3c004d40 2024-04-20T10:30:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.479+0000 7f1d59800640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d54146670 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e760 tx=0x7f1d4800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.480+0000 7f1d51600640 1 -- 172.21.15.47:0/3093428468 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4800c8d0 con 0x7f1d3c004d40 2024-04-20T10:30:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.480+0000 7f1d51600640 1 -- 172.21.15.47:0/3093428468 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d48018070 con 0x7f1d3c004d40 2024-04-20T10:30:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.480+0000 7f1d51600640 1 -- 172.21.15.47:0/3093428468 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4800ccd0 con 0x7f1d3c004d40 2024-04-20T10:30:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.480+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d54154c70 con 0x7f1d3c004d40 2024-04-20T10:30:59.481 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.480+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d541580f0 con 0x7f1d3c004d40 2024-04-20T10:30:59.482 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.482+0000 7f1d51600640 1 -- 172.21.15.47:0/3093428468 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1d48013420 con 0x7f1d3c004d40 2024-04-20T10:30:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.483+0000 7f1d51600640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d3806c920 0x7f1d3806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.483 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.483+0000 7f1d51600640 1 -- 172.21.15.47:0/3093428468 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f1d4808ba80 con 0x7f1d3c004d40 2024-04-20T10:30:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.484+0000 7f1d5ef81740 1 Processor -- start 2024-04-20T10:30:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.484+0000 7f1d58e00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d3806c920 0x7f1d3806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.484 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.484+0000 7f1d5ef81740 1 -- start start 2024-04-20T10:30:59.485 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d219ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d21a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d214b90 0x55de7d244980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55de7d1809d0 con 0x55de7d262670 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55de7d127270 con 0x55de7d214b90 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55de7d17c990 con 0x55de7d2186f0 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d219ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d219ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33824/0 (socket says 172.21.15.47:33824) 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 -- 172.21.15.47:0/4137934777 learned_addr learned my addr 172.21.15.47:0/4137934777 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d58e00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d3806c920 0x7f1d3806ede0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1d54153ee0 tx=0x7f1d44008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 -- 172.21.15.47:0/4137934777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 msgr2=0x55de7d21a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 --2- 172.21.15.47:0/4137934777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d21a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 -- 172.21.15.47:0/4137934777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d214b90 msgr2=0x55de7d244980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 --2- 172.21.15.47:0/4137934777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d214b90 0x55de7d244980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.485+0000 7f1d59800640 1 -- 172.21.15.47:0/4137934777 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7ce932d0 con 0x55de7d262670 2024-04-20T10:30:59.487 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.486+0000 7f1d59800640 1 --2- 172.21.15.47:0/4137934777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d219ca0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1d48055930 tx=0x7f1d48058ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0ca22db2a6e752 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.486+0000 7f1d42000640 1 -- 172.21.15.47:0/4137934777 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4805d880 con 0x55de7d262670 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.486+0000 7f1d42000640 1 -- 172.21.15.47:0/4137934777 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d480620b0 con 0x55de7d262670 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.486+0000 7f1d42000640 1 -- 172.21.15.47:0/4137934777 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48055b40 con 0x55de7d262670 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 -- 172.21.15.47:0/4137934777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 msgr2=0x55de7d219ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/4137934777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d219ca0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1d48055930 tx=0x7f1d48058ef0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 -- 172.21.15.47:0/4137934777 shutdown_connections 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/4137934777 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d214b90 0x55de7d244980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/4137934777 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d21a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/4137934777 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d219ca0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 -- 172.21.15.47:0/4137934777 >> 172.21.15.47:0/4137934777 conn(0x55de7d216e00 msgr2=0x55de7d25f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 -- 172.21.15.47:0/4137934777 shutdown_connections 2024-04-20T10:30:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.487+0000 7f1d5ef81740 1 -- 172.21.15.47:0/4137934777 wait complete. 2024-04-20T10:30:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.488+0000 7f1d5ef81740 1 -- start start 2024-04-20T10:30:59.489 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.488+0000 7f1d5ef81740 1 Processor -- start 2024-04-20T10:30:59.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d2186f0 0x55de7d23b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d262670 0x55de7d23c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2442e0 0x55de7d23cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55de7d218c00 con 0x55de7d2186f0 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55de7d13a5e0 con 0x55de7d262670 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55de7d218ee0 con 0x55de7d2442e0 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d5ef81740 10 client.0 _ll_register_callbacks cb 0x55de7ce92ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d59800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d2186f0 0x55de7d23b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d59800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d2186f0 0x55de7d23b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33826/0 (socket says 172.21.15.47:33826) 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d59800640 1 -- 172.21.15.47:0/756505641 learned_addr learned my addr 172.21.15.47:0/756505641 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d59800640 1 -- 172.21.15.47:0/756505641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2442e0 msgr2=0x55de7d23cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.489+0000 7f1d59800640 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2442e0 0x55de7d23cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d59800640 1 -- 172.21.15.47:0/756505641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d262670 msgr2=0x55de7d23c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d59800640 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d262670 0x55de7d23c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d59800640 1 -- 172.21.15.47:0/756505641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de7cf5d6b0 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d59800640 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d2186f0 0x55de7d23b080 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1d48059690 tx=0x7f1d48001b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1250839efea0bd66 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d36a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d36a00640 1 -- 172.21.15.47:0/756505641 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d48076060 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d36a00640 1 -- 172.21.15.47:0/756505641 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f1d48061c20 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d480620b0 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d36a00640 1 -- 172.21.15.47:0/756505641 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1d48089070 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d5ef81740 10 client.6300 fetch_fsmap learned FSMap version 41 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.490+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x55de7d265520 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d36a00640 1 -- 172.21.15.47:0/756505641 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d480620b0 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d36a00640 1 -- 172.21.15.47:0/756505641 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) ==== 3495+0+0 (secure 0 0 0) 0x7f1d48058920 con 0x55de7d2186f0 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d5ef81740 10 client.6300 fetch_fsmap finished waiting for FSMap version 41 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d5ef81740 10 client.6300 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:30:59.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d5ef81740 20 client.6300 populate_metadata read hostname 'smithi047' 2024-04-20T10:30:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d265ea0 0x55de7d2645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d5ef81740 4 client.6300 mds_command: new command op to 6264 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-20T10:30:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 --> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de7d18e3f0 con 0x55de7d265ea0 2024-04-20T10:30:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.491+0000 7f1d58e00640 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d265ea0 0x55de7d2645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.492+0000 7f1d58e00640 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d265ea0 0x55de7d2645e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91bdfb56d88a8d30 server_cookie=6b549c53d275a0f9 in_seq=0 out_seq=0 2024-04-20T10:30:59.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.492+0000 7f1d36a00640 10 client.6300 ms_handle_connect on v2:172.21.15.47:6832/566678338 2024-04-20T10:30:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d36a00640 1 -- 172.21.15.47:0/756505641 <== mds.0 v2:172.21.15.47:6832/566678338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de7d18e3f0 con 0x55de7d265ea0 2024-04-20T10:30:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d36a00640 10 client.6300 handle_command_reply: tid=0 2024-04-20T10:30:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 client.6300 shutdown 2024-04-20T10:30:59.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d2186f0 msgr2=0x55de7d23b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d2186f0 0x55de7d23b080 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f1d48059690 tx=0x7f1d48001b80 comp rx=0 tx=0).stop 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 shutdown_connections 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d265ea0 0x55de7d2645e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2442e0 0x55de7d23cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d262670 0x55de7d23c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/756505641 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d2186f0 0x55de7d23b080 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 >> 172.21.15.47:0/756505641 conn(0x55de7d214b90 msgr2=0x55de7d25f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d36a00640 0 client.6300 ms_handle_reset on v2:172.21.15.47:6832/566678338 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 shutdown_connections 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 1 -- 172.21.15.47:0/756505641 wait complete. 2024-04-20T10:30:59.496 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.493+0000 7f1d5ef81740 20 client.6300 trim_cache size 0 max 16384 2024-04-20T10:30:59.496 INFO:tasks.ceph.mds.a.smithi047.stderr:2024-04-20T10:30:59.495+0000 7f18b6600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:30:59.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.513+0000 7f1d5ef81740 1 Processor -- start 2024-04-20T10:30:59.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.514+0000 7f1d5ef81740 1 -- start start 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.514+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d23a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.514+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d2645e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 0x55de7d264b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1d54200a10 con 0x55de7d262670 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d541ffda0 con 0x55de7d23a800 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d542003c0 con 0x55de7d2186f0 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d2645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d2645e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33836/0 (socket says 172.21.15.47:33836) 2024-04-20T10:30:59.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 -- 172.21.15.47:0/809081191 learned_addr learned my addr 172.21.15.47:0/809081191 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:59.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 -- 172.21.15.47:0/809081191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 msgr2=0x55de7d23a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 --2- 172.21.15.47:0/809081191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d23a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 -- 172.21.15.47:0/809081191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 msgr2=0x55de7d264b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 --2- 172.21.15.47:0/809081191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 0x55de7d264b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.515+0000 7f1d58e00640 1 -- 172.21.15.47:0/809081191 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de7d243130 con 0x55de7d262670 2024-04-20T10:30:59.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.516+0000 7f1d58e00640 1 --2- 172.21.15.47:0/809081191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d2645e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400c200 tx=0x7f1d44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7fccb296b0d39f server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.516+0000 7f1d41600640 1 -- 172.21.15.47:0/809081191 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4400f040 con 0x55de7d262670 2024-04-20T10:30:59.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.516+0000 7f1d41600640 1 -- 172.21.15.47:0/809081191 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d44029030 con 0x55de7d262670 2024-04-20T10:30:59.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.516+0000 7f1d41600640 1 -- 172.21.15.47:0/809081191 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d44016d40 con 0x55de7d262670 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 -- 172.21.15.47:0/809081191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 msgr2=0x55de7d2645e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/809081191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d2645e0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400c200 tx=0x7f1d44004730 comp rx=0 tx=0).stop 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 -- 172.21.15.47:0/809081191 shutdown_connections 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/809081191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 0x55de7d264b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/809081191 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d2645e0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/809081191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d23a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 -- 172.21.15.47:0/809081191 >> 172.21.15.47:0/809081191 conn(0x55de7d214b90 msgr2=0x55de7d261dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 -- 172.21.15.47:0/809081191 shutdown_connections 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.517+0000 7f1d5ef81740 1 -- 172.21.15.47:0/809081191 wait complete. 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.518+0000 7f1d5ef81740 1 -- start start 2024-04-20T10:30:59.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.518+0000 7f1d5ef81740 1 Processor -- start 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d2403b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 0x55de7d2645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5ef81740 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d264af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x55de7d265d20 con 0x55de7d262670 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55de7d218c00 con 0x55de7d23a800 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5ef81740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55de7d264050 con 0x55de7d2186f0 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5ef81740 10 client.0 _ll_register_callbacks cb 0x7f1d48059320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d264af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5a200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d264af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33838/0 (socket says 172.21.15.47:33838) 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5a200640 1 -- 172.21.15.47:0/921317110 learned_addr learned my addr 172.21.15.47:0/921317110 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d59800640 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d2403b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5a200640 1 -- 172.21.15.47:0/921317110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 msgr2=0x55de7d2403b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5a200640 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d2403b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5a200640 1 -- 172.21.15.47:0/921317110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 msgr2=0x55de7d2645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.519+0000 7f1d5a200640 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 0x55de7d2645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d5a200640 1 -- 172.21.15.47:0/921317110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de7d23d3a0 con 0x55de7d262670 2024-04-20T10:30:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d5a200640 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d264af0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c02c060 tx=0x7f1d4c006f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb52b4a5185c07be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d36a00640 10 client.0 ms_handle_connect on v2:172.21.15.47:3300/0 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d36a00640 1 -- 172.21.15.47:0/921317110 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c003b80 con 0x55de7d262670 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d36a00640 1 -- 172.21.15.47:0/921317110 <== mon.0 v2:172.21.15.47:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f1d4c003d20 con 0x55de7d262670 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1d44029030 con 0x55de7d262670 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d36a00640 1 -- 172.21.15.47:0/921317110 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f1d4c04b050 con 0x55de7d262670 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.520+0000 7f1d5ef81740 10 client.6306 fetch_fsmap learned FSMap version 41 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=2+}) -- 0x55de7d2408f0 con 0x55de7d262670 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d36a00640 1 -- 172.21.15.47:0/921317110 <== mon.0 v2:172.21.15.47:3300/0 4 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1d44029030 con 0x55de7d262670 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d36a00640 1 -- 172.21.15.47:0/921317110 <== mon.0 v2:172.21.15.47:3300/0 5 ==== fsmap(e 41) ==== 3495+0+0 (secure 0 0 0) 0x7f1d4c04e020 con 0x55de7d262670 2024-04-20T10:30:59.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d5ef81740 10 client.6306 fetch_fsmap finished waiting for FSMap version 41 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d5ef81740 10 client.6306 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d5ef81740 20 client.6306 populate_metadata read hostname 'smithi047' 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d255310 0x55de7d2556f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d5ef81740 4 client.6306 mds_command: new command op to 6264 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 --> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] -- command(tid 0: {"prefix": "status"}) -- 0x55de7d188260 con 0x55de7d255310 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.521+0000 7f1d59800640 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d255310 0x55de7d2556f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d59800640 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d255310 0x55de7d2556f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34433e7e846b84f9 server_cookie=6655c60af9249165 in_seq=0 out_seq=0 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d36a00640 10 client.6306 ms_handle_connect on v2:172.21.15.47:6832/566678338 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d36a00640 1 -- 172.21.15.47:0/921317110 <== mds.0 v2:172.21.15.47:6832/566678338 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55de7d188260 con 0x55de7d255310 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d36a00640 10 client.6306 handle_command_reply: tid=0 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 client.6306 shutdown 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 msgr2=0x55de7d264af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d264af0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c02c060 tx=0x7f1d4c006f80 comp rx=0 tx=0).stop 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 shutdown_connections 2024-04-20T10:30:59.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:6832/566678338,v1:172.21.15.47:6833/566678338] conn(0x55de7d255310 0x55de7d2556f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x55de7d262670 0x55de7d264af0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55de7d23a800 0x55de7d2645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 --2- 172.21.15.47:0/921317110 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55de7d2186f0 0x55de7d2403b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.522+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 >> 172.21.15.47:0/921317110 conn(0x55de7d214b90 msgr2=0x55de7d261dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.523+0000 7f1d36a00640 0 client.6306 ms_handle_reset on v2:172.21.15.47:6832/566678338 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.523+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 shutdown_connections 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.523+0000 7f1d5ef81740 1 -- 172.21.15.47:0/921317110 wait complete. 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.523+0000 7f1d5ef81740 20 client.6306 trim_cache size 0 max 16384 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "cluster_fsid": "775946e6-93c9-4843-aaf4-db758801eec1", 2024-04-20T10:30:59.525 INFO:teuthology.orchestra.run.smithi047.stdout: "whoami": 0, 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "id": 6264, 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "want_state": "up:active", 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "state": "up:active", 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "fs_name": "cephfs", 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "rank_uptime": 5.0306523189999997, 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "mdsmap_epoch": 41, 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch": 58, 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "osdmap_epoch_barrier": 58, 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout: "uptime": 5.8092876389999999 2024-04-20T10:30:59.526 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d3806c920 msgr2=0x7f1d3806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d3806c920 0x7f1d3806ede0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1d54153ee0 tx=0x7f1d44008040 comp rx=0 tx=0).stop 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 msgr2=0x7f1d54146670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d54146670 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800e760 tx=0x7f1d4800ec20 comp rx=0 tx=0).stop 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 shutdown_connections 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d54159110 0x7f1d541562a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d54146e90 0x7f1d54158bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1d3806c920 0x7f1d3806ede0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 --2- 172.21.15.47:0/3093428468 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1d3c004d40 0x7f1d54146670 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 >> 172.21.15.47:0/3093428468 conn(0x7f1d540d0040 msgr2=0x7f1d5415c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 shutdown_connections 2024-04-20T10:30:59.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.524+0000 7f1d5ac00640 1 -- 172.21.15.47:0/3093428468 wait complete. 2024-04-20T10:30:59.542 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-20T10:30:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.729+0000 7f2736200640 1 Processor -- start 2024-04-20T10:30:59.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.730+0000 7f2736200640 1 -- start start 2024-04-20T10:30:59.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.731+0000 7f2736200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f273015e960 0x7f27300d4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.731+0000 7f2736200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f27300b6830 0x7f27300d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.731+0000 7f2736200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f273015ed40 con 0x7f27300d1c10 2024-04-20T10:30:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.731+0000 7f2736200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f27300b6c60 con 0x7f27300b6830 2024-04-20T10:30:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.731+0000 7f2736200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f27300b6e90 con 0x7f273015e960 2024-04-20T10:30:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f2727e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f27300d1c10 0x7f27300b6120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57894/0 (socket says 172.21.15.47:57894) 2024-04-20T10:30:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f2727e00640 1 -- 172.21.15.47:0/3932280132 learned_addr learned my addr 172.21.15.47:0/3932280132 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:30:59.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f272fe00640 1 --2- 172.21.15.47:0/3932280132 >> v2:172.21.15.47:3300/0 conn(0x7f27300b6830 0x7f27300d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f272f400640 1 --2- 172.21.15.47:0/3932280132 >> v2:172.21.15.175:3300/0 conn(0x7f273015e960 0x7f27300d4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f272fe00640 1 -- 172.21.15.47:0/3932280132 >> v2:172.21.15.175:3300/0 conn(0x7f273015e960 msgr2=0x7f27300d4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:30:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f272fe00640 1 --2- 172.21.15.47:0/3932280132 >> v2:172.21.15.175:3300/0 conn(0x7f273015e960 0x7f27300d4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f272fe00640 1 -- 172.21.15.47:0/3932280132 >> v1:172.21.15.47:6789/0 conn(0x7f27300d1c10 legacy=0x7f27300b6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.732+0000 7f272fe00640 1 -- 172.21.15.47:0/3932280132 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27300d0e50 con 0x7f27300b6830 2024-04-20T10:30:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.733+0000 7f272fe00640 1 --2- 172.21.15.47:0/3932280132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27300b6830 0x7f27300d4fe0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2730068070 tx=0x7f272802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7b8898709c3996 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f272ea00640 1 -- 172.21.15.47:0/3932280132 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728039070 con 0x7f27300b6830 2024-04-20T10:30:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f272ea00640 1 -- 172.21.15.47:0/3932280132 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f272802cc70 con 0x7f27300b6830 2024-04-20T10:30:59.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f272ea00640 1 -- 172.21.15.47:0/3932280132 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728034660 con 0x7f27300b6830 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 -- 172.21.15.47:0/3932280132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27300b6830 msgr2=0x7f27300d4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 --2- 172.21.15.47:0/3932280132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27300b6830 0x7f27300d4fe0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f2730068070 tx=0x7f272802bec0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 -- 172.21.15.47:0/3932280132 shutdown_connections 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 --2- 172.21.15.47:0/3932280132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f27300b6830 0x7f27300d4fe0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 --2- 172.21.15.47:0/3932280132 >> v2:172.21.15.175:3300/0 conn(0x7f273015e960 0x7f27300d4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 -- 172.21.15.47:0/3932280132 >> 172.21.15.47:0/3932280132 conn(0x7f27300b96f0 msgr2=0x7f27300b9b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 -- 172.21.15.47:0/3932280132 shutdown_connections 2024-04-20T10:30:59.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.734+0000 7f2736200640 1 -- 172.21.15.47:0/3932280132 wait complete. 2024-04-20T10:30:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.736+0000 7f2736200640 1 Processor -- start 2024-04-20T10:30:59.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.736+0000 7f2736200640 1 -- start start 2024-04-20T10:30:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.737+0000 7f2736200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27300b6830 0x7f273006bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.737+0000 7f2736200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27300d1c10 0x7f27301f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.737+0000 7f2736200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273015e960 0x7f27300b7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.737+0000 7f2736200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f27300d16f0 con 0x7f273015e960 2024-04-20T10:30:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.737+0000 7f2736200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27300d1410 con 0x7f27300d1c10 2024-04-20T10:30:59.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.737+0000 7f2736200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27300d3160 con 0x7f27300b6830 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273015e960 0x7f27300b7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273015e960 0x7f27300b7920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33864/0 (socket says 172.21.15.47:33864) 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 -- 172.21.15.47:0/1695071659 learned_addr learned my addr 172.21.15.47:0/1695071659 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272f400640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27300b6830 0x7f273006bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 -- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27300b6830 msgr2=0x7f273006bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27300b6830 0x7f273006bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 -- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27300d1c10 msgr2=0x7f27301f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27300d1c10 0x7f27301f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:30:59.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.738+0000 7f272fe00640 1 -- 172.21.15.47:0/1695071659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f273006c360 con 0x7f273015e960 2024-04-20T10:30:59.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.739+0000 7f272fe00640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273015e960 0x7f27300b7920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2728009e80 tx=0x7f2728004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.739+0000 7f2727400640 1 -- 172.21.15.47:0/1695071659 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728039050 con 0x7f273015e960 2024-04-20T10:30:59.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.739+0000 7f2727400640 1 -- 172.21.15.47:0/1695071659 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2728045070 con 0x7f273015e960 2024-04-20T10:30:59.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.739+0000 7f2727400640 1 -- 172.21.15.47:0/1695071659 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272803e830 con 0x7f273015e960 2024-04-20T10:30:59.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.739+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f273006c640 con 0x7f273015e960 2024-04-20T10:30:59.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.739+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27300cc640 con 0x7f273015e960 2024-04-20T10:30:59.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.740+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2730068980 con 0x7f273015e960 2024-04-20T10:30:59.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.741+0000 7f2727400640 1 -- 172.21.15.47:0/1695071659 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f272803e9d0 con 0x7f273015e960 2024-04-20T10:30:59.742 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.742+0000 7f2727400640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f270c06c8f0 0x7f270c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:30:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.743+0000 7f2727400640 1 -- 172.21.15.47:0/1695071659 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f27280aebb0 con 0x7f273015e960 2024-04-20T10:30:59.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.743+0000 7f272f400640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f270c06c8f0 0x7f270c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:30:59.744 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.745+0000 7f272f400640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f270c06c8f0 0x7f270c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2720006fd0 tx=0x7f2720008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:30:59.747 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.747+0000 7f2727400640 1 -- 172.21.15.47:0/1695071659 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f272802ce20 con 0x7f273015e960 2024-04-20T10:30:59.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.995+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f27300b39b0 con 0x7f273015e960 2024-04-20T10:30:59.996 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.996+0000 7f2727400640 1 -- 172.21.15.47:0/1695071659 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f272807ce20 con 0x7f273015e960 2024-04-20T10:30:59.996 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:30:59.996 INFO:teuthology.orchestra.run.smithi047.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-20T10:30:59.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f270c06c8f0 msgr2=0x7f270c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f270c06c8f0 0x7f270c06edb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2720006fd0 tx=0x7f2720008040 comp rx=0 tx=0).stop 2024-04-20T10:30:59.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273015e960 msgr2=0x7f27300b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:30:59.999 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273015e960 0x7f27300b7920 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f2728009e80 tx=0x7f2728004a40 comp rx=0 tx=0).stop 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 shutdown_connections 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f270c06c8f0 0x7f270c06edb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f273015e960 0x7f27300b7920 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27300d1c10 0x7f27301f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 --2- 172.21.15.47:0/1695071659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27300b6830 0x7f273006bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 >> 172.21.15.47:0/1695071659 conn(0x7f27300b96f0 msgr2=0x7f27300b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 shutdown_connections 2024-04-20T10:31:00.000 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:30:59.999+0000 7f2736200640 1 -- 172.21.15.47:0/1695071659 wait complete. 2024-04-20T10:31:00.019 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-04-20T10:31:00.019 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-20T10:31:00.019 DEBUG:teuthology.orchestra.run.smithi047:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:31:00.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-20T10:31:00.233 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-20T10:31:00.233 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:31:00.233 DEBUG:tasks.cephfs.mount:terminating RemoteProcess(client=, args=['stdin-killer', '--', 'python3', '-c', '\nimport fcntl\nimport os\nimport sys\nimport time\n\nfcntl.fcntl(sys.stdin, fcntl.F_SETFL, fcntl.fcntl(sys.stdin, fcntl.F_GETFL) | os.O_NONBLOCK)\n\nfd = os.open("/home/ubuntu/cephtest/mnt.0/background_file", os.O_RDWR | os.O_CREAT, 0o644)\ntry:\n while True:\n print("write_background: writing", file=sys.stderr)\n os.write(fd, b\'content\')\n if not True:\n break\n try:\n if os.read(0, 4096) == b"":\n break\n except BlockingIOError:\n pass\n time.sleep(2)\nexcept IOError as e:\n pass\nos.close(fd)\n'], check_status=True, hostname='smithi047') 2024-04-20T10:31:00.234 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:31:00.234 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:00.294 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.504+0000 7f8f7ee00640 1 Processor -- start 2024-04-20T10:31:00.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.504+0000 7f8f7ee00640 1 -- start start 2024-04-20T10:31:00.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.505+0000 7f8f7ee00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f8f7815b400 0x7f8f78072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.505+0000 7f8f7ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 0x7f8f7806ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.505+0000 7f8f7ee00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8f7815c7f0 con 0x7f8f7815eec0 2024-04-20T10:31:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.505+0000 7f8f7ee00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f8f7806d280 con 0x7f8f7815b400 2024-04-20T10:31:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.505+0000 7f8f7ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7806ccc0 con 0x7f8f7815fa40 2024-04-20T10:31:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.506+0000 7f8f7d000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8f7815eec0 0x7f8f7815f330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57902/0 (socket says 172.21.15.47:57902) 2024-04-20T10:31:00.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.506+0000 7f8f7d000640 1 -- 172.21.15.47:0/3945136944 learned_addr learned my addr 172.21.15.47:0/3945136944 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.506+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3945136944 >> v2:172.21.15.175:3300/0 conn(0x7f8f7815b400 0x7f8f78072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.506+0000 7f8f7e400640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 0x7f8f7806ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.506+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1541044484 0 0) 0x7f8f7815c7f0 con 0x7f8f7815eec0 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.506+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f8f60003520 con 0x7f8f7815eec0 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4143494812 0 0) 0x7f8f60003520 con 0x7f8f7815eec0 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 msgr2=0x7f8f7806ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 0x7f8f7806ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 >> v2:172.21.15.175:3300/0 conn(0x7f8f7815b400 msgr2=0x7f8f78072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 --2- 172.21.15.47:0/3945136944 >> v2:172.21.15.175:3300/0 conn(0x7f8f7815b400 0x7f8f78072d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f7806cfa0 con 0x7f8f7815eec0 2024-04-20T10:31:00.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f8f6c0019f0 con 0x7f8f7815eec0 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 >> v1:172.21.15.47:6789/0 conn(0x7f8f7815eec0 legacy=0x7f8f7815f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f600039b0 0x7f8f60003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 0x7f8f60004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f60004d40 0x7f8f600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7806d280 con 0x7f8f600039b0 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7806ccc0 con 0x7f8f60004d40 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f6000dbd0 con 0x7f8f7815fa40 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.507+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f600039b0 0x7f8f60003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.508+0000 7f8f7d000640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 0x7f8f60004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.508+0000 7f8f7da00640 1 -- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 msgr2=0x7f8f60004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.508+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f7815fa40 0x7f8f60004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.508+0000 7f8f7e400640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f60004d40 0x7f8f600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.508+0000 7f8f7da00640 1 -- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f60004d40 msgr2=0x7f8f600051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.508+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f60004d40 0x7f8f600051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.508+0000 7f8f7da00640 1 -- 172.21.15.47:0/3945136944 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f60003520 con 0x7f8f600039b0 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.509+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f600039b0 0x7f8f60003dd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009ac0 tx=0x7f8f60005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9155844dc56e892b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.509+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68039070 con 0x7f8f600039b0 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.509+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6802c710 con 0x7f8f600039b0 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.509+0000 7f8f77e00640 1 -- 172.21.15.47:0/3945136944 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f68009690 con 0x7f8f600039b0 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.510+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3945136944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f600039b0 msgr2=0x7f8f60003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.510+0000 7f8f7ee00640 1 --2- 172.21.15.47:0/3945136944 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f600039b0 0x7f8f60003dd0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009ac0 tx=0x7f8f60005830 comp rx=0 tx=0).stop 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.510+0000 7f8f7e400640 1 -- 172.21.15.47:0/3945136944 reap_dead start 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.510+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3945136944 shutdown_connections 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.510+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3945136944 >> 172.21.15.47:0/3945136944 conn(0x7f8f780d0040 msgr2=0x7f8f780d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.510+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3945136944 shutdown_connections 2024-04-20T10:31:00.511 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.510+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3945136944 wait complete. 2024-04-20T10:31:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.512+0000 7f8f7ee00640 1 Processor -- start 2024-04-20T10:31:00.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.512+0000 7f8f7ee00640 1 -- start start 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.513+0000 7f8f7ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f600039b0 0x7f8f781493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.513+0000 7f8f7ee00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f7809d220 0x7f8f7814c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.513+0000 7f8f7ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f78149c30 0x7f8f78159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.513+0000 7f8f7ee00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78073240 con 0x7f8f7809d220 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.513+0000 7f8f7ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7806e720 con 0x7f8f78149c30 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.513+0000 7f8f7ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f7806b900 con 0x7f8f600039b0 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f7809d220 0x7f8f7814c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f7809d220 0x7f8f7814c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33884/0 (socket says 172.21.15.47:33884) 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 -- 172.21.15.47:0/3403683434 learned_addr learned my addr 172.21.15.47:0/3403683434 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:00.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f600039b0 0x7f8f781493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 -- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f600039b0 msgr2=0x7f8f781493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f600039b0 0x7f8f781493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 -- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f78149c30 msgr2=0x7f8f78159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f78149c30 0x7f8f78159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.514+0000 7f8f7d000640 1 -- 172.21.15.47:0/3403683434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f78149920 con 0x7f8f7809d220 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.515+0000 7f8f7d000640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f7809d220 0x7f8f7814c670 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c001af0 tx=0x7f8f6c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.515+0000 7f8f75600640 1 -- 172.21.15.47:0/3403683434 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c019070 con 0x7f8f7809d220 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.515+0000 7f8f75600640 1 -- 172.21.15.47:0/3403683434 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8f6c004d60 con 0x7f8f7809d220 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.515+0000 7f8f75600640 1 -- 172.21.15.47:0/3403683434 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f6c0072f0 con 0x7f8f7809d220 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.515+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f7809d7f0 con 0x7f8f7809d220 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.515+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f78159140 con 0x7f8f7809d220 2024-04-20T10:31:00.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.516+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f78075b60 con 0x7f8f7809d220 2024-04-20T10:31:00.517 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.518+0000 7f8f75600640 1 -- 172.21.15.47:0/3403683434 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8f6c01c070 con 0x7f8f7809d220 2024-04-20T10:31:00.518 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.519+0000 7f8f75600640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.519+0000 7f8f75600640 1 -- 172.21.15.47:0/3403683434 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f8f6c08bf20 con 0x7f8f7809d220 2024-04-20T10:31:00.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.519+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.520 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.521+0000 7f8f7da00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8f68036000 tx=0x7f8f68030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:00.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.523+0000 7f8f75600640 1 -- 172.21.15.47:0/3403683434 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8f6c05a100 con 0x7f8f7809d220 2024-04-20T10:31:00.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.732+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f780b5220 con 0x7f8f7809d220 2024-04-20T10:31:00.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.735+0000 7f8f75600640 1 -- 172.21.15.47:0/3403683434 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+17151 (secure 0 0 0) 0x7f8f6c059a00 con 0x7f8f7809d220 2024-04-20T10:31:00.735 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:00.736 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:54.470486+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:30:19.610070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-20T10:30:20.722858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6833/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6832/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6835/3814505199":"2024-04-21T10:30:53.464895+0000","172.21.15.47:6834/3814505199":"2024-04-21T10:30:53.464895+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.737+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f5c06c8f0 msgr2=0x7f8f5c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f8f68036000 tx=0x7f8f68030000 comp rx=0 tx=0).stop 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f7809d220 msgr2=0x7f8f7814c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f7809d220 0x7f8f7814c670 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c001af0 tx=0x7f8f6c00b4a0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 shutdown_connections 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f78149c30 0x7f8f78159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8f5c06c8f0 0x7f8f5c06edb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8f7809d220 0x7f8f7814c670 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.738 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 --2- 172.21.15.47:0/3403683434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f600039b0 0x7f8f781493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 >> 172.21.15.47:0/3403683434 conn(0x7f8f780d0040 msgr2=0x7f8f780cac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 shutdown_connections 2024-04-20T10:31:00.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.738+0000 7f8f7ee00640 1 -- 172.21.15.47:0/3403683434 wait complete. 2024-04-20T10:31:00.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:30:19.610070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:30:20.722858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:31:00.758 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:31:00.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.966+0000 7f4878e00640 1 Processor -- start 2024-04-20T10:31:00.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.966+0000 7f4878e00640 1 -- start start 2024-04-20T10:31:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.967+0000 7f4878e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4874156c20 0x7f487406b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4878e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48740d0360 0x7f48740d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4878e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4874141c60 con 0x7f487415ca40 2024-04-20T10:31:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4878e00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f48740d0d10 con 0x7f48740d0360 2024-04-20T10:31:00.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4878e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f48740adab0 con 0x7f4874156c20 2024-04-20T10:31:00.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4872800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f48740d0360 0x7f48740d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4871400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f487415ca40 0x7f48740ad3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57918/0 (socket says 172.21.15.47:57918) 2024-04-20T10:31:00.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4871400640 1 -- 172.21.15.47:0/785818749 learned_addr learned my addr 172.21.15.47:0/785818749 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:00.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.968+0000 7f4871e00640 1 --2- 172.21.15.47:0/785818749 >> v2:172.21.15.175:3300/0 conn(0x7f4874156c20 0x7f487406b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.969+0000 7f4872800640 1 -- 172.21.15.47:0/785818749 >> v2:172.21.15.175:3300/0 conn(0x7f4874156c20 msgr2=0x7f487406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:00.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.969+0000 7f4872800640 1 --2- 172.21.15.47:0/785818749 >> v2:172.21.15.175:3300/0 conn(0x7f4874156c20 0x7f487406b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.969+0000 7f4872800640 1 -- 172.21.15.47:0/785818749 >> v1:172.21.15.47:6789/0 conn(0x7f487415ca40 legacy=0x7f48740ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.969+0000 7f4870a00640 1 -- 172.21.15.47:0/785818749 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 109407572 0 0) 0x7f4874141c60 con 0x7f487415ca40 2024-04-20T10:31:00.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.969+0000 7f4872800640 1 -- 172.21.15.47:0/785818749 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487415cf90 con 0x7f48740d0360 2024-04-20T10:31:00.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.970+0000 7f4872800640 1 --2- 172.21.15.47:0/785818749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48740d0360 0x7f48740d07d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f487406fdc0 tx=0x7f486c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924b5875580e1412 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:00.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4870a00640 1 -- 172.21.15.47:0/785818749 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c035020 con 0x7f48740d0360 2024-04-20T10:31:00.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4870a00640 1 -- 172.21.15.47:0/785818749 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c031830 con 0x7f48740d0360 2024-04-20T10:31:00.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4870a00640 1 -- 172.21.15.47:0/785818749 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c031c00 con 0x7f48740d0360 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4878e00640 1 -- 172.21.15.47:0/785818749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48740d0360 msgr2=0x7f48740d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4878e00640 1 --2- 172.21.15.47:0/785818749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48740d0360 0x7f48740d07d0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f487406fdc0 tx=0x7f486c02b8d0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4878e00640 1 -- 172.21.15.47:0/785818749 shutdown_connections 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4878e00640 1 --2- 172.21.15.47:0/785818749 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f48740d0360 0x7f48740d07d0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4878e00640 1 --2- 172.21.15.47:0/785818749 >> v2:172.21.15.175:3300/0 conn(0x7f4874156c20 0x7f487406b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4878e00640 1 -- 172.21.15.47:0/785818749 >> 172.21.15.47:0/785818749 conn(0x7f48740b3350 msgr2=0x7f48740b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.971+0000 7f4878e00640 1 -- 172.21.15.47:0/785818749 shutdown_connections 2024-04-20T10:31:00.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.972+0000 7f4878e00640 1 -- 172.21.15.47:0/785818749 wait complete. 2024-04-20T10:31:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.973+0000 7f4878e00640 1 Processor -- start 2024-04-20T10:31:00.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.973+0000 7f4878e00640 1 -- start start 2024-04-20T10:31:00.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.974+0000 7f4878e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48740d0360 0x7f487406e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.974+0000 7f4878e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4874156c20 0x7f487415b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.974 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.974+0000 7f4878e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487415ca40 0x7f48741f74b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.974+0000 7f4878e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f487415d830 con 0x7f487415ca40 2024-04-20T10:31:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.974+0000 7f4878e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f487415d550 con 0x7f4874156c20 2024-04-20T10:31:00.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.974+0000 7f4878e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f487406be40 con 0x7f48740d0360 2024-04-20T10:31:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487415ca40 0x7f48741f74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487415ca40 0x7f48741f74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33902/0 (socket says 172.21.15.47:33902) 2024-04-20T10:31:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 -- 172.21.15.47:0/2726096550 learned_addr learned my addr 172.21.15.47:0/2726096550 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4871400640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4874156c20 0x7f487415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.976 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4871e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48740d0360 0x7f487406e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 -- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48740d0360 msgr2=0x7f487406e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48740d0360 0x7f487406e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 -- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4874156c20 msgr2=0x7f487415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4874156c20 0x7f487415b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4872800640 1 -- 172.21.15.47:0/2726096550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f487406eb00 con 0x7f487415ca40 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.975+0000 7f4871e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48740d0360 0x7f487406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.976+0000 7f4872800640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487415ca40 0x7f48741f74b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f486c009890 tx=0x7f486c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.976+0000 7f486a000640 1 -- 172.21.15.47:0/2726096550 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c035040 con 0x7f487415ca40 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.976+0000 7f486a000640 1 -- 172.21.15.47:0/2726096550 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f486c03b840 con 0x7f487415ca40 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.976+0000 7f486a000640 1 -- 172.21.15.47:0/2726096550 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f486c03bc30 con 0x7f487415ca40 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.976+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f487406edb0 con 0x7f487415ca40 2024-04-20T10:31:00.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.976+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f487406f370 con 0x7f487415ca40 2024-04-20T10:31:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.978+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4874068980 con 0x7f487415ca40 2024-04-20T10:31:00.978 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.978+0000 7f486a000640 1 -- 172.21.15.47:0/2726096550 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f486c038020 con 0x7f487415ca40 2024-04-20T10:31:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.979+0000 7f486a000640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f485006c8f0 0x7f485006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:00.979 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.979+0000 7f486a000640 1 -- 172.21.15.47:0/2726096550 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f486c0addf0 con 0x7f487415ca40 2024-04-20T10:31:00.980 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.980+0000 7f4871e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f485006c8f0 0x7f485006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:00.981 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.981+0000 7f4871e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f485006c8f0 0x7f485006edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4864007520 tx=0x7f486400b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:00.984 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:00.984+0000 7f486a000640 1 -- 172.21.15.47:0/2726096550 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f486c07bfd0 con 0x7f487415ca40 2024-04-20T10:31:01.190 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.190+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f48740cfd90 con 0x7f487415ca40 2024-04-20T10:31:01.191 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.191+0000 7f486a000640 1 -- 172.21.15.47:0/2726096550 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f486c07b8d0 con 0x7f487415ca40 2024-04-20T10:31:01.191 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:31:01.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.193+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f485006c8f0 msgr2=0x7f485006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.193+0000 7f4878e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f485006c8f0 0x7f485006edb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4864007520 tx=0x7f486400b040 comp rx=0 tx=0).stop 2024-04-20T10:31:01.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487415ca40 msgr2=0x7f48741f74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487415ca40 0x7f48741f74b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f486c009890 tx=0x7f486c0046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 shutdown_connections 2024-04-20T10:31:01.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f485006c8f0 0x7f485006edb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.194 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f487415ca40 0x7f48741f74b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4874156c20 0x7f487415b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 --2- 172.21.15.47:0/2726096550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48740d0360 0x7f487406e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 >> 172.21.15.47:0/2726096550 conn(0x7f48740b3350 msgr2=0x7f48740b3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:01.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 shutdown_connections 2024-04-20T10:31:01.195 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.194+0000 7f4878e00640 1 -- 172.21.15.47:0/2726096550 wait complete. 2024-04-20T10:31:01.212 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:31:01.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.426+0000 7f94a9200640 1 Processor -- start 2024-04-20T10:31:01.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.426+0000 7f94a9200640 1 -- start start 2024-04-20T10:31:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.427+0000 7f94a9200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94a40aee60 0x7f94a40af240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.427+0000 7f94a9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a415d4f0 0x7f94a40ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.427+0000 7f94a9200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f94a40af780 con 0x7f94a415fe10 2024-04-20T10:31:01.427 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.427+0000 7f94a9200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f94a406c510 con 0x7f94a40aee60 2024-04-20T10:31:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.427+0000 7f94a9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94a406b900 con 0x7f94a415d4f0 2024-04-20T10:31:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.428+0000 7f94a2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a415d4f0 0x7f94a40ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.428+0000 7f94a3400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94a40aee60 0x7f94a40af240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.428+0000 7f94a3400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f94a40aee60 0x7f94a40af240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:41568/0 (socket says 172.21.15.47:41568) 2024-04-20T10:31:01.428 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.428+0000 7f94a3400640 1 -- 172.21.15.47:0/109366242 learned_addr learned my addr 172.21.15.47:0/109366242 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.429+0000 7f94a3400640 1 -- 172.21.15.47:0/109366242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a415d4f0 msgr2=0x7f94a40ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.429+0000 7f94a3400640 1 --2- 172.21.15.47:0/109366242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a415d4f0 0x7f94a40ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.429+0000 7f94a3400640 1 -- 172.21.15.47:0/109366242 >> v1:172.21.15.175:6789/0 conn(0x7f94a415fe10 legacy=0x7f94a4160270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.429+0000 7f94a3400640 1 -- 172.21.15.47:0/109366242 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a406bbe0 con 0x7f94a40aee60 2024-04-20T10:31:01.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.429+0000 7f94a2000640 1 -- 172.21.15.47:0/109366242 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1681356013 0 0) 0x7f94a40af780 con 0x7f94a415fe10 2024-04-20T10:31:01.430 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.431+0000 7f94a3400640 1 --2- 172.21.15.47:0/109366242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a40aee60 0x7f94a40af240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9494009e50 tx=0x7f949402bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33cfe92d5e6c6819 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.431+0000 7f94a2000640 1 -- 172.21.15.47:0/109366242 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949402cb00 con 0x7f94a40aee60 2024-04-20T10:31:01.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.431+0000 7f94a2000640 1 -- 172.21.15.47:0/109366242 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949402cca0 con 0x7f94a40aee60 2024-04-20T10:31:01.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.431+0000 7f94a2000640 1 -- 172.21.15.47:0/109366242 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9494034610 con 0x7f94a40aee60 2024-04-20T10:31:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 -- 172.21.15.47:0/109366242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a40aee60 msgr2=0x7f94a40af240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 --2- 172.21.15.47:0/109366242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a40aee60 0x7f94a40af240 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9494009e50 tx=0x7f949402bf60 comp rx=0 tx=0).stop 2024-04-20T10:31:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 -- 172.21.15.47:0/109366242 shutdown_connections 2024-04-20T10:31:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 --2- 172.21.15.47:0/109366242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a415d4f0 0x7f94a40ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.432 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 --2- 172.21.15.47:0/109366242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a40aee60 0x7f94a40af240 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 -- 172.21.15.47:0/109366242 >> 172.21.15.47:0/109366242 conn(0x7f94a40b5270 msgr2=0x7f94a40b3210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 -- 172.21.15.47:0/109366242 shutdown_connections 2024-04-20T10:31:01.433 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.432+0000 7f94a9200640 1 -- 172.21.15.47:0/109366242 wait complete. 2024-04-20T10:31:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.434+0000 7f94a9200640 1 Processor -- start 2024-04-20T10:31:01.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.434+0000 7f94a9200640 1 -- start start 2024-04-20T10:31:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a9200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a40aee60 0x7f94a4072d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a9200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a415d4f0 0x7f94a414c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a9200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a415fe10 0x7f94a409b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a9200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f94a406f070 con 0x7f94a415fe10 2024-04-20T10:31:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a9200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94a406c1a0 con 0x7f94a415d4f0 2024-04-20T10:31:01.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a9200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94a406c9a0 con 0x7f94a40aee60 2024-04-20T10:31:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a415fe10 0x7f94a409b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a3e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a415fe10 0x7f94a409b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33916/0 (socket says 172.21.15.47:33916) 2024-04-20T10:31:01.436 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.435+0000 7f94a3e00640 1 -- 172.21.15.47:0/4177275462 learned_addr learned my addr 172.21.15.47:0/4177275462 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a3400640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a40aee60 0x7f94a4072d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a2a00640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a415d4f0 0x7f94a414c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a3e00640 1 -- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a40aee60 msgr2=0x7f94a4072d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a3e00640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a40aee60 0x7f94a4072d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a3e00640 1 -- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a415d4f0 msgr2=0x7f94a414c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a3e00640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a415d4f0 0x7f94a414c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a3e00640 1 -- 172.21.15.47:0/4177275462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a414c6d0 con 0x7f94a415fe10 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f94a3e00640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a415fe10 0x7f94a409b8a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c4b0 tx=0x7f949c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.437 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.436+0000 7f949b400640 1 -- 172.21.15.47:0/4177275462 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c005480 con 0x7f94a415fe10 2024-04-20T10:31:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.437+0000 7f949b400640 1 -- 172.21.15.47:0/4177275462 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949c0035f0 con 0x7f94a415fe10 2024-04-20T10:31:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.437+0000 7f949b400640 1 -- 172.21.15.47:0/4177275462 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0039c0 con 0x7f94a415fe10 2024-04-20T10:31:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.437+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a409a650 con 0x7f94a415fe10 2024-04-20T10:31:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.437+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a409ac10 con 0x7f94a415fe10 2024-04-20T10:31:01.438 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.438+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a41504d0 con 0x7f94a415fe10 2024-04-20T10:31:01.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.439+0000 7f949b400640 1 -- 172.21.15.47:0/4177275462 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f949c003b60 con 0x7f94a415fe10 2024-04-20T10:31:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.440+0000 7f949b400640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f948006ca90 0x7f948006ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.440+0000 7f949b400640 1 -- 172.21.15.47:0/4177275462 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f949c08ba50 con 0x7f94a415fe10 2024-04-20T10:31:01.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.440+0000 7f94a3400640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f948006ca90 0x7f948006ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.442+0000 7f94a3400640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f948006ca90 0x7f948006ef50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9494035000 tx=0x7f9494030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.444+0000 7f949b400640 1 -- 172.21.15.47:0/4177275462 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f949c059b10 con 0x7f94a415fe10 2024-04-20T10:31:01.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.652+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f94a4075b30 con 0x7f94a415fe10 2024-04-20T10:31:01.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.652+0000 7f949b400640 1 -- 172.21.15.47:0/4177275462 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f949c059410 con 0x7f94a415fe10 2024-04-20T10:31:01.653 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.655+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f948006ca90 msgr2=0x7f948006ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.655+0000 7f94a9200640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f948006ca90 0x7f948006ef50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f9494035000 tx=0x7f9494030000 comp rx=0 tx=0).stop 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.655+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a415fe10 msgr2=0x7f94a409b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.655+0000 7f94a9200640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a415fe10 0x7f94a409b8a0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c4b0 tx=0x7f949c00c970 comp rx=0 tx=0).stop 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 shutdown_connections 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f948006ca90 0x7f948006ef50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f94a415fe10 0x7f94a409b8a0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a415d4f0 0x7f94a414c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 --2- 172.21.15.47:0/4177275462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a40aee60 0x7f94a4072d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 >> 172.21.15.47:0/4177275462 conn(0x7f94a40b5270 msgr2=0x7f94a40b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:01.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 shutdown_connections 2024-04-20T10:31:01.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.656+0000 7f94a9200640 1 -- 172.21.15.47:0/4177275462 wait complete. 2024-04-20T10:31:01.675 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:31:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.890+0000 7f7610a00640 1 Processor -- start 2024-04-20T10:31:01.890 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.890+0000 7f7610a00640 1 -- start start 2024-04-20T10:31:01.891 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.891+0000 7f7610a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f760c0c7740 0x7f760c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.892+0000 7f7610a00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f760c141c10 con 0x7f760c1412e0 2024-04-20T10:31:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.892+0000 7f7610a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f760c141ef0 con 0x7f760c1416c0 2024-04-20T10:31:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.892+0000 7f7610a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f760c142820 con 0x7f760c0c7740 2024-04-20T10:31:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.892+0000 7f760b400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f760c1412e0 0x7f760c0c7030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57934/0 (socket says 172.21.15.47:57934) 2024-04-20T10:31:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.892+0000 7f760aa00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f760c0c7740 0x7f760c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.892 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.892+0000 7f760b400640 1 -- 172.21.15.47:0/3200581537 learned_addr learned my addr 172.21.15.47:0/3200581537 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.893+0000 7f760a000640 1 -- 172.21.15.47:0/3200581537 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3409837136 0 0) 0x7f760c141c10 con 0x7f760c1412e0 2024-04-20T10:31:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.893+0000 7f760a000640 1 -- 172.21.15.47:0/3200581537 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f75f0003520 con 0x7f760c1412e0 2024-04-20T10:31:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.893+0000 7f760aa00640 1 -- 172.21.15.47:0/3200581537 >> v1:172.21.15.175:6789/0 conn(0x7f760c1416c0 legacy=0x7f760c162490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.893+0000 7f760aa00640 1 -- 172.21.15.47:0/3200581537 >> v1:172.21.15.47:6789/0 conn(0x7f760c1412e0 legacy=0x7f760c0c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.893 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.893+0000 7f760aa00640 1 -- 172.21.15.47:0/3200581537 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c1421d0 con 0x7f760c0c7740 2024-04-20T10:31:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.894+0000 7f760aa00640 1 --2- 172.21.15.47:0/3200581537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c0c7740 0x7f760c06d2a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f75f80097a0 tx=0x7f75f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d52a1e12ab93d5a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.894+0000 7f760a000640 1 -- 172.21.15.47:0/3200581537 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8035020 con 0x7f760c0c7740 2024-04-20T10:31:01.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.894+0000 7f760a000640 1 -- 172.21.15.47:0/3200581537 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f8031830 con 0x7f760c0c7740 2024-04-20T10:31:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.894+0000 7f760a000640 1 -- 172.21.15.47:0/3200581537 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f8031be0 con 0x7f760c0c7740 2024-04-20T10:31:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.895+0000 7f7610a00640 1 -- 172.21.15.47:0/3200581537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c0c7740 msgr2=0x7f760c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.895+0000 7f7610a00640 1 --2- 172.21.15.47:0/3200581537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c0c7740 0x7f760c06d2a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f75f80097a0 tx=0x7f75f802b8d0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.895+0000 7f7610a00640 1 -- 172.21.15.47:0/3200581537 shutdown_connections 2024-04-20T10:31:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.895+0000 7f7610a00640 1 --2- 172.21.15.47:0/3200581537 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c0c7740 0x7f760c06d2a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.895+0000 7f7610a00640 1 -- 172.21.15.47:0/3200581537 >> 172.21.15.47:0/3200581537 conn(0x7f760c0b3dc0 msgr2=0x7f760c0b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:01.895 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.895+0000 7f7610a00640 1 -- 172.21.15.47:0/3200581537 shutdown_connections 2024-04-20T10:31:01.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.895+0000 7f7610a00640 1 -- 172.21.15.47:0/3200581537 wait complete. 2024-04-20T10:31:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.897+0000 7f7610a00640 1 Processor -- start 2024-04-20T10:31:01.897 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.897+0000 7f7610a00640 1 -- start start 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f7610a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c0c7740 0x7f760c157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f7610a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c1412e0 0x7f760c155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f7610a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c1416c0 0x7f760c156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f7610a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f760c162d20 con 0x7f760c1412e0 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f7610a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f760c162ba0 con 0x7f760c0c7740 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f7610a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f760c162ea0 con 0x7f760c1416c0 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f760aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c1412e0 0x7f760c155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f760aa00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c1412e0 0x7f760c155c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33934/0 (socket says 172.21.15.47:33934) 2024-04-20T10:31:01.902 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.898+0000 7f760aa00640 1 -- 172.21.15.47:0/3026010177 learned_addr learned my addr 172.21.15.47:0/3026010177 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760b400640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c0c7740 0x7f760c157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760aa00640 1 -- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c1416c0 msgr2=0x7f760c156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760be00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c1416c0 0x7f760c156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760aa00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c1416c0 0x7f760c156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760aa00640 1 -- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c0c7740 msgr2=0x7f760c157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760aa00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c0c7740 0x7f760c157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760aa00640 1 -- 172.21.15.47:0/3026010177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c1567b0 con 0x7f760c1412e0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760b400640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c0c7740 0x7f760c157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760be00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c1416c0 0x7f760c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f760aa00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c1412e0 0x7f760c155c40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f75f802b910 tx=0x7f75f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f7603400640 1 -- 172.21.15.47:0/3026010177 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f803a8b0 con 0x7f760c1412e0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.899+0000 7f7603400640 1 -- 172.21.15.47:0/3026010177 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f803aa50 con 0x7f760c1412e0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.900+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c156a90 con 0x7f760c1412e0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.900+0000 7f7603400640 1 -- 172.21.15.47:0/3026010177 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f803ae40 con 0x7f760c1412e0 2024-04-20T10:31:01.903 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.900+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c157050 con 0x7f760c1412e0 2024-04-20T10:31:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.901+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c075b30 con 0x7f760c1412e0 2024-04-20T10:31:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.902+0000 7f7603400640 1 -- 172.21.15.47:0/3026010177 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f75f8038020 con 0x7f760c1412e0 2024-04-20T10:31:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.903+0000 7f7603400640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f75ec06c8f0 0x7f75ec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.903+0000 7f7603400640 1 -- 172.21.15.47:0/3026010177 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f75f80adb00 con 0x7f760c1412e0 2024-04-20T10:31:01.904 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.904+0000 7f760b400640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f75ec06c8f0 0x7f75ec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:01.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.905+0000 7f760b400640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f75ec06c8f0 0x7f75ec06edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f75fc005c00 tx=0x7f75fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:01.908 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:01.908+0000 7f7603400640 1 -- 172.21.15.47:0/3026010177 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f75f8031800 con 0x7f760c1412e0 2024-04-20T10:31:02.108 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.108+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f760c06df70 con 0x7f760c1412e0 2024-04-20T10:31:02.110 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.110+0000 7f7603400640 1 -- 172.21.15.47:0/3026010177 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7f75f807bce0 con 0x7f760c1412e0 2024-04-20T10:31:02.110 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:31:02.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f75ec06c8f0 msgr2=0x7f75ec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:02.112 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f75ec06c8f0 0x7f75ec06edb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f75fc005c00 tx=0x7f75fc009040 comp rx=0 tx=0).stop 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c1412e0 msgr2=0x7f760c155c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c1412e0 0x7f760c155c40 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f75f802b910 tx=0x7f75f80046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 shutdown_connections 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f760c1416c0 0x7f760c156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f75ec06c8f0 0x7f75ec06edb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f760c1412e0 0x7f760c155c40 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 --2- 172.21.15.47:0/3026010177 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f760c0c7740 0x7f760c157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 >> 172.21.15.47:0/3026010177 conn(0x7f760c0b3dc0 msgr2=0x7f760c1436b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 shutdown_connections 2024-04-20T10:31:02.113 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.112+0000 7f7610a00640 1 -- 172.21.15.47:0/3026010177 wait complete. 2024-04-20T10:31:02.132 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-20T10:31:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.342+0000 7efeef200640 1 Processor -- start 2024-04-20T10:31:02.342 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.342+0000 7efeef200640 1 -- start start 2024-04-20T10:31:02.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.343+0000 7efeef200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7efef00ce2d0 0x7efef015c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:02.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.343+0000 7efeef200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efef00d15d0 0x7efef00cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:02.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.343+0000 7efeef200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef00b5d30 con 0x7efef00b54e0 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.343+0000 7efeef200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7efef00b6010 con 0x7efef00d15d0 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.343+0000 7efeef200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7efef00b62f0 con 0x7efef00ce2d0 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeee800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efef00d15d0 0x7efef00cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeee800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7efef00d15d0 0x7efef00cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33936/0 (socket says 172.21.15.47:33936) 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeee800640 1 -- 172.21.15.47:0/790167053 learned_addr learned my addr 172.21.15.47:0/790167053 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeede00640 1 --2- 172.21.15.47:0/790167053 >> v2:172.21.15.175:3300/0 conn(0x7efef00ce2d0 0x7efef015c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeee800640 1 -- 172.21.15.47:0/790167053 >> v2:172.21.15.175:3300/0 conn(0x7efef00ce2d0 msgr2=0x7efef015c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeee800640 1 --2- 172.21.15.47:0/790167053 >> v2:172.21.15.175:3300/0 conn(0x7efef00ce2d0 0x7efef015c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeee800640 1 -- 172.21.15.47:0/790167053 >> v1:172.21.15.175:6789/0 conn(0x7efef00b54e0 legacy=0x7efef00d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeee800640 1 -- 172.21.15.47:0/790167053 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef00b65d0 con 0x7efef00d15d0 2024-04-20T10:31:02.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.344+0000 7efeede00640 1 --2- 172.21.15.47:0/790167053 >> v2:172.21.15.175:3300/0 conn(0x7efef00ce2d0 0x7efef015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.345+0000 7efeee800640 1 --2- 172.21.15.47:0/790167053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef00cc930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efef00737f0 tx=0x7efee802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd46b000e28232a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeeca00640 1 -- 172.21.15.47:0/790167053 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee802ca80 con 0x7efef00d15d0 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeeca00640 1 -- 172.21.15.47:0/790167053 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee802cc20 con 0x7efef00d15d0 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeeca00640 1 -- 172.21.15.47:0/790167053 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee80343f0 con 0x7efef00d15d0 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeef200640 1 -- 172.21.15.47:0/790167053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 msgr2=0x7efef00cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeef200640 1 --2- 172.21.15.47:0/790167053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef00cc930 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efef00737f0 tx=0x7efee802bee0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeef200640 1 -- 172.21.15.47:0/790167053 shutdown_connections 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeef200640 1 --2- 172.21.15.47:0/790167053 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef00cc930 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeef200640 1 --2- 172.21.15.47:0/790167053 >> v2:172.21.15.175:3300/0 conn(0x7efef00ce2d0 0x7efef015c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.346+0000 7efeef200640 1 -- 172.21.15.47:0/790167053 >> 172.21.15.47:0/790167053 conn(0x7efef00ced60 msgr2=0x7efef00cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.347+0000 7efeef200640 1 -- 172.21.15.47:0/790167053 shutdown_connections 2024-04-20T10:31:02.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.347+0000 7efeef200640 1 -- 172.21.15.47:0/790167053 wait complete. 2024-04-20T10:31:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.348+0000 7efeef200640 1 Processor -- start 2024-04-20T10:31:02.348 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.348+0000 7efeef200640 1 -- start start 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.349+0000 7efeef200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef00b54e0 0x7efef00b0770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.349+0000 7efeef200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef00ce2d0 0x7efef015f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.349+0000 7efeef200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef015f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.349+0000 7efeef200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7efef00b5970 con 0x7efef00d15d0 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.349+0000 7efeef200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efef00ce6b0 con 0x7efef00ce2d0 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.349+0000 7efeef200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efef00b5af0 con 0x7efef00b54e0 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef015f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:02.350 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef015f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33942/0 (socket says 172.21.15.47:33942) 2024-04-20T10:31:02.351 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 -- 172.21.15.47:0/2440256467 learned_addr learned my addr 172.21.15.47:0/2440256467 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeede00640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef00b54e0 0x7efef00b0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 -- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef00b54e0 msgr2=0x7efef00b0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef00b54e0 0x7efef00b0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 -- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef00ce2d0 msgr2=0x7efef015f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef00ce2d0 0x7efef015f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.350+0000 7efeee800640 1 -- 172.21.15.47:0/2440256467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef00c8bf0 con 0x7efef00d15d0 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.351+0000 7efeee800640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef015f5d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efee8036040 tx=0x7efee8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.351+0000 7efee6000640 1 -- 172.21.15.47:0/2440256467 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee8043020 con 0x7efef00d15d0 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.351+0000 7efee6000640 1 -- 172.21.15.47:0/2440256467 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efee803e450 con 0x7efef00d15d0 2024-04-20T10:31:02.352 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.351+0000 7efee6000640 1 -- 172.21.15.47:0/2440256467 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee803e860 con 0x7efef00d15d0 2024-04-20T10:31:02.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.351+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef015fdf0 con 0x7efef00d15d0 2024-04-20T10:31:02.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.351+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01603b0 con 0x7efef00d15d0 2024-04-20T10:31:02.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.353+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef0150880 con 0x7efef00d15d0 2024-04-20T10:31:02.353 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.353+0000 7efee6000640 1 -- 172.21.15.47:0/2440256467 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7efee8046020 con 0x7efef00d15d0 2024-04-20T10:31:02.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.354+0000 7efee6000640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efecc06c8f0 0x7efecc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:02.354 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.355+0000 7efee6000640 1 -- 172.21.15.47:0/2440256467 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7efee80ae7a0 con 0x7efef00d15d0 2024-04-20T10:31:02.355 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.355+0000 7efeede00640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efecc06c8f0 0x7efecc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:02.356 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.356+0000 7efeede00640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efecc06c8f0 0x7efecc06edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efef00c8e00 tx=0x7efee0008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:02.358 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.359+0000 7efee6000640 1 -- 172.21.15.47:0/2440256467 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7efee804d030 con 0x7efef00d15d0 2024-04-20T10:31:02.575 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.575+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efef0075b30 con 0x7efef00d15d0 2024-04-20T10:31:02.576 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.576+0000 7efee6000640 1 -- 172.21.15.47:0/2440256467 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v58) ==== 80+0+242 (secure 0 0 0) 0x7efee807c8f0 con 0x7efef00d15d0 2024-04-20T10:31:02.576 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6833/793057653 2024-04-21T10:30:54.470443+0000 2024-04-20T10:31:02.576 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6832/793057653 2024-04-21T10:30:54.470443+0000 2024-04-20T10:31:02.576 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6835/3814505199 2024-04-21T10:30:53.464895+0000 2024-04-20T10:31:02.576 INFO:teuthology.orchestra.run.smithi047.stdout:172.21.15.47:6834/3814505199 2024-04-21T10:30:53.464895+0000 2024-04-20T10:31:02.576 INFO:teuthology.orchestra.run.smithi047.stderr:listed 4 entries 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.578+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efecc06c8f0 msgr2=0x7efecc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.578+0000 7efeef200640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efecc06c8f0 0x7efecc06edb0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7efef00c8e00 tx=0x7efee0008040 comp rx=0 tx=0).stop 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.578+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 msgr2=0x7efef015f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.578+0000 7efeef200640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef015f5d0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efee8036040 tx=0x7efee8004a40 comp rx=0 tx=0).stop 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 shutdown_connections 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7efecc06c8f0 0x7efecc06edb0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7efef00d15d0 0x7efef015f5d0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef00ce2d0 0x7efef015f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 --2- 172.21.15.47:0/2440256467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef00b54e0 0x7efef00b0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 >> 172.21.15.47:0/2440256467 conn(0x7efef00ced60 msgr2=0x7efef00b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:02.579 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 shutdown_connections 2024-04-20T10:31:02.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:02.579+0000 7efeef200640 1 -- 172.21.15.47:0/2440256467 wait complete. 2024-04-20T10:31:02.599 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi047.front.sepia.ceph.com... 2024-04-20T10:31:02.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:02.599 DEBUG:teuthology.orchestra.run.smithi047:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-04-20T10:31:02.680 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi047.stderr:ceph-fuse[41728]: fuse finished with error 0 and tester_r 0 2024-04-20T10:31:02.685 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:31:02.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:02.685 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-20T10:31:02.843 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:31:02.844 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:31:02.844 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:02.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:31:02.903 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:31:02.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:02.903 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-20T10:31:02.957 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2024-04-20T10:31:02.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:31:02.958 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:31:02.958 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:31:02.958 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:03.019 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T10:31:03.019 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:31:03.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:03.019 DEBUG:teuthology.orchestra.run.smithi047:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-20T10:31:03.075 INFO:teuthology.orchestra.run.smithi047.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2024-04-20T10:31:03.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:31:03.077 INFO:tasks.cephfs.mount:Terminating background process 2024-04-20T10:31:03.077 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:31:03.077 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:03.133 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:31:03.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:31:03.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:03.134 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-20T10:31:03.187 INFO:teuthology.orchestra.run.smithi175.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2024-04-20T10:31:03.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:31:03.189 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.409+0000 7fb72a800640 1 Processor -- start 2024-04-20T10:31:03.409 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.409+0000 7fb72a800640 1 -- start start 2024-04-20T10:31:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.410+0000 7fb72a800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fb7240b2f70 0x7fb7240b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.411+0000 7fb72a800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb724071a80 con 0x7fb7240adf60 2024-04-20T10:31:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.411+0000 7fb72a800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fb7240ae3f0 con 0x7fb72406b900 2024-04-20T10:31:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.411+0000 7fb72a800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fb724071cf0 con 0x7fb7240b2f70 2024-04-20T10:31:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.411+0000 7fb728a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fb7240adf60 0x7fb72415f5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:57946/0 (socket says 172.21.15.47:57946) 2024-04-20T10:31:03.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.411+0000 7fb728a00640 1 -- 172.21.15.47:0/2403412042 learned_addr learned my addr 172.21.15.47:0/2403412042 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.411+0000 7fb729400640 1 --2- 172.21.15.47:0/2403412042 >> v2:172.21.15.175:3300/0 conn(0x7fb7240b2f70 0x7fb7240b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 624943207 0 0) 0x7fb724071a80 con 0x7fb7240adf60 2024-04-20T10:31:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fb70c003520 con 0x7fb7240adf60 2024-04-20T10:31:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1028988638 0 0) 0x7fb70c003520 con 0x7fb7240adf60 2024-04-20T10:31:03.413 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 >> v2:172.21.15.175:3300/0 conn(0x7fb7240b2f70 msgr2=0x7fb7240b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 --2- 172.21.15.47:0/2403412042 >> v2:172.21.15.175:3300/0 conn(0x7fb7240b2f70 0x7fb7240b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 >> v1:172.21.15.175:6789/0 conn(0x7fb72406b900 legacy=0x7fb72406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb724071fd0 con 0x7fb7240adf60 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fb710001670 con 0x7fb7240adf60 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 >> v1:172.21.15.47:6789/0 conn(0x7fb7240adf60 legacy=0x7fb72415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0039b0 0x7fb70c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c004800 0x7fb70c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0051c0 0x7fb70c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb724071cf0 con 0x7fb70c0051c0 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb70c00c060 con 0x7fb70c004800 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.412+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb70c00c260 con 0x7fb70c0039b0 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb729e00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0051c0 0x7fb70c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.414 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb728a00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c004800 0x7fb70c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb729400640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0039b0 0x7fb70c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb729e00640 1 -- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0039b0 msgr2=0x7fb70c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb729e00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0039b0 0x7fb70c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb729e00640 1 -- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c004800 msgr2=0x7fb70c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb729e00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb70c004800 0x7fb70c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.415 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.413+0000 7fb729e00640 1 -- 172.21.15.47:0/2403412042 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb70c003520 con 0x7fb70c0051c0 2024-04-20T10:31:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.414+0000 7fb729e00640 1 -- 172.21.15.47:0/2403412042 reap_dead start 2024-04-20T10:31:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.414+0000 7fb729e00640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0051c0 0x7fb70c00b9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb71c001e50 tx=0x7fb70c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11c1ba7d44a5d8c server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c001a70 con 0x7fb70c0051c0 2024-04-20T10:31:03.416 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c003bb0 con 0x7fb70c0051c0 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb723e00640 1 -- 172.21.15.47:0/2403412042 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c0050d0 con 0x7fb70c0051c0 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb72a800640 1 -- 172.21.15.47:0/2403412042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0051c0 msgr2=0x7fb70c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb72a800640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0051c0 0x7fb70c00b9c0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fb71c001e50 tx=0x7fb70c005830 comp rx=0 tx=0).stop 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb72a800640 1 -- 172.21.15.47:0/2403412042 shutdown_connections 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb72a800640 1 --2- 172.21.15.47:0/2403412042 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb70c0051c0 0x7fb70c00b9c0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.415+0000 7fb72a800640 1 -- 172.21.15.47:0/2403412042 >> 172.21.15.47:0/2403412042 conn(0x7fb7240d0040 msgr2=0x7fb7240d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.416+0000 7fb72a800640 1 -- 172.21.15.47:0/2403412042 shutdown_connections 2024-04-20T10:31:03.417 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.416+0000 7fb72a800640 1 -- 172.21.15.47:0/2403412042 wait complete. 2024-04-20T10:31:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.418+0000 7fb72a800640 1 Processor -- start 2024-04-20T10:31:03.418 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.418+0000 7fb72a800640 1 -- start start 2024-04-20T10:31:03.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.419+0000 7fb72a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0051c0 0x7fb7241f47e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.419+0000 7fb72a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb724156e40 0x7fb724159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.419+0000 7fb72a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb72409cf10 0x7fb72409d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.420 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.419+0000 7fb72a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240b4320 con 0x7fb72409cf10 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.419+0000 7fb72a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7240736c0 con 0x7fb724156e40 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.419+0000 7fb72a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7240b44a0 con 0x7fb70c0051c0 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb72409cf10 0x7fb72409d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb72409cf10 0x7fb72409d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:33968/0 (socket says 172.21.15.47:33968) 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 -- 172.21.15.47:0/2257379379 learned_addr learned my addr 172.21.15.47:0/2257379379 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729400640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0051c0 0x7fb7241f47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb728a00640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb724156e40 0x7fb724159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 -- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0051c0 msgr2=0x7fb7241f47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0051c0 0x7fb7241f47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 -- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb724156e40 msgr2=0x7fb724159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb724156e40 0x7fb724159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.420+0000 7fb729e00640 1 -- 172.21.15.47:0/2257379379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb72415a420 con 0x7fb72409cf10 2024-04-20T10:31:03.421 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.421+0000 7fb729e00640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb72409cf10 0x7fb72409d390 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb71c001f70 tx=0x7fb71c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.422 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.421+0000 7fb721600640 1 -- 172.21.15.47:0/2257379379 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c006d60 con 0x7fb72409cf10 2024-04-20T10:31:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.421+0000 7fb721600640 1 -- 172.21.15.47:0/2257379379 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb71c007030 con 0x7fb72409cf10 2024-04-20T10:31:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.421+0000 7fb721600640 1 -- 172.21.15.47:0/2257379379 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71c007460 con 0x7fb72409cf10 2024-04-20T10:31:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.421+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7241572c0 con 0x7fb72409cf10 2024-04-20T10:31:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.421+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7241f51d0 con 0x7fb72409cf10 2024-04-20T10:31:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.422+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb724075b60 con 0x7fb72409cf10 2024-04-20T10:31:03.423 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.423+0000 7fb721600640 1 -- 172.21.15.47:0/2257379379 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fb71c007600 con 0x7fb72409cf10 2024-04-20T10:31:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.424+0000 7fb721600640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb70406c8f0 0x7fb70406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.424 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.424+0000 7fb721600640 1 -- 172.21.15.47:0/2257379379 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7fb71c0add00 con 0x7fb72409cf10 2024-04-20T10:31:03.425 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.425+0000 7fb729400640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb70406c8f0 0x7fb70406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.426+0000 7fb729400640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb70406c8f0 0x7fb70406edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb718009990 tx=0x7fb718008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.429 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.429+0000 7fb721600640 1 -- 172.21.15.47:0/2257379379 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fb71c049030 con 0x7fb72409cf10 2024-04-20T10:31:03.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.676+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72406d220 con 0x7fb72409cf10 2024-04-20T10:31:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.678+0000 7fb721600640 1 -- 172.21.15.47:0/2257379379 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5203 (secure 0 0 0) 0x7fb71c033030 con 0x7fb72409cf10 2024-04-20T10:31:03.678 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:03.678 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3774977814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3774977814},{"type":"v1","addr":"172.21.15.47:6835","nonce":3774977814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:57.508717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6833/566678338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":566678338},{"type":"v1","addr":"172.21.15.47:6833","nonce":566678338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6264,"qdb_cluster":[6264]},"id":3}]} 2024-04-20T10:31:03.678 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb70406c8f0 msgr2=0x7fb70406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb70406c8f0 0x7fb70406edb0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb718009990 tx=0x7fb718008040 comp rx=0 tx=0).stop 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb72409cf10 msgr2=0x7fb72409d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb72409cf10 0x7fb72409d390 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fb71c001f70 tx=0x7fb71c0014a0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 shutdown_connections 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fb70406c8f0 0x7fb70406edb0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fb72409cf10 0x7fb72409d390 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb724156e40 0x7fb724159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 --2- 172.21.15.47:0/2257379379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb70c0051c0 0x7fb7241f47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.680+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 >> 172.21.15.47:0/2257379379 conn(0x7fb7240d0040 msgr2=0x7fb7240afbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.681+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 shutdown_connections 2024-04-20T10:31:03.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.681+0000 7fb72a800640 1 -- 172.21.15.47:0/2257379379 wait complete. 2024-04-20T10:31:03.702 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.912+0000 7f47bac00640 1 Processor -- start 2024-04-20T10:31:03.913 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.913+0000 7f47bac00640 1 -- start start 2024-04-20T10:31:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47bac00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47b40cfa00 0x7f47b415ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47bac00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f47b4158780 0x7f47b4158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47bac00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47b40ce4a0 con 0x7f47b40cfde0 2024-04-20T10:31:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47bac00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f47b40d0680 con 0x7f47b40cfa00 2024-04-20T10:31:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47bac00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f47b4159130 con 0x7f47b4158780 2024-04-20T10:31:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47b3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47b40cfa00 0x7f47b415ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.915 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47b3e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47b40cfa00 0x7f47b415ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38312/0 (socket says 172.21.15.47:38312) 2024-04-20T10:31:03.916 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.915+0000 7f47b3e00640 1 -- 172.21.15.47:0/2645969012 learned_addr learned my addr 172.21.15.47:0/2645969012 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.916+0000 7f47b8c00640 1 --2- 172.21.15.47:0/2645969012 >> v2:172.21.15.175:3300/0 conn(0x7f47b4158780 0x7f47b4158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.916+0000 7f47b3e00640 1 -- 172.21.15.47:0/2645969012 >> v2:172.21.15.175:3300/0 conn(0x7f47b4158780 msgr2=0x7f47b4158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.916+0000 7f47b3e00640 1 --2- 172.21.15.47:0/2645969012 >> v2:172.21.15.175:3300/0 conn(0x7f47b4158780 0x7f47b4158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.916+0000 7f47b3e00640 1 -- 172.21.15.47:0/2645969012 >> v1:172.21.15.175:6789/0 conn(0x7f47b40cfde0 legacy=0x7f47b40cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.916+0000 7f47b3e00640 1 -- 172.21.15.47:0/2645969012 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b40ce780 con 0x7f47b40cfa00 2024-04-20T10:31:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.916+0000 7f47b8c00640 1 --2- 172.21.15.47:0/2645969012 >> v2:172.21.15.175:3300/0 conn(0x7f47b4158780 0x7f47b4158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:03.917 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.917+0000 7f47b3e00640 1 --2- 172.21.15.47:0/2645969012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b40cfa00 0x7f47b415ebe0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009e80 tx=0x7f47a402be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e37974a6eb2302 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.918 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.917+0000 7f47b2a00640 1 -- 172.21.15.47:0/2645969012 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a402c9d0 con 0x7f47b40cfa00 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.917+0000 7f47b2a00640 1 -- 172.21.15.47:0/2645969012 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47a402cb70 con 0x7f47b40cfa00 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.917+0000 7f47b2a00640 1 -- 172.21.15.47:0/2645969012 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a40343f0 con 0x7f47b40cfa00 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 -- 172.21.15.47:0/2645969012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b40cfa00 msgr2=0x7f47b415ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 --2- 172.21.15.47:0/2645969012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b40cfa00 0x7f47b415ebe0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009e80 tx=0x7f47a402be30 comp rx=0 tx=0).stop 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 -- 172.21.15.47:0/2645969012 shutdown_connections 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 --2- 172.21.15.47:0/2645969012 >> v2:172.21.15.175:3300/0 conn(0x7f47b4158780 0x7f47b4158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 --2- 172.21.15.47:0/2645969012 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b40cfa00 0x7f47b415ebe0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 -- 172.21.15.47:0/2645969012 >> 172.21.15.47:0/2645969012 conn(0x7f47b4070450 msgr2=0x7f47b4070860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 -- 172.21.15.47:0/2645969012 shutdown_connections 2024-04-20T10:31:03.919 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.918+0000 7f47bac00640 1 -- 172.21.15.47:0/2645969012 wait complete. 2024-04-20T10:31:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.920+0000 7f47bac00640 1 Processor -- start 2024-04-20T10:31:03.920 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.920+0000 7f47bac00640 1 -- start start 2024-04-20T10:31:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47bac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47b40cfa00 0x7f47b406ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47bac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47b40cfde0 0x7f47b406f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47bac00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b4158780 0x7f47b406d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47bac00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f47b40b4350 con 0x7f47b4158780 2024-04-20T10:31:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47bac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47b40ced40 con 0x7f47b40cfa00 2024-04-20T10:31:03.921 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47bac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47b40b0220 con 0x7f47b40cfde0 2024-04-20T10:31:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47b8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b4158780 0x7f47b406d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47b8c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b4158780 0x7f47b406d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38328/0 (socket says 172.21.15.47:38328) 2024-04-20T10:31:03.922 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.921+0000 7f47b8c00640 1 -- 172.21.15.47:0/2736496575 learned_addr learned my addr 172.21.15.47:0/2736496575 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b3e00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47b40cfa00 0x7f47b406ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b8c00640 1 -- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47b40cfde0 msgr2=0x7f47b406f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b8c00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47b40cfde0 0x7f47b406f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b8c00640 1 -- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47b40cfa00 msgr2=0x7f47b406ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b8c00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47b40cfa00 0x7f47b406ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:03.923 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b8c00640 1 -- 172.21.15.47:0/2736496575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b4153720 con 0x7f47b4158780 2024-04-20T10:31:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b3e00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47b40cfa00 0x7f47b406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47b8c00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b4158780 0x7f47b406d4b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c7f0 tx=0x7f47ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47abe00640 1 -- 172.21.15.47:0/2736496575 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac015020 con 0x7f47b4158780 2024-04-20T10:31:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47abe00640 1 -- 172.21.15.47:0/2736496575 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47ac004230 con 0x7f47b4158780 2024-04-20T10:31:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47abe00640 1 -- 172.21.15.47:0/2736496575 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47ac007620 con 0x7f47b4158780 2024-04-20T10:31:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b4152680 con 0x7f47b4158780 2024-04-20T10:31:03.924 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.922+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b4152c40 con 0x7f47b4158780 2024-04-20T10:31:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.923+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b4068980 con 0x7f47b4158780 2024-04-20T10:31:03.925 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.925+0000 7f47abe00640 1 -- 172.21.15.47:0/2736496575 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47ac018020 con 0x7f47b4158780 2024-04-20T10:31:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.926+0000 7f47abe00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479006c920 0x7f479006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.926+0000 7f47abe00640 1 -- 172.21.15.47:0/2736496575 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f47ac08bfe0 con 0x7f47b4158780 2024-04-20T10:31:03.926 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.927+0000 7f47b3e00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479006c920 0x7f479006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:03.928 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.928+0000 7f47b3e00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479006c920 0x7f479006ede0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47a4035000 tx=0x7f47a4030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:03.931 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:03.931+0000 7f47abe00640 1 -- 172.21.15.47:0/2736496575 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f47ac01f020 con 0x7f47b4158780 2024-04-20T10:31:04.136 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.136+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47b415eea0 con 0x7f47b4158780 2024-04-20T10:31:04.139 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.139+0000 7f47abe00640 1 -- 172.21.15.47:0/2736496575 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+17151 (secure 0 0 0) 0x7f47ac05a230 con 0x7f47b4158780 2024-04-20T10:31:04.140 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:04.140 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:54.470486+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:30:19.610070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-20T10:30:20.722858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6833/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6832/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6835/3814505199":"2024-04-21T10:30:53.464895+0000","172.21.15.47:6834/3814505199":"2024-04-21T10:30:53.464895+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:04.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479006c920 msgr2=0x7f479006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479006c920 0x7f479006ede0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47a4035000 tx=0x7f47a4030000 comp rx=0 tx=0).stop 2024-04-20T10:31:04.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b4158780 msgr2=0x7f47b406d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b4158780 0x7f47b406d4b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f47ac00c7f0 tx=0x7f47ac00ccb0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 shutdown_connections 2024-04-20T10:31:04.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479006c920 0x7f479006ede0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.142 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47b4158780 0x7f47b406d4b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47b40cfde0 0x7f47b406f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 --2- 172.21.15.47:0/2736496575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47b40cfa00 0x7f47b406ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 >> 172.21.15.47:0/2736496575 conn(0x7f47b4070450 msgr2=0x7f47b4073460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:04.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 shutdown_connections 2024-04-20T10:31:04.143 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.142+0000 7f47bac00640 1 -- 172.21.15.47:0/2736496575 wait complete. 2024-04-20T10:31:04.161 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-20T10:27:21.096307+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-20T10:30:19.610070+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-20T10:30:20.722858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-20T10:31:04.162 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-20T10:31:04.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.368+0000 7f1bb3200640 1 Processor -- start 2024-04-20T10:31:04.368 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.368+0000 7f1bb3200640 1 -- start start 2024-04-20T10:31:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.370+0000 7f1bb3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 0x7f1bac167390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.370+0000 7f1bb3200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bac15cc20 con 0x7f1bac15a130 2024-04-20T10:31:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.370+0000 7f1bb3200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1bac15cec0 con 0x7f1bac073b80 2024-04-20T10:31:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.370+0000 7f1bb3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1bac15d7f0 con 0x7f1bac071a60 2024-04-20T10:31:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.370+0000 7f1bb1e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f1bac15a130 0x7f1bac073470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59558/0 (socket says 172.21.15.47:59558) 2024-04-20T10:31:04.370 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.370+0000 7f1bb1e00640 1 -- 172.21.15.47:0/72186979 learned_addr learned my addr 172.21.15.47:0/72186979 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.370+0000 7f1bb2800640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 0x7f1bac167390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1161224891 0 0) 0x7f1bac15cc20 con 0x7f1bac15a130 2024-04-20T10:31:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f1b94003520 con 0x7f1bac15a130 2024-04-20T10:31:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2465475985 0 0) 0x7f1b94003520 con 0x7f1bac15a130 2024-04-20T10:31:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 msgr2=0x7f1bac167390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 0x7f1bac167390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.372 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 >> v1:172.21.15.175:6789/0 conn(0x7f1bac073b80 legacy=0x7f1bac06f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac15d1a0 con 0x7f1bac15a130 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f1ba0001670 con 0x7f1bac15a130 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 >> v1:172.21.15.47:6789/0 conn(0x7f1bac15a130 legacy=0x7f1bac073470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b940039e0 0x7f1b94003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 0x7f1b94004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b94004d40 0x7f1b940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac15d7f0 con 0x7f1b940039e0 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b9400bb80 con 0x7f1b94004d40 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b9400bd80 con 0x7f1bac071a60 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.371+0000 7f1bb1e00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b940039e0 0x7f1b94003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb1e00640 1 -- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 msgr2=0x7f1b94004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb2800640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b94004d40 0x7f1b940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb1400640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 0x7f1b94004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb1e00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 0x7f1b94004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.373 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb1e00640 1 -- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b94004d40 msgr2=0x7f1b940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb1e00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b94004d40 0x7f1b940051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb1e00640 1 -- 172.21.15.47:0/72186979 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b94003520 con 0x7f1b940039e0 2024-04-20T10:31:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb2800640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b94004d40 0x7f1b940051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:04.374 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.372+0000 7f1bb1400640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac071a60 0x7f1b94004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.373+0000 7f1bb1e00640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b940039e0 0x7f1b94003e00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c190 tx=0x7f1b94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff808e30a3f0ca6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0007c60 con 0x7f1b940039e0 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1ba0030070 con 0x7f1b940039e0 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb0a00640 1 -- 172.21.15.47:0/72186979 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0005d70 con 0x7f1b940039e0 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb3200640 1 -- 172.21.15.47:0/72186979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b940039e0 msgr2=0x7f1b94003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb3200640 1 --2- 172.21.15.47:0/72186979 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1b940039e0 0x7f1b94003e00 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c190 tx=0x7f1b94005830 comp rx=0 tx=0).stop 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb2800640 1 -- 172.21.15.47:0/72186979 reap_dead start 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb3200640 1 -- 172.21.15.47:0/72186979 shutdown_connections 2024-04-20T10:31:04.375 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb3200640 1 -- 172.21.15.47:0/72186979 >> 172.21.15.47:0/72186979 conn(0x7f1bac0cffa0 msgr2=0x7f1bac0d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb3200640 1 -- 172.21.15.47:0/72186979 shutdown_connections 2024-04-20T10:31:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.374+0000 7f1bb3200640 1 -- 172.21.15.47:0/72186979 wait complete. 2024-04-20T10:31:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.376+0000 7f1bb3200640 1 Processor -- start 2024-04-20T10:31:04.376 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.376+0000 7f1bb3200640 1 -- start start 2024-04-20T10:31:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b940039e0 0x7f1bac149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb3200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bac15dad0 0x7f1bac15df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac09ce30 0x7f1bac09d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb3200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac06f9f0 con 0x7f1bac15dad0 2024-04-20T10:31:04.377 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bac06f870 con 0x7f1b940039e0 2024-04-20T10:31:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1bac06fb70 con 0x7f1bac09ce30 2024-04-20T10:31:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bac15dad0 0x7f1bac15df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb1400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bac15dad0 0x7f1bac15df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38344/0 (socket says 172.21.15.47:38344) 2024-04-20T10:31:04.378 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.377+0000 7f1bb1400640 1 -- 172.21.15.47:0/4181379561 learned_addr learned my addr 172.21.15.47:0/4181379561 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb1e00640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b940039e0 0x7f1bac149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb2800640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac09ce30 0x7f1bac09d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb1400640 1 -- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac09ce30 msgr2=0x7f1bac09d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb1400640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac09ce30 0x7f1bac09d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb1400640 1 -- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b940039e0 msgr2=0x7f1bac149c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.379 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb1400640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b940039e0 0x7f1bac149c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb1400640 1 -- 172.21.15.47:0/4181379561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bac09d7f0 con 0x7f1bac15dad0 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb2800640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac09ce30 0x7f1bac09d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.378+0000 7f1bb1400640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bac15dad0 0x7f1bac15df50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00d560 tx=0x7f1b9c00da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.379+0000 7f1ba6000640 1 -- 172.21.15.47:0/4181379561 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c005480 con 0x7f1bac15dad0 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.379+0000 7f1ba6000640 1 -- 172.21.15.47:0/4181379561 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b9c017070 con 0x7f1bac15dad0 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.379+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bac0b3060 con 0x7f1bac15dad0 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.379+0000 7f1ba6000640 1 -- 172.21.15.47:0/4181379561 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b9c003810 con 0x7f1bac15dad0 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.379+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bac0b3620 con 0x7f1bac15dad0 2024-04-20T10:31:04.380 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.380+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bac075b30 con 0x7f1bac15dad0 2024-04-20T10:31:04.381 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.381+0000 7f1ba6000640 1 -- 172.21.15.47:0/4181379561 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1b9c0039b0 con 0x7f1bac15dad0 2024-04-20T10:31:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.383+0000 7f1ba6000640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b9006c8f0 0x7f1b9006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.383+0000 7f1ba6000640 1 -- 172.21.15.47:0/4181379561 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f1b9c08bbc0 con 0x7f1bac15dad0 2024-04-20T10:31:04.383 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.384+0000 7f1bb1e00640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b9006c8f0 0x7f1b9006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.385 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.385+0000 7f1bb1e00640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b9006c8f0 0x7f1b9006edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c160 tx=0x7f1ba0009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:04.387 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.387+0000 7f1ba6000640 1 -- 172.21.15.47:0/4181379561 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1b9c019310 con 0x7f1bac15dad0 2024-04-20T10:31:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.595+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1bac15c960 con 0x7f1bac15dad0 2024-04-20T10:31:04.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.596+0000 7f1ba6000640 1 -- 172.21.15.47:0/4181379561 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) ==== 93+0+10 (secure 0 0 0) 0x7f1b9c059da0 con 0x7f1bac15dad0 2024-04-20T10:31:04.597 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 1 2024-04-20T10:31:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b9006c8f0 msgr2=0x7f1b9006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b9006c8f0 0x7f1b9006edb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1ba000c160 tx=0x7f1ba0009030 comp rx=0 tx=0).stop 2024-04-20T10:31:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bac15dad0 msgr2=0x7f1bac15df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.600 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bac15dad0 0x7f1bac15df50 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c00d560 tx=0x7f1b9c00da20 comp rx=0 tx=0).stop 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 shutdown_connections 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bac09ce30 0x7f1bac09d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f1b9006c8f0 0x7f1b9006edb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1bac15dad0 0x7f1bac15df50 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 --2- 172.21.15.47:0/4181379561 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b940039e0 0x7f1bac149c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 >> 172.21.15.47:0/4181379561 conn(0x7f1bac0cffa0 msgr2=0x7f1bac0cf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 shutdown_connections 2024-04-20T10:31:04.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.600+0000 7f1bb3200640 1 -- 172.21.15.47:0/4181379561 wait complete. 2024-04-20T10:31:04.620 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-20T10:31:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.839+0000 7f3b99e00640 1 Processor -- start 2024-04-20T10:31:04.839 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.839+0000 7f3b99e00640 1 -- start start 2024-04-20T10:31:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.840+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 0x7f3b9406ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.841+0000 7f3b99e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b940d15b0 con 0x7f3b940d46f0 2024-04-20T10:31:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.841+0000 7f3b99e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3b940d1890 con 0x7f3b940d1010 2024-04-20T10:31:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.841+0000 7f3b99e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b940b6d10 con 0x7f3b9415e8c0 2024-04-20T10:31:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.841+0000 7f3b99400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f3b940d46f0 0x7f3b940cfa10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59566/0 (socket says 172.21.15.47:59566) 2024-04-20T10:31:04.841 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.841+0000 7f3b99400640 1 -- 172.21.15.47:0/752080567 learned_addr learned my addr 172.21.15.47:0/752080567 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.841+0000 7f3b98a00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 0x7f3b9406ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2191487177 0 0) 0x7f3b940d15b0 con 0x7f3b940d46f0 2024-04-20T10:31:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3b78003520 con 0x7f3b940d46f0 2024-04-20T10:31:04.843 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 797589711 0 0) 0x7f3b78003520 con 0x7f3b940d46f0 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 msgr2=0x7f3b9406ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 0x7f3b9406ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 >> v1:172.21.15.175:6789/0 conn(0x7f3b940d1010 legacy=0x7f3b940b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b940d1b70 con 0x7f3b940d46f0 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f3b8c0019f0 con 0x7f3b940d46f0 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 >> v1:172.21.15.47:6789/0 conn(0x7f3b940d46f0 legacy=0x7f3b940cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 0x7f3b780039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b78003fb0 0x7f3b78004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78004d40 0x7f3b780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3b940b6d10 con 0x7f3b78003fb0 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7800bb00 con 0x7f3b78004d40 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b7800bd00 con 0x7f3b9415e8c0 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.842+0000 7f3b8be00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b78003fb0 0x7f3b78004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.844 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.843+0000 7f3b98a00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 0x7f3b780039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.843+0000 7f3b99400640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78004d40 0x7f3b780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.843+0000 7f3b8be00640 1 -- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 msgr2=0x7f3b780039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.843+0000 7f3b8be00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b9415e8c0 0x7f3b780039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.843+0000 7f3b8be00640 1 -- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78004d40 msgr2=0x7f3b780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.843+0000 7f3b8be00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78004d40 0x7f3b780051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.843+0000 7f3b8be00640 1 -- 172.21.15.47:0/752080567 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b78003520 con 0x7f3b78003fb0 2024-04-20T10:31:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.844+0000 7f3b8be00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b78003fb0 0x7f3b78004800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c001750 tx=0x7f3b78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5ba896d5c423e49 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.844+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c001370 con 0x7f3b78003fb0 2024-04-20T10:31:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b7c0052d0 con 0x7f3b78003fb0 2024-04-20T10:31:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b93e00640 1 -- 172.21.15.47:0/752080567 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c003830 con 0x7f3b78003fb0 2024-04-20T10:31:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b99e00640 1 -- 172.21.15.47:0/752080567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b78003fb0 msgr2=0x7f3b78004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b99e00640 1 --2- 172.21.15.47:0/752080567 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b78003fb0 0x7f3b78004800 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c001750 tx=0x7f3b78005830 comp rx=0 tx=0).stop 2024-04-20T10:31:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b99400640 1 -- 172.21.15.47:0/752080567 reap_dead start 2024-04-20T10:31:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b99e00640 1 -- 172.21.15.47:0/752080567 shutdown_connections 2024-04-20T10:31:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b99e00640 1 -- 172.21.15.47:0/752080567 >> 172.21.15.47:0/752080567 conn(0x7f3b9409d6f0 msgr2=0x7f3b940b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b99e00640 1 -- 172.21.15.47:0/752080567 shutdown_connections 2024-04-20T10:31:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.845+0000 7f3b99e00640 1 -- 172.21.15.47:0/752080567 wait complete. 2024-04-20T10:31:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.847+0000 7f3b99e00640 1 Processor -- start 2024-04-20T10:31:04.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.847+0000 7f3b99e00640 1 -- start start 2024-04-20T10:31:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.848+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78003fb0 0x7f3b9406e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.848+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b941f7fa0 0x7f3b94160360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.848+0000 7f3b99e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b941f8400 0x7f3b941f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.848+0000 7f3b99e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3b940d0120 con 0x7f3b941f8400 2024-04-20T10:31:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.848+0000 7f3b99e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3b940b6170 con 0x7f3b78003fb0 2024-04-20T10:31:04.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.848+0000 7f3b99e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3b940d02a0 con 0x7f3b941f7fa0 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b941f8400 0x7f3b941f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b941f8400 0x7f3b941f5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38352/0 (socket says 172.21.15.47:38352) 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 -- 172.21.15.47:0/1215870383 learned_addr learned my addr 172.21.15.47:0/1215870383 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b8be00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b941f7fa0 0x7f3b94160360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b98a00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78003fb0 0x7f3b9406e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 -- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b941f7fa0 msgr2=0x7f3b94160360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b941f7fa0 0x7f3b94160360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 -- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78003fb0 msgr2=0x7f3b9406e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78003fb0 0x7f3b9406e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 -- 172.21.15.47:0/1215870383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9415f2c0 con 0x7f3b941f8400 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b98a00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78003fb0 0x7f3b9406e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:04.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.849+0000 7f3b99400640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b941f8400 0x7f3b941f5900 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3b9408da00 tx=0x7f3b8c007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.850+0000 7f3b91600640 1 -- 172.21.15.47:0/1215870383 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00eb00 con 0x7f3b941f8400 2024-04-20T10:31:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.850+0000 7f3b91600640 1 -- 172.21.15.47:0/1215870383 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3b8c018070 con 0x7f3b941f8400 2024-04-20T10:31:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.850+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9415f5a0 con 0x7f3b941f8400 2024-04-20T10:31:04.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.850+0000 7f3b91600640 1 -- 172.21.15.47:0/1215870383 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c013420 con 0x7f3b941f8400 2024-04-20T10:31:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.850+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9415fb60 con 0x7f3b941f8400 2024-04-20T10:31:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.851+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b94075b30 con 0x7f3b941f8400 2024-04-20T10:31:04.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.852+0000 7f3b91600640 1 -- 172.21.15.47:0/1215870383 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f3b8c0135c0 con 0x7f3b941f8400 2024-04-20T10:31:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.853+0000 7f3b91600640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b7006c920 0x7f3b7006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.853+0000 7f3b91600640 1 -- 172.21.15.47:0/1215870383 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f3b8c08c0f0 con 0x7f3b941f8400 2024-04-20T10:31:04.853 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.854+0000 7f3b98a00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b7006c920 0x7f3b7006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:04.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.855+0000 7f3b98a00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b7006c920 0x7f3b7006ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009a00 tx=0x7f3b84008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:04.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:04.858+0000 7f3b91600640 1 -- 172.21.15.47:0/1215870383 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f3b8c05a240 con 0x7f3b941f8400 2024-04-20T10:31:05.065 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.065+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f3b940d24d0 con 0x7f3b941f8400 2024-04-20T10:31:05.066 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.066+0000 7f3b91600640 1 -- 172.21.15.47:0/1215870383 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v58) ==== 104+0+11 (secure 0 0 0) 0x7f3b8c059b40 con 0x7f3b941f8400 2024-04-20T10:31:05.066 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 32 2024-04-20T10:31:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b7006c920 msgr2=0x7f3b7006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.069 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b7006c920 0x7f3b7006ede0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009a00 tx=0x7f3b84008040 comp rx=0 tx=0).stop 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b941f8400 msgr2=0x7f3b941f5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b941f8400 0x7f3b941f5900 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f3b9408da00 tx=0x7f3b8c007bc0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 shutdown_connections 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f3b7006c920 0x7f3b7006ede0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3b941f8400 0x7f3b941f5900 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3b941f7fa0 0x7f3b94160360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 --2- 172.21.15.47:0/1215870383 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3b78003fb0 0x7f3b9406e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 >> 172.21.15.47:0/1215870383 conn(0x7f3b9409d6f0 msgr2=0x7f3b940b9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 shutdown_connections 2024-04-20T10:31:05.070 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.069+0000 7f3b99e00640 1 -- 172.21.15.47:0/1215870383 wait complete. 2024-04-20T10:31:05.089 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-20T10:31:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.313+0000 7f4d78a00640 1 Processor -- start 2024-04-20T10:31:05.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.313+0000 7f4d78a00640 1 -- start start 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.314+0000 7f4d78a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d74161b40 0x7f4d740c92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d78a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f4d740ae3e0 0x7f4d740c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d78a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d740cdea0 0x7f4d741620d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d78a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f4d740aded0 con 0x7f4d740cdea0 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d78a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f4d740ad2c0 con 0x7f4d740ae3e0 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d78a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d740ad5a0 con 0x7f4d74161b40 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d73200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d740cdea0 0x7f4d741620d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d73200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f4d740cdea0 0x7f4d741620d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38364/0 (socket says 172.21.15.47:38364) 2024-04-20T10:31:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d73200640 1 -- 172.21.15.47:0/2087206961 learned_addr learned my addr 172.21.15.47:0/2087206961 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d72800640 1 --2- 172.21.15.47:0/2087206961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d74161b40 0x7f4d740c92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.316 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.315+0000 7f4d6be00640 1 --2- 172.21.15.47:0/2087206961 >> v2:172.21.15.175:3300/0 conn(0x7f4d740ae3e0 0x7f4d740c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.316+0000 7f4d73200640 1 -- 172.21.15.47:0/2087206961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d74161b40 msgr2=0x7f4d740c92f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.316+0000 7f4d73200640 1 --2- 172.21.15.47:0/2087206961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d74161b40 0x7f4d740c92f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.316+0000 7f4d73200640 1 -- 172.21.15.47:0/2087206961 >> v2:172.21.15.175:3300/0 conn(0x7f4d740ae3e0 msgr2=0x7f4d740c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.316+0000 7f4d73200640 1 --2- 172.21.15.47:0/2087206961 >> v2:172.21.15.175:3300/0 conn(0x7f4d740ae3e0 0x7f4d740c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.316+0000 7f4d73200640 1 -- 172.21.15.47:0/2087206961 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d740ad880 con 0x7f4d740cdea0 2024-04-20T10:31:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.317+0000 7f4d73200640 1 --2- 172.21.15.47:0/2087206961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d740cdea0 0x7f4d741620d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4d7406d6b0 tx=0x7f4d6c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a725c0a8a2846e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.317+0000 7f4d71e00640 1 -- 172.21.15.47:0/2087206961 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c02ca80 con 0x7f4d740cdea0 2024-04-20T10:31:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.317+0000 7f4d71e00640 1 -- 172.21.15.47:0/2087206961 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c02cc20 con 0x7f4d740cdea0 2024-04-20T10:31:05.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.317+0000 7f4d71e00640 1 -- 172.21.15.47:0/2087206961 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c0343f0 con 0x7f4d740cdea0 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 -- 172.21.15.47:0/2087206961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d740cdea0 msgr2=0x7f4d741620d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 --2- 172.21.15.47:0/2087206961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d740cdea0 0x7f4d741620d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f4d7406d6b0 tx=0x7f4d6c02bee0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 -- 172.21.15.47:0/2087206961 shutdown_connections 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 --2- 172.21.15.47:0/2087206961 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d740cdea0 0x7f4d741620d0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 --2- 172.21.15.47:0/2087206961 >> v2:172.21.15.175:3300/0 conn(0x7f4d740ae3e0 0x7f4d740c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 --2- 172.21.15.47:0/2087206961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d74161b40 0x7f4d740c92f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 -- 172.21.15.47:0/2087206961 >> 172.21.15.47:0/2087206961 conn(0x7f4d740cffa0 msgr2=0x7f4d740d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 -- 172.21.15.47:0/2087206961 shutdown_connections 2024-04-20T10:31:05.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.318+0000 7f4d78a00640 1 -- 172.21.15.47:0/2087206961 wait complete. 2024-04-20T10:31:05.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.320+0000 7f4d78a00640 1 Processor -- start 2024-04-20T10:31:05.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.321+0000 7f4d78a00640 1 -- start start 2024-04-20T10:31:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d78a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d740ae3e0 0x7f4d74142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d78a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d740cdea0 0x7f4d7415f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d78a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d74161b40 0x7f4d74148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d78a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f4d74162610 con 0x7f4d74161b40 2024-04-20T10:31:05.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d78a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d740acf50 con 0x7f4d740ae3e0 2024-04-20T10:31:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d78a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d74162790 con 0x7f4d740cdea0 2024-04-20T10:31:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d73200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d74161b40 0x7f4d74148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d73200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d74161b40 0x7f4d74148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38368/0 (socket says 172.21.15.47:38368) 2024-04-20T10:31:05.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.322+0000 7f4d73200640 1 -- 172.21.15.47:0/3742871824 learned_addr learned my addr 172.21.15.47:0/3742871824 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d72800640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d740ae3e0 0x7f4d74142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d6be00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d740cdea0 0x7f4d7415f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d73200640 1 -- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d740cdea0 msgr2=0x7f4d7415f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d73200640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d740cdea0 0x7f4d7415f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d73200640 1 -- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d740ae3e0 msgr2=0x7f4d74142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d73200640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d740ae3e0 0x7f4d74142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d73200640 1 -- 172.21.15.47:0/3742871824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d7415fcc0 con 0x7f4d74161b40 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d73200640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d74161b40 0x7f4d74148730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009f40 tx=0x7f4d6c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d6aa00640 1 -- 172.21.15.47:0/3742871824 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c043020 con 0x7f4d74161b40 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d6aa00640 1 -- 172.21.15.47:0/3742871824 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4d6c03e450 con 0x7f4d74161b40 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d7415ffa0 con 0x7f4d74161b40 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.323+0000 7f4d6aa00640 1 -- 172.21.15.47:0/3742871824 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d6c009510 con 0x7f4d74161b40 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.324+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d74160560 con 0x7f4d74161b40 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.325+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d74075b30 con 0x7f4d74161b40 2024-04-20T10:31:05.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.326+0000 7f4d6aa00640 1 -- 172.21.15.47:0/3742871824 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f4d6c046020 con 0x7f4d74161b40 2024-04-20T10:31:05.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.327+0000 7f4d6aa00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d5006c8f0 0x7f4d5006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.327+0000 7f4d6aa00640 1 -- 172.21.15.47:0/3742871824 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f4d6c0ae8c0 con 0x7f4d74161b40 2024-04-20T10:31:05.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.327+0000 7f4d72800640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d5006c8f0 0x7f4d5006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.329+0000 7f4d72800640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d5006c8f0 0x7f4d5006edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d64006fd0 tx=0x7f4d64008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:05.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.332+0000 7f4d6aa00640 1 -- 172.21.15.47:0/3742871824 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f4d6c04d020 con 0x7f4d74161b40 2024-04-20T10:31:05.540 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.540+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4d740ae110 con 0x7f4d74161b40 2024-04-20T10:31:05.541 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.541+0000 7f4d6aa00640 1 -- 172.21.15.47:0/3742871824 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v58) ==== 100+0+11 (secure 0 0 0) 0x7f4d6c07caa0 con 0x7f4d74161b40 2024-04-20T10:31:05.541 INFO:teuthology.orchestra.run.smithi047.stdout:pg_num: 64 2024-04-20T10:31:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.543+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d5006c8f0 msgr2=0x7f4d5006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.543+0000 7f4d78a00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d5006c8f0 0x7f4d5006edb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d64006fd0 tx=0x7f4d64008040 comp rx=0 tx=0).stop 2024-04-20T10:31:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.543+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d74161b40 msgr2=0x7f4d74148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.543+0000 7f4d78a00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d74161b40 0x7f4d74148730 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c009f40 tx=0x7f4d6c004a40 comp rx=0 tx=0).stop 2024-04-20T10:31:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 shutdown_connections 2024-04-20T10:31:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f4d5006c8f0 0x7f4d5006edb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.544 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f4d74161b40 0x7f4d74148730 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d740cdea0 0x7f4d7415f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 --2- 172.21.15.47:0/3742871824 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d740ae3e0 0x7f4d74142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 >> 172.21.15.47:0/3742871824 conn(0x7f4d740cffa0 msgr2=0x7f4d740afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 shutdown_connections 2024-04-20T10:31:05.545 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.544+0000 7f4d78a00640 1 -- 172.21.15.47:0/3742871824 wait complete. 2024-04-20T10:31:05.565 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:05.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.778+0000 7f300b200640 1 Processor -- start 2024-04-20T10:31:05.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.778+0000 7f300b200640 1 -- start start 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f300b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f300b200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f300c15eec0 0x7f300c15f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f300b200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f300c06ccc0 con 0x7f300c15f840 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f300b200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f300c06d280 con 0x7f300c15eec0 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f300b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f300c06cfa0 con 0x7f300c15b400 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f3009400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f300c15eec0 0x7f300c15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f3009400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f300c15eec0 0x7f300c15f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38380/0 (socket says 172.21.15.47:38380) 2024-04-20T10:31:05.780 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.780+0000 7f3009400640 1 -- 172.21.15.47:0/3220557000 learned_addr learned my addr 172.21.15.47:0/3220557000 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:05.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.781+0000 7f3009400640 1 -- 172.21.15.47:0/3220557000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 msgr2=0x7f300c072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:05.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.781+0000 7f3009e00640 1 --2- 172.21.15.47:0/3220557000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.781+0000 7f3009400640 1 --2- 172.21.15.47:0/3220557000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.781+0000 7f3009400640 1 -- 172.21.15.47:0/3220557000 >> v1:172.21.15.175:6789/0 conn(0x7f300c15f840 legacy=0x7f300c15b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.781+0000 7f3009400640 1 -- 172.21.15.47:0/3220557000 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c06e440 con 0x7f300c15eec0 2024-04-20T10:31:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.782+0000 7f3009400640 1 --2- 172.21.15.47:0/3220557000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c15f300 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009870 tx=0x7f2ff802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1cd9a801bbba61 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:05.782 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.782+0000 7f3008a00640 1 -- 172.21.15.47:0/3220557000 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8035020 con 0x7f300c15eec0 2024-04-20T10:31:05.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.782+0000 7f3008a00640 1 -- 172.21.15.47:0/3220557000 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8031640 con 0x7f300c15eec0 2024-04-20T10:31:05.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.782+0000 7f3008a00640 1 -- 172.21.15.47:0/3220557000 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff802d040 con 0x7f300c15eec0 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 -- 172.21.15.47:0/3220557000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 msgr2=0x7f300c15f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 --2- 172.21.15.47:0/3220557000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c15f300 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8009870 tx=0x7f2ff802b8f0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 -- 172.21.15.47:0/3220557000 shutdown_connections 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 --2- 172.21.15.47:0/3220557000 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c15f300 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 --2- 172.21.15.47:0/3220557000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c072d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 -- 172.21.15.47:0/3220557000 >> 172.21.15.47:0/3220557000 conn(0x7f300c0d0040 msgr2=0x7f300c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 -- 172.21.15.47:0/3220557000 shutdown_connections 2024-04-20T10:31:05.784 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.783+0000 7f300b200640 1 -- 172.21.15.47:0/3220557000 wait complete. 2024-04-20T10:31:05.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.785+0000 7f300b200640 1 Processor -- start 2024-04-20T10:31:05.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.785+0000 7f300b200640 1 -- start start 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.786+0000 7f300b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.786+0000 7f300b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.786+0000 7f300b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c15f840 0x7f300c0b0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.786+0000 7f300b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f300c073240 con 0x7f300c15eec0 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.786+0000 7f300b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f300c15fc90 con 0x7f300c15f840 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.786+0000 7f300b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f300c06fc50 con 0x7f300c15b400 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.786+0000 7f3009400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c152250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38388/0 (socket says 172.21.15.47:38388) 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 -- 172.21.15.47:0/3457552488 learned_addr learned my addr 172.21.15.47:0/3457552488 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f300a800640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c15f840 0x7f300c0b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.788 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009e00640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 -- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 msgr2=0x7f300c15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 -- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c15f840 msgr2=0x7f300c0b0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c15f840 0x7f300c0b0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 -- 172.21.15.47:0/3457552488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f300c15db70 con 0x7f300c15eec0 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f300a800640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c15f840 0x7f300c0b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009e00640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.787+0000 7f3009400640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c152250 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000c00 tx=0x7f2ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.788+0000 7f3002000640 1 -- 172.21.15.47:0/3457552488 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8040070 con 0x7f300c15eec0 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.788+0000 7f3002000640 1 -- 172.21.15.47:0/3457552488 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8039bf0 con 0x7f300c15eec0 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.788+0000 7f3002000640 1 -- 172.21.15.47:0/3457552488 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ff8035040 con 0x7f300c15eec0 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.788+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f300c15de50 con 0x7f300c15eec0 2024-04-20T10:31:05.789 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.788+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f300c15e410 con 0x7f300c15eec0 2024-04-20T10:31:05.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.789+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f300c075b60 con 0x7f300c15eec0 2024-04-20T10:31:05.790 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.790+0000 7f3002000640 1 -- 172.21.15.47:0/3457552488 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2ff8038020 con 0x7f300c15eec0 2024-04-20T10:31:05.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.791+0000 7f3002000640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2fec06c8f0 0x7f2fec06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:05.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.791+0000 7f3002000640 1 -- 172.21.15.47:0/3457552488 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f2ff80ad8f0 con 0x7f300c15eec0 2024-04-20T10:31:05.791 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.792+0000 7f3009e00640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2fec06c8f0 0x7f2fec06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:05.793 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.793+0000 7f3009e00640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2fec06c8f0 0x7f2fec06edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc007f50 tx=0x7f2ffc0074a0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:05.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:05.795+0000 7f3002000640 1 -- 172.21.15.47:0/3457552488 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2ff807bb70 con 0x7f300c15eec0 2024-04-20T10:31:06.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.039+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f300c0cfdb0 con 0x7f300c15eec0 2024-04-20T10:31:06.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.040+0000 7f3002000640 1 -- 172.21.15.47:0/3457552488 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5203 (secure 0 0 0) 0x7f2ff807b470 con 0x7f300c15eec0 2024-04-20T10:31:06.041 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:06.041 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3774977814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3774977814},{"type":"v1","addr":"172.21.15.47:6835","nonce":3774977814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:57.508717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6833/566678338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":566678338},{"type":"v1","addr":"172.21.15.47:6833","nonce":566678338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6264,"qdb_cluster":[6264]},"id":3}]} 2024-04-20T10:31:06.041 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-04-20T10:31:06.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2fec06c8f0 msgr2=0x7f2fec06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2fec06c8f0 0x7f2fec06edb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc007f50 tx=0x7f2ffc0074a0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 msgr2=0x7f300c152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c152250 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8000c00 tx=0x7f2ff80046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 shutdown_connections 2024-04-20T10:31:06.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f300c15f840 0x7f300c0b0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2fec06c8f0 0x7f2fec06edb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f300c15eec0 0x7f300c152250 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 --2- 172.21.15.47:0/3457552488 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f300c15b400 0x7f300c15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 >> 172.21.15.47:0/3457552488 conn(0x7f300c0d0040 msgr2=0x7f300c06c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:06.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 shutdown_connections 2024-04-20T10:31:06.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.043+0000 7f300b200640 1 -- 172.21.15.47:0/3457552488 wait complete. 2024-04-20T10:31:06.064 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:06.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.275+0000 7fce0ce00640 1 Processor -- start 2024-04-20T10:31:06.275 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.275+0000 7fce0ce00640 1 -- start start 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.276+0000 7fce0ce00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce080d0a70 0x7fce0806e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce0ce00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0806f060 con 0x7fce0806dbb0 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce0ce00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fce0806e7c0 con 0x7fce0815e8a0 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce0ce00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fce0806eaa0 con 0x7fce080d0a70 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce05400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce080d0a70 0x7fce0806e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce05400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fce080d0a70 0x7fce0806e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38394/0 (socket says 172.21.15.47:38394) 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce05400640 1 -- 172.21.15.47:0/2400141711 learned_addr learned my addr 172.21.15.47:0/2400141711 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:06.277 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce06800640 1 --1- 172.21.15.47:0/2400141711 >> v1:172.21.15.47:6789/0 conn(0x7fce0806dbb0 0x7fce080ad680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59580/0 (socket says 172.21.15.47:59580) 2024-04-20T10:31:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce05400640 1 -- 172.21.15.47:0/2400141711 >> v1:172.21.15.175:6789/0 conn(0x7fce0815e8a0 legacy=0x7fce080d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce05400640 1 -- 172.21.15.47:0/2400141711 >> v1:172.21.15.47:6789/0 conn(0x7fce0806dbb0 legacy=0x7fce080ad680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.278+0000 7fce05400640 1 -- 172.21.15.47:0/2400141711 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce0806ed80 con 0x7fce080d0a70 2024-04-20T10:31:06.278 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.277+0000 7fce04a00640 1 -- 172.21.15.47:0/2400141711 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1077312910 0 0) 0x7fce0806f060 con 0x7fce0806dbb0 2024-04-20T10:31:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.279+0000 7fce05400640 1 --2- 172.21.15.47:0/2400141711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce0806e1e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009870 tx=0x7fcdf002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc1dcc51a03f440 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.279+0000 7fce04a00640 1 -- 172.21.15.47:0/2400141711 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0034070 con 0x7fce080d0a70 2024-04-20T10:31:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.279+0000 7fce04a00640 1 -- 172.21.15.47:0/2400141711 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf0030860 con 0x7fce080d0a70 2024-04-20T10:31:06.279 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.279+0000 7fce04a00640 1 -- 172.21.15.47:0/2400141711 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0030c30 con 0x7fce080d0a70 2024-04-20T10:31:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.280+0000 7fce0ce00640 1 -- 172.21.15.47:0/2400141711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 msgr2=0x7fce0806e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.280+0000 7fce0ce00640 1 --2- 172.21.15.47:0/2400141711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce0806e1e0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009870 tx=0x7fcdf002b9a0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.280+0000 7fce0ce00640 1 -- 172.21.15.47:0/2400141711 shutdown_connections 2024-04-20T10:31:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.280+0000 7fce0ce00640 1 --2- 172.21.15.47:0/2400141711 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce0806e1e0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.280 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.280+0000 7fce0ce00640 1 -- 172.21.15.47:0/2400141711 >> 172.21.15.47:0/2400141711 conn(0x7fce080b3350 msgr2=0x7fce080b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.280+0000 7fce0ce00640 1 -- 172.21.15.47:0/2400141711 shutdown_connections 2024-04-20T10:31:06.281 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.280+0000 7fce0ce00640 1 -- 172.21.15.47:0/2400141711 wait complete. 2024-04-20T10:31:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.282+0000 7fce0ce00640 1 Processor -- start 2024-04-20T10:31:06.282 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.282+0000 7fce0ce00640 1 -- start start 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce0ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce0806dbb0 0x7fce08155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce0ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce08152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce0ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce0815e8a0 0x7fce081547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce0ce00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fce0806fef0 con 0x7fce080d0a70 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce0ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce0806fd70 con 0x7fce0806dbb0 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce0ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce080add90 con 0x7fce0815e8a0 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce05400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce08152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce05400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce08152e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38406/0 (socket says 172.21.15.47:38406) 2024-04-20T10:31:06.284 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.283+0000 7fce05400640 1 -- 172.21.15.47:0/3311923988 learned_addr learned my addr 172.21.15.47:0/3311923988 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce05e00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce0806dbb0 0x7fce08155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce06800640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce0815e8a0 0x7fce081547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce05400640 1 -- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce0815e8a0 msgr2=0x7fce081547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce05400640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce0815e8a0 0x7fce081547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce05400640 1 -- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce0806dbb0 msgr2=0x7fce08155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce05400640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce0806dbb0 0x7fce08155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.285 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce05400640 1 -- 172.21.15.47:0/3311923988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce08153720 con 0x7fce080d0a70 2024-04-20T10:31:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.284+0000 7fce05400640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce08152e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009870 tx=0x7fcdf0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.285+0000 7fcdfe000640 1 -- 172.21.15.47:0/3311923988 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0048070 con 0x7fce080d0a70 2024-04-20T10:31:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.285+0000 7fcdfe000640 1 -- 172.21.15.47:0/3311923988 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdf0030660 con 0x7fce080d0a70 2024-04-20T10:31:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.285+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce08153a00 con 0x7fce080d0a70 2024-04-20T10:31:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.285+0000 7fcdfe000640 1 -- 172.21.15.47:0/3311923988 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0034040 con 0x7fce080d0a70 2024-04-20T10:31:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.285+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce081542a0 con 0x7fce080d0a70 2024-04-20T10:31:06.286 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.286+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce08153bd0 con 0x7fce080d0a70 2024-04-20T10:31:06.287 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.287+0000 7fcdfe000640 1 -- 172.21.15.47:0/3311923988 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdf004b070 con 0x7fce080d0a70 2024-04-20T10:31:06.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.288+0000 7fcdfe000640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.288 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.289+0000 7fcdfe000640 1 -- 172.21.15.47:0/3311923988 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7fcdf00ade40 con 0x7fce080d0a70 2024-04-20T10:31:06.289 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.289+0000 7fce05e00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.290 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.290+0000 7fce05e00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcde406c8f0 0x7fcde406edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009f60 tx=0x7fcdf80094f0 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.293 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.293+0000 7fcdfe000640 1 -- 172.21.15.47:0/3311923988 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fcdf007c020 con 0x7fce080d0a70 2024-04-20T10:31:06.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.500+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fce08068980 con 0x7fce080d0a70 2024-04-20T10:31:06.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.503+0000 7fcdfe000640 1 -- 172.21.15.47:0/3311923988 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+17151 (secure 0 0 0) 0x7fcdf007b920 con 0x7fce080d0a70 2024-04-20T10:31:06.504 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:06.504 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:54.470486+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:30:19.610070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-20T10:30:20.722858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6833/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6832/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6835/3814505199":"2024-04-21T10:30:53.464895+0000","172.21.15.47:6834/3814505199":"2024-04-21T10:30:53.464895+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcde406c8f0 msgr2=0x7fcde406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcde406c8f0 0x7fcde406edb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8009f60 tx=0x7fcdf80094f0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 msgr2=0x7fce08152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce08152e20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009870 tx=0x7fcdf0004730 comp rx=0 tx=0).stop 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 shutdown_connections 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce0815e8a0 0x7fce081547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcde406c8f0 0x7fcde406edb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fce080d0a70 0x7fce08152e20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 --2- 172.21.15.47:0/3311923988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce0806dbb0 0x7fce08155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.506+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 >> 172.21.15.47:0/3311923988 conn(0x7fce080b3350 msgr2=0x7fce0815cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.507+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 shutdown_connections 2024-04-20T10:31:06.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.507+0000 7fce0ce00640 1 -- 172.21.15.47:0/3311923988 wait complete. 2024-04-20T10:31:06.527 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T10:31:06.527 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:06.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.725+0000 7fbe91c00640 1 Processor -- start 2024-04-20T10:31:06.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.725+0000 7fbe91c00640 1 -- start start 2024-04-20T10:31:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91c00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbe8c06cde0 0x7fbe8c0d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe8c0b3330 0x7fbe8c0b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91c00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbe8c06c190 con 0x7fbe8c161d50 2024-04-20T10:31:06.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c162130 con 0x7fbe8c0b3330 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91c00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbe8c06c470 con 0x7fbe8c06cde0 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe8c0b3330 0x7fbe8c0b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbe8c0b3330 0x7fbe8c0b3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38420/0 (socket says 172.21.15.47:38420) 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe91200640 1 -- 172.21.15.47:0/1509935970 learned_addr learned my addr 172.21.15.47:0/1509935970 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.727+0000 7fbe8b400640 1 --2- 172.21.15.47:0/1509935970 >> v2:172.21.15.175:3300/0 conn(0x7fbe8c06cde0 0x7fbe8c0d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.728+0000 7fbe91200640 1 -- 172.21.15.47:0/1509935970 >> v2:172.21.15.175:3300/0 conn(0x7fbe8c06cde0 msgr2=0x7fbe8c0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.728+0000 7fbe91200640 1 --2- 172.21.15.47:0/1509935970 >> v2:172.21.15.175:3300/0 conn(0x7fbe8c06cde0 0x7fbe8c0d38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.728+0000 7fbe91200640 1 -- 172.21.15.47:0/1509935970 >> v1:172.21.15.175:6789/0 conn(0x7fbe8c161d50 legacy=0x7fbe8c06c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.728+0000 7fbe91200640 1 -- 172.21.15.47:0/1509935970 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c0d2e00 con 0x7fbe8c0b3330 2024-04-20T10:31:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.729+0000 7fbe91200640 1 --2- 172.21.15.47:0/1509935970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c0b3330 0x7fbe8c0b3880 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0d1f00 tx=0x7fbe8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=601ce524919d94e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.729+0000 7fbe8aa00640 1 -- 172.21.15.47:0/1509935970 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84035020 con 0x7fbe8c0b3330 2024-04-20T10:31:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.729+0000 7fbe8aa00640 1 -- 172.21.15.47:0/1509935970 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe84031870 con 0x7fbe8c0b3330 2024-04-20T10:31:06.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.729+0000 7fbe8aa00640 1 -- 172.21.15.47:0/1509935970 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe84031c40 con 0x7fbe8c0b3330 2024-04-20T10:31:06.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 -- 172.21.15.47:0/1509935970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c0b3330 msgr2=0x7fbe8c0b3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 --2- 172.21.15.47:0/1509935970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c0b3330 0x7fbe8c0b3880 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0d1f00 tx=0x7fbe8402b910 comp rx=0 tx=0).stop 2024-04-20T10:31:06.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 -- 172.21.15.47:0/1509935970 shutdown_connections 2024-04-20T10:31:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 --2- 172.21.15.47:0/1509935970 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c0b3330 0x7fbe8c0b3880 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 --2- 172.21.15.47:0/1509935970 >> v2:172.21.15.175:3300/0 conn(0x7fbe8c06cde0 0x7fbe8c0d38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 -- 172.21.15.47:0/1509935970 >> 172.21.15.47:0/1509935970 conn(0x7fbe8c0d4450 msgr2=0x7fbe8c0d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 -- 172.21.15.47:0/1509935970 shutdown_connections 2024-04-20T10:31:06.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.730+0000 7fbe91c00640 1 -- 172.21.15.47:0/1509935970 wait complete. 2024-04-20T10:31:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.732+0000 7fbe91c00640 1 Processor -- start 2024-04-20T10:31:06.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.732+0000 7fbe91c00640 1 -- start start 2024-04-20T10:31:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.733+0000 7fbe91c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c06cde0 0x7fbe8c1f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.733+0000 7fbe91c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe8c0b3330 0x7fbe8c15b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.733+0000 7fbe91c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe8c161d50 0x7fbe8c15b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.733+0000 7fbe91c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0b7660 con 0x7fbe8c06cde0 2024-04-20T10:31:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.733+0000 7fbe91c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbe8c0d33c0 con 0x7fbe8c161d50 2024-04-20T10:31:06.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.733+0000 7fbe91c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbe8c0b77e0 con 0x7fbe8c0b3330 2024-04-20T10:31:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c06cde0 0x7fbe8c1f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c06cde0 0x7fbe8c1f1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38432/0 (socket says 172.21.15.47:38432) 2024-04-20T10:31:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 -- 172.21.15.47:0/2899349250 learned_addr learned my addr 172.21.15.47:0/2899349250 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe91200640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe8c161d50 0x7fbe8c15b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8b400640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe8c0b3330 0x7fbe8c15b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 -- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe8c0b3330 msgr2=0x7fbe8c15b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe8c0b3330 0x7fbe8c15b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 -- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe8c161d50 msgr2=0x7fbe8c15b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe8c161d50 0x7fbe8c15b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe8be00640 1 -- 172.21.15.47:0/2899349250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe8c1f52d0 con 0x7fbe8c06cde0 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.734+0000 7fbe91200640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe8c161d50 0x7fbe8c15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.735+0000 7fbe8be00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c06cde0 0x7fbe8c1f1930 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00ba50 tx=0x7fbe7c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.735+0000 7fbe83e00640 1 -- 172.21.15.47:0/2899349250 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c019070 con 0x7fbe8c06cde0 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.735+0000 7fbe83e00640 1 -- 172.21.15.47:0/2899349250 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbe7c00cc20 con 0x7fbe8c06cde0 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.735+0000 7fbe83e00640 1 -- 172.21.15.47:0/2899349250 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe7c014680 con 0x7fbe8c06cde0 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.735+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe8c1f4510 con 0x7fbe8c06cde0 2024-04-20T10:31:06.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.735+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe8c1f4ad0 con 0x7fbe8c06cde0 2024-04-20T10:31:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.737+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe8c075b60 con 0x7fbe8c06cde0 2024-04-20T10:31:06.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.737+0000 7fbe83e00640 1 -- 172.21.15.47:0/2899349250 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbe7c01c070 con 0x7fbe8c06cde0 2024-04-20T10:31:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.739+0000 7fbe83e00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbe6806c8f0 0x7fbe6806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.739+0000 7fbe83e00640 1 -- 172.21.15.47:0/2899349250 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7fbe7c08ba30 con 0x7fbe8c06cde0 2024-04-20T10:31:06.739 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.739+0000 7fbe8b400640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbe6806c8f0 0x7fbe6806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:06.740 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.741+0000 7fbe8b400640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbe6806c8f0 0x7fbe6806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0c89f0 tx=0x7fbe74008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:06.743 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.743+0000 7fbe83e00640 1 -- 172.21.15.47:0/2899349250 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fbe7c0595d0 con 0x7fbe8c06cde0 2024-04-20T10:31:06.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.988+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe8c0b9590 con 0x7fbe8c06cde0 2024-04-20T10:31:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.990+0000 7fbe83e00640 1 -- 172.21.15.47:0/2899349250 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5203 (secure 0 0 0) 0x7fbe7c05d8d0 con 0x7fbe8c06cde0 2024-04-20T10:31:06.990 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:06.990 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3774977814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3774977814},{"type":"v1","addr":"172.21.15.47:6835","nonce":3774977814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:57.508717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6833/566678338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":566678338},{"type":"v1","addr":"172.21.15.47:6833","nonce":566678338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6264,"qdb_cluster":[6264]},"id":3}]} 2024-04-20T10:31:06.990 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbe6806c8f0 msgr2=0x7fbe6806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbe6806c8f0 0x7fbe6806edb0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c0c89f0 tx=0x7fbe74008040 comp rx=0 tx=0).stop 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c06cde0 msgr2=0x7fbe8c1f1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c06cde0 0x7fbe8c1f1930 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbe7c00ba50 tx=0x7fbe7c00bf10 comp rx=0 tx=0).stop 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 shutdown_connections 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbe8c161d50 0x7fbe8c15b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbe8c0b3330 0x7fbe8c15b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbe6806c8f0 0x7fbe6806edb0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 --2- 172.21.15.47:0/2899349250 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbe8c06cde0 0x7fbe8c1f1930 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.992+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 >> 172.21.15.47:0/2899349250 conn(0x7fbe8c0d4450 msgr2=0x7fbe8c0676c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.993+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 shutdown_connections 2024-04-20T10:31:06.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:06.993+0000 7fbe91c00640 1 -- 172.21.15.47:0/2899349250 wait complete. 2024-04-20T10:31:07.014 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:07.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.223+0000 7f57fc200640 1 Processor -- start 2024-04-20T10:31:07.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.224+0000 7f57fc200640 1 -- start start 2024-04-20T10:31:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.225+0000 7f57fc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4072cc0 0x7f57f40ad8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.225+0000 7f57fc200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f406e840 con 0x7f57f4071b50 2024-04-20T10:31:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.225+0000 7f57fc200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f57f415b350 con 0x7f57f415abb0 2024-04-20T10:31:07.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.225+0000 7f57fc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57f406eb00 con 0x7f57f4072cc0 2024-04-20T10:31:07.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.226+0000 7f57fa200640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f57f4071b50 0x7f57f40adde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59588/0 (socket says 172.21.15.47:59588) 2024-04-20T10:31:07.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.226+0000 7f57fa200640 1 -- 172.21.15.47:0/3916675651 learned_addr learned my addr 172.21.15.47:0/3916675651 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:07.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.226+0000 7f57f8e00640 1 --2- 172.21.15.47:0/3916675651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4072cc0 0x7f57f40ad8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.226+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3727358384 0 0) 0x7f57f406e840 con 0x7f57f4071b50 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.226+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57d8005730 con 0x7f57f4071b50 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.227+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 <== mon.1 v1:172.21.15.175:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 312464632 0 0) 0x7f57f415b350 con 0x7f57f415abb0 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.227+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 --> v1:172.21.15.175:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f57f406e840 con 0x7f57f415abb0 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.227+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4121401873 0 0) 0x7f57d8005730 con 0x7f57f4071b50 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.227+0000 7f57f8e00640 1 -- 172.21.15.47:0/3916675651 >> v1:172.21.15.175:6789/0 conn(0x7f57f415abb0 legacy=0x7f57f40725b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.227+0000 7f57f8e00640 1 -- 172.21.15.47:0/3916675651 >> v1:172.21.15.47:6789/0 conn(0x7f57f4071b50 legacy=0x7f57f40adde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.227+0000 7f57f8e00640 1 -- 172.21.15.47:0/3916675651 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f415b630 con 0x7f57f4072cc0 2024-04-20T10:31:07.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.228+0000 7f57f8e00640 1 --2- 172.21.15.47:0/3916675651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4072cc0 0x7f57f40ad8a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f57e00097d0 tx=0x7f57e002b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba36bec457747e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.229+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57e0037070 con 0x7f57f4072cc0 2024-04-20T10:31:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.229+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57e0030510 con 0x7f57f4072cc0 2024-04-20T10:31:07.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.229+0000 7f57f3e00640 1 -- 172.21.15.47:0/3916675651 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0019f0 con 0x7f57f4072cc0 2024-04-20T10:31:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.229+0000 7f57fc200640 1 -- 172.21.15.47:0/3916675651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4072cc0 msgr2=0x7f57f40ad8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.229+0000 7f57fc200640 1 --2- 172.21.15.47:0/3916675651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4072cc0 0x7f57f40ad8a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f57e00097d0 tx=0x7f57e002b800 comp rx=0 tx=0).stop 2024-04-20T10:31:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.230+0000 7f57fc200640 1 -- 172.21.15.47:0/3916675651 shutdown_connections 2024-04-20T10:31:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.230+0000 7f57fc200640 1 --2- 172.21.15.47:0/3916675651 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4072cc0 0x7f57f40ad8a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.230+0000 7f57fc200640 1 -- 172.21.15.47:0/3916675651 >> 172.21.15.47:0/3916675651 conn(0x7f57f40b3350 msgr2=0x7f57f40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.230+0000 7f57fc200640 1 -- 172.21.15.47:0/3916675651 shutdown_connections 2024-04-20T10:31:07.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.230+0000 7f57fc200640 1 -- 172.21.15.47:0/3916675651 wait complete. 2024-04-20T10:31:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.231+0000 7f57fc200640 1 Processor -- start 2024-04-20T10:31:07.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.231+0000 7f57fc200640 1 -- start start 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.232+0000 7f57fc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4071b50 0x7f57f4152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.232+0000 7f57fc200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f4072cc0 0x7f57f4158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.232+0000 7f57fc200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f415abb0 0x7f57f415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.232+0000 7f57fc200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f57f415bbf0 con 0x7f57f415abb0 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.232+0000 7f57fc200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57f415bf60 con 0x7f57f4072cc0 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.232+0000 7f57fc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57f415af90 con 0x7f57f4071b50 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f415abb0 0x7f57f415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f415abb0 0x7f57f415d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38448/0 (socket says 172.21.15.47:38448) 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 -- 172.21.15.47:0/2019666599 learned_addr learned my addr 172.21.15.47:0/2019666599 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57f9800640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4071b50 0x7f57f4152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 -- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4071b50 msgr2=0x7f57f4152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4071b50 0x7f57f4152770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 -- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f4072cc0 msgr2=0x7f57f4158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57f8e00640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f4072cc0 0x7f57f4158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f4072cc0 0x7f57f4158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.233+0000 7f57fa200640 1 -- 172.21.15.47:0/2019666599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57f4159590 con 0x7f57f415abb0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.234+0000 7f57fa200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f415abb0 0x7f57f415d160 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00cc90 tx=0x7f57ec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.234+0000 7f57f1600640 1 -- 172.21.15.47:0/2019666599 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec012400 con 0x7f57f415abb0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.234+0000 7f57f1600640 1 -- 172.21.15.47:0/2019666599 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f57ec0125c0 con 0x7f57f415abb0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.234+0000 7f57f1600640 1 -- 172.21.15.47:0/2019666599 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0129b0 con 0x7f57f415abb0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.234+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57f4159870 con 0x7f57f415abb0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.234+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57f4159e30 con 0x7f57f415abb0 2024-04-20T10:31:07.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.235+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57f4068980 con 0x7f57f415abb0 2024-04-20T10:31:07.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.236+0000 7f57f1600640 1 -- 172.21.15.47:0/2019666599 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f57ec012b50 con 0x7f57f415abb0 2024-04-20T10:31:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.237+0000 7f57f1600640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.237+0000 7f57f1600640 1 -- 172.21.15.47:0/2019666599 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f57ec08c680 con 0x7f57f415abb0 2024-04-20T10:31:07.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.238+0000 7f57f9800640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.239+0000 7f57f9800640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f57e40011e0 tx=0x7f57e4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:07.242 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.242+0000 7f57f1600640 1 -- 172.21.15.47:0/2019666599 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f57ec0182b0 con 0x7f57f415abb0 2024-04-20T10:31:07.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.486+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57f40cfe90 con 0x7f57f415abb0 2024-04-20T10:31:07.486 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.487+0000 7f57f1600640 1 -- 172.21.15.47:0/2019666599 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+5203 (secure 0 0 0) 0x7f57ec05a7d0 con 0x7f57f415abb0 2024-04-20T10:31:07.487 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:07.487 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":41,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3774977814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3774977814},{"type":"v1","addr":"172.21.15.47:6835","nonce":3774977814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[{"mdsmap":{"epoch":41,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-20T10:30:21.745506+0000","modified":"2024-04-20T10:30:57.508717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":58,"snap_rstat":true,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6264},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6264":{"gid":6264,"name":"a","rank":0,"incarnation":38,"state":"up:active","state_seq":4,"addr":"172.21.15.47:6833/566678338","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":566678338},{"type":"v1","addr":"172.21.15.47:6833","nonce":566678338}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6264,"qdb_cluster":[6264]},"id":3}]} 2024-04-20T10:31:07.487 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 41 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 msgr2=0x7f57d406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f57e40011e0 tx=0x7f57e4009040 comp rx=0 tx=0).stop 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f415abb0 msgr2=0x7f57f415d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f415abb0 0x7f57f415d160 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f57ec00cc90 tx=0x7f57ec00a430 comp rx=0 tx=0).stop 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 shutdown_connections 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f57d406c8f0 0x7f57d406edb0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f57f415abb0 0x7f57f415d160 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57f4072cc0 0x7f57f4158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.489+0000 7f57fc200640 1 --2- 172.21.15.47:0/2019666599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57f4071b50 0x7f57f4152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.490+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 >> 172.21.15.47:0/2019666599 conn(0x7f57f40b3350 msgr2=0x7f57f4155950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.490+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 shutdown_connections 2024-04-20T10:31:07.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.490+0000 7f57fc200640 1 -- 172.21.15.47:0/2019666599 wait complete. 2024-04-20T10:31:07.509 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:07.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.718+0000 7f6d87e00640 1 Processor -- start 2024-04-20T10:31:07.718 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.718+0000 7f6d87e00640 1 -- start start 2024-04-20T10:31:07.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.719+0000 7f6d87e00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d880d15d0 0x7f6d881695b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.719+0000 7f6d87e00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d881623c0 con 0x7f6d880d1e00 2024-04-20T10:31:07.719 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.719+0000 7f6d87e00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d881626a0 con 0x7f6d8815e680 2024-04-20T10:31:07.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.719+0000 7f6d87e00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6d88162fd0 con 0x7f6d880d15d0 2024-04-20T10:31:07.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.719+0000 7f6d86000640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f6d880d1e00 0x7f6d880d0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59596/0 (socket says 172.21.15.47:59596) 2024-04-20T10:31:07.720 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.719+0000 7f6d86000640 1 -- 172.21.15.47:0/4032710926 learned_addr learned my addr 172.21.15.47:0/4032710926 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d87400640 1 --2- 172.21.15.47:0/4032710926 >> v2:172.21.15.175:3300/0 conn(0x7f6d880d15d0 0x7f6d881695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1070695365 0 0) 0x7f6d881623c0 con 0x7f6d880d1e00 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6d6c003520 con 0x7f6d880d1e00 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 1253589466 0 0) 0x7f6d6c003520 con 0x7f6d880d1e00 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 >> v2:172.21.15.175:3300/0 conn(0x7f6d880d15d0 msgr2=0x7f6d881695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 --2- 172.21.15.47:0/4032710926 >> v2:172.21.15.175:3300/0 conn(0x7f6d880d15d0 0x7f6d881695b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 >> v1:172.21.15.175:6789/0 conn(0x7f6d8815e680 legacy=0x7f6d880b5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d88162980 con 0x7f6d880d1e00 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.720+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f6d70001670 con 0x7f6d880d1e00 2024-04-20T10:31:07.722 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 >> v1:172.21.15.47:6789/0 conn(0x7f6d880d1e00 legacy=0x7f6d880d0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d85600640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0039b0 0x7f6d6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d85600640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6c004800 0x7f6d6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d85600640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6c0051c0 0x7f6d6c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d88162fd0 con 0x7f6d6c0051c0 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6c00c060 con 0x7f6d6c0039b0 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6c00c260 con 0x7f6d6c004800 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d87400640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6c0051c0 0x7f6d6c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d86a00640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0039b0 0x7f6d6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d86000640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6c004800 0x7f6d6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d87400640 1 -- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6c004800 msgr2=0x7f6d6c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d87400640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6c004800 0x7f6d6c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d87400640 1 -- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0039b0 msgr2=0x7f6d6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d87400640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0039b0 0x7f6d6c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.721+0000 7f6d87400640 1 -- 172.21.15.47:0/4032710926 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6c003520 con 0x7f6d6c0051c0 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.722+0000 7f6d87400640 1 -- 172.21.15.47:0/4032710926 reap_dead start 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d87400640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6c0051c0 0x7f6d6c00b9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009ac0 tx=0x7f6d6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b1736a8b5d8f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80009690 con 0x7f6d6c0051c0 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d80004020 con 0x7f6d6c0051c0 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d85600640 1 -- 172.21.15.47:0/4032710926 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80004410 con 0x7f6d6c0051c0 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d87e00640 1 -- 172.21.15.47:0/4032710926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6c0051c0 msgr2=0x7f6d6c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d87e00640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6c0051c0 0x7f6d6c00b9c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f6d80009ac0 tx=0x7f6d6c005830 comp rx=0 tx=0).stop 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d87e00640 1 -- 172.21.15.47:0/4032710926 shutdown_connections 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d87e00640 1 --2- 172.21.15.47:0/4032710926 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6c0051c0 0x7f6d6c00b9c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.723+0000 7f6d87e00640 1 -- 172.21.15.47:0/4032710926 >> 172.21.15.47:0/4032710926 conn(0x7f6d880ced60 msgr2=0x7f6d880cf170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.724+0000 7f6d87e00640 1 -- 172.21.15.47:0/4032710926 shutdown_connections 2024-04-20T10:31:07.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.724+0000 7f6d87e00640 1 -- 172.21.15.47:0/4032710926 wait complete. 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.725+0000 7f6d87e00640 1 Processor -- start 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.725+0000 7f6d87e00640 1 -- start start 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.726+0000 7f6d87e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0051c0 0x7f6d880c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.726+0000 7f6d87e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d880c6ee0 0x7f6d8815ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.726+0000 7f6d87e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8814b680 0x7f6d880cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.726+0000 7f6d87e00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880d1040 con 0x7f6d8814b680 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.726+0000 7f6d87e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d880d0ec0 con 0x7f6d6c0051c0 2024-04-20T10:31:07.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.726+0000 7f6d87e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d880d11c0 con 0x7f6d880c6ee0 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8814b680 0x7f6d880cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8814b680 0x7f6d880cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38460/0 (socket says 172.21.15.47:38460) 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 -- 172.21.15.47:0/3301196722 learned_addr learned my addr 172.21.15.47:0/3301196722 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d86000640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d880c6ee0 0x7f6d8815ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d86a00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0051c0 0x7f6d880c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 -- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d880c6ee0 msgr2=0x7f6d8815ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d880c6ee0 0x7f6d8815ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 -- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0051c0 msgr2=0x7f6d880c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0051c0 0x7f6d880c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.727+0000 7f6d87400640 1 -- 172.21.15.47:0/3301196722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d880c7430 con 0x7f6d8814b680 2024-04-20T10:31:07.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.728+0000 7f6d87400640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8814b680 0x7f6d880cab70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6d8002c380 tx=0x7f6d80004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:07.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.728+0000 7f6d7ea00640 1 -- 172.21.15.47:0/3301196722 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d8002cbd0 con 0x7f6d8814b680 2024-04-20T10:31:07.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.728+0000 7f6d7ea00640 1 -- 172.21.15.47:0/3301196722 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d8002cd90 con 0x7f6d8814b680 2024-04-20T10:31:07.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.728+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d880c7710 con 0x7f6d8814b680 2024-04-20T10:31:07.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.728+0000 7f6d7ea00640 1 -- 172.21.15.47:0/3301196722 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d80003850 con 0x7f6d8814b680 2024-04-20T10:31:07.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.728+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d880c7cd0 con 0x7f6d8814b680 2024-04-20T10:31:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.729+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d88075b30 con 0x7f6d8814b680 2024-04-20T10:31:07.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.730+0000 7f6d7ea00640 1 -- 172.21.15.47:0/3301196722 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d800039f0 con 0x7f6d8814b680 2024-04-20T10:31:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.731+0000 7f6d7ea00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d6406c8f0 0x7f6d6406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:07.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.731+0000 7f6d7ea00640 1 -- 172.21.15.47:0/3301196722 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7f6d800acb40 con 0x7f6d8814b680 2024-04-20T10:31:07.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.732+0000 7f6d86a00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d6406c8f0 0x7f6d6406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:07.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.733+0000 7f6d86a00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d6406c8f0 0x7f6d6406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007fd0 tx=0x7f6d78009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:07.735 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.736+0000 7f6d7ea00640 1 -- 172.21.15.47:0/3301196722 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f6d8007adc0 con 0x7f6d8814b680 2024-04-20T10:31:07.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.934+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6d880d1b30 con 0x7f6d8814b680 2024-04-20T10:31:07.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.937+0000 7f6d7ea00640 1 -- 172.21.15.47:0/3301196722 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+17151 (secure 0 0 0) 0x7f6d8007a6c0 con 0x7f6d8814b680 2024-04-20T10:31:07.938 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:07.938 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":58,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:30:54.470486+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-20T10:30:19.610070+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-20T10:30:20.722858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6833/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6832/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6835/3814505199":"2024-04-21T10:30:53.464895+0000","172.21.15.47:6834/3814505199":"2024-04-21T10:30:53.464895+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:07.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d6406c8f0 msgr2=0x7f6d6406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d6406c8f0 0x7f6d6406edb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6d78007fd0 tx=0x7f6d78009040 comp rx=0 tx=0).stop 2024-04-20T10:31:07.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8814b680 msgr2=0x7f6d880cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:07.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8814b680 0x7f6d880cab70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f6d8002c380 tx=0x7f6d80004730 comp rx=0 tx=0).stop 2024-04-20T10:31:07.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 shutdown_connections 2024-04-20T10:31:07.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d6406c8f0 0x7f6d6406edb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d8814b680 0x7f6d880cab70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d880c6ee0 0x7f6d8815ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 --2- 172.21.15.47:0/3301196722 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d6c0051c0 0x7f6d880c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 >> 172.21.15.47:0/3301196722 conn(0x7f6d880ced60 msgr2=0x7f6d8815d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 shutdown_connections 2024-04-20T10:31:07.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:07.940+0000 7f6d87e00640 1 -- 172.21.15.47:0/3301196722 wait complete. 2024-04-20T10:31:07.960 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-20T10:31:08.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.163+0000 7fe9d9800640 1 Processor -- start 2024-04-20T10:31:08.163 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.163+0000 7fe9d9800640 1 -- start start 2024-04-20T10:31:08.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.165+0000 7fe9d9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d40b2f70 0x7fe9d40b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.165+0000 7fe9d9800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9d40adf60 0x7fe9d415f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.165+0000 7fe9d9800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe9d4071a80 con 0x7fe9d415faf0 2024-04-20T10:31:08.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.165+0000 7fe9d9800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe9d40ae3c0 con 0x7fe9d40adf60 2024-04-20T10:31:08.165 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.165+0000 7fe9d9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d4071cf0 con 0x7fe9d40b2f70 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.165+0000 7fe9d3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9d40adf60 0x7fe9d415f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.166+0000 7fe9d3400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe9d40adf60 0x7fe9d415f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38470/0 (socket says 172.21.15.47:38470) 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.166+0000 7fe9d3400640 1 -- 172.21.15.47:0/2115030530 learned_addr learned my addr 172.21.15.47:0/2115030530 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.166+0000 7fe9d3e00640 1 --2- 172.21.15.47:0/2115030530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d40b2f70 0x7fe9d40b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.166+0000 7fe9d3400640 1 -- 172.21.15.47:0/2115030530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d40b2f70 msgr2=0x7fe9d40b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.166+0000 7fe9d3400640 1 --2- 172.21.15.47:0/2115030530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d40b2f70 0x7fe9d40b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.166+0000 7fe9d3400640 1 -- 172.21.15.47:0/2115030530 >> v1:172.21.15.175:6789/0 conn(0x7fe9d415faf0 legacy=0x7fe9d406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.166 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.166+0000 7fe9d3400640 1 -- 172.21.15.47:0/2115030530 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d4071fd0 con 0x7fe9d40adf60 2024-04-20T10:31:08.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.167+0000 7fe9d3400640 1 --2- 172.21.15.47:0/2115030530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40adf60 0x7fe9d415f5b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009870 tx=0x7fe9c002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962ad814d4f79aa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d2a00640 1 -- 172.21.15.47:0/2115030530 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0034070 con 0x7fe9d40adf60 2024-04-20T10:31:08.167 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d2a00640 1 -- 172.21.15.47:0/2115030530 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c0030840 con 0x7fe9d40adf60 2024-04-20T10:31:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d2a00640 1 -- 172.21.15.47:0/2115030530 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0030c10 con 0x7fe9d40adf60 2024-04-20T10:31:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 -- 172.21.15.47:0/2115030530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40adf60 msgr2=0x7fe9d415f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.168 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 --2- 172.21.15.47:0/2115030530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40adf60 0x7fe9d415f5b0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0009870 tx=0x7fe9c002b980 comp rx=0 tx=0).stop 2024-04-20T10:31:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 -- 172.21.15.47:0/2115030530 shutdown_connections 2024-04-20T10:31:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 --2- 172.21.15.47:0/2115030530 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40adf60 0x7fe9d415f5b0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 --2- 172.21.15.47:0/2115030530 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d40b2f70 0x7fe9d40b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 -- 172.21.15.47:0/2115030530 >> 172.21.15.47:0/2115030530 conn(0x7fe9d40d0040 msgr2=0x7fe9d40d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 -- 172.21.15.47:0/2115030530 shutdown_connections 2024-04-20T10:31:08.169 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.168+0000 7fe9d9800640 1 -- 172.21.15.47:0/2115030530 wait complete. 2024-04-20T10:31:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.170+0000 7fe9d9800640 1 Processor -- start 2024-04-20T10:31:08.170 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.170+0000 7fe9d9800640 1 -- start start 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40adf60 0x7fe9d40c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d9800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40b2f70 0x7fe9d41f78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d415faf0 0x7fe9d4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d9800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d406c010 con 0x7fe9d40b2f70 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d40736c0 con 0x7fe9d40adf60 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9d406c190 con 0x7fe9d415faf0 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40b2f70 0x7fe9d41f78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.172 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d3400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40b2f70 0x7fe9d41f78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38482/0 (socket says 172.21.15.47:38482) 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.171+0000 7fe9d3400640 1 -- 172.21.15.47:0/3830562435 learned_addr learned my addr 172.21.15.47:0/3830562435 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d8e00640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d415faf0 0x7fe9d4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3e00640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40adf60 0x7fe9d40c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3400640 1 -- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d415faf0 msgr2=0x7fe9d4158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3400640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d415faf0 0x7fe9d4158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3400640 1 -- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40adf60 msgr2=0x7fe9d40c9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3400640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40adf60 0x7fe9d40c9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3400640 1 -- 172.21.15.47:0/3830562435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d41f42b0 con 0x7fe9d40b2f70 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d8e00640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d415faf0 0x7fe9d4158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3e00640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40adf60 0x7fe9d40c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:08.173 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.172+0000 7fe9d3400640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40b2f70 0x7fe9d41f78a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000c00 tx=0x7fe9c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.173+0000 7fe9cbe00640 1 -- 172.21.15.47:0/3830562435 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0048070 con 0x7fe9d40b2f70 2024-04-20T10:31:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.173+0000 7fe9cbe00640 1 -- 172.21.15.47:0/3830562435 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe9c0030640 con 0x7fe9d40b2f70 2024-04-20T10:31:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.173+0000 7fe9cbe00640 1 -- 172.21.15.47:0/3830562435 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0034040 con 0x7fe9d40b2f70 2024-04-20T10:31:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.173+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d41f7de0 con 0x7fe9d40b2f70 2024-04-20T10:31:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.173+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d41f8340 con 0x7fe9d40b2f70 2024-04-20T10:31:08.174 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.174+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d4075b60 con 0x7fe9d40b2f70 2024-04-20T10:31:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.175+0000 7fe9cbe00640 1 -- 172.21.15.47:0/3830562435 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe9c004b070 con 0x7fe9d40b2f70 2024-04-20T10:31:08.175 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.176+0000 7fe9cbe00640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.176+0000 7fe9cbe00640 1 -- 172.21.15.47:0/3830562435 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 6273+0+0 (secure 0 0 0) 0x7fe9c00add80 con 0x7fe9d40b2f70 2024-04-20T10:31:08.176 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.176+0000 7fe9d3e00640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.177 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.177+0000 7fe9d3e00640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.180 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.180+0000 7fe9cbe00640 1 -- 172.21.15.47:0/3830562435 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe9d41f7de0 con 0x7fe9d40b2f70 2024-04-20T10:31:08.426 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.426+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe9d40b4200 con 0x7fe9d40b2f70 2024-04-20T10:31:08.427 INFO:tasks.ceph.mon.a.smithi047.stderr:2024-04-20T10:31:08.428+0000 7f06e1000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-20T10:31:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.431+0000 7fe9cbe00640 1 -- 172.21.15.47:0/3830562435 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) ==== 161+0+0 (secure 0 0 0) 0x7fe9d41f42b0 con 0x7fe9d40b2f70 2024-04-20T10:31:08.431 INFO:teuthology.orchestra.run.smithi047.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-20T10:31:08.434 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 msgr2=0x7fe9b406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fe9c4006fd0 tx=0x7fe9c4008040 comp rx=0 tx=0).stop 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40b2f70 msgr2=0x7fe9d41f78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40b2f70 0x7fe9d41f78a0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe9c0000c00 tx=0x7fe9c0004730 comp rx=0 tx=0).stop 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 shutdown_connections 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9d415faf0 0x7fe9d4158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe9b406c8f0 0x7fe9b406edb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe9d40b2f70 0x7fe9d41f78a0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 --2- 172.21.15.47:0/3830562435 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9d40adf60 0x7fe9d40c9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.434+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 >> 172.21.15.47:0/3830562435 conn(0x7fe9d40d0040 msgr2=0x7fe9d40d0420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.435+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 shutdown_connections 2024-04-20T10:31:08.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.435+0000 7fe9d9800640 1 -- 172.21.15.47:0/3830562435 wait complete. 2024-04-20T10:31:08.445 INFO:tasks.ceph.mds.a.smithi047.stderr: -374> 2024-04-20T10:30:59.495+0000 7f18b6600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-20T10:31:08.456 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-20T10:31:08.495 INFO:tasks.ceph.mds.a.smithi047.stdout:starting mds.a at 2024-04-20T10:31:08.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.641+0000 7fa7f4400640 1 Processor -- start 2024-04-20T10:31:08.641 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.641+0000 7fa7f4400640 1 -- start start 2024-04-20T10:31:08.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7f4400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 0x7fa7ec0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.643 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7f4400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ec141c60 con 0x7fa7ec156c20 2024-04-20T10:31:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7f4400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa7ec0adab0 con 0x7fa7ec15ca40 2024-04-20T10:31:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7f4400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ec06c010 con 0x7fa7ec0d0360 2024-04-20T10:31:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7f1a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fa7ec156c20 0x7fa7ec06b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59608/0 (socket says 172.21.15.47:59608) 2024-04-20T10:31:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7f1a00640 1 -- 172.21.15.47:0/119110795 learned_addr learned my addr 172.21.15.47:0/119110795 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7f2400640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 0x7fa7ec0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.643+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3875387864 0 0) 0x7fa7ec141c60 con 0x7fa7ec156c20 2024-04-20T10:31:08.644 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa7d4003520 con 0x7fa7ec156c20 2024-04-20T10:31:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 484275551 0 0) 0x7fa7d4003520 con 0x7fa7ec156c20 2024-04-20T10:31:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 msgr2=0x7fa7ec0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 0x7fa7ec0d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.646 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 >> v1:172.21.15.175:6789/0 conn(0x7fa7ec15ca40 legacy=0x7fa7ec0ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec15cf90 con 0x7fa7ec156c20 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fa7e0001670 con 0x7fa7ec156c20 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 >> v1:172.21.15.47:6789/0 conn(0x7fa7ec156c20 legacy=0x7fa7ec06b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7d40039e0 0x7fa7d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 0x7fa7d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7d4004d40 0x7fa7d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec06c010 con 0x7fa7d40039e0 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7d400baa0 con 0x7fa7d4004d40 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7d400bca0 con 0x7fa7ec0d0360 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.644+0000 7fa7f1a00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7d40039e0 0x7fa7d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.645+0000 7fa7f1000640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 0x7fa7d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.647 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.645+0000 7fa7f1a00640 1 -- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 msgr2=0x7fa7d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.645+0000 7fa7f1a00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 0x7fa7d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.645+0000 7fa7f1a00640 1 -- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7d4004d40 msgr2=0x7fa7d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.645+0000 7fa7f1a00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7d4004d40 0x7fa7d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.645+0000 7fa7f1a00640 1 -- 172.21.15.47:0/119110795 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7d4003520 con 0x7fa7d40039e0 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.645+0000 7fa7f1000640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7ec0d0360 0x7fa7d4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.646+0000 7fa7f1a00640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7d40039e0 0x7fa7d4003e00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c160 tx=0x7fa7d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6e07be99c86edc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.646+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0007c60 con 0x7fa7d40039e0 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.646+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e0030070 con 0x7fa7d40039e0 2024-04-20T10:31:08.648 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.646+0000 7fa7ebe00640 1 -- 172.21.15.47:0/119110795 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e0005d50 con 0x7fa7d40039e0 2024-04-20T10:31:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.647+0000 7fa7f4400640 1 -- 172.21.15.47:0/119110795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7d40039e0 msgr2=0x7fa7d4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.649 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.647+0000 7fa7f4400640 1 --2- 172.21.15.47:0/119110795 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7d40039e0 0x7fa7d4003e00 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fa7e000c160 tx=0x7fa7d4005830 comp rx=0 tx=0).stop 2024-04-20T10:31:08.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.647+0000 7fa7f2400640 1 -- 172.21.15.47:0/119110795 reap_dead start 2024-04-20T10:31:08.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.647+0000 7fa7f4400640 1 -- 172.21.15.47:0/119110795 shutdown_connections 2024-04-20T10:31:08.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.647+0000 7fa7f4400640 1 -- 172.21.15.47:0/119110795 >> 172.21.15.47:0/119110795 conn(0x7fa7ec0b3350 msgr2=0x7fa7ec0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:08.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.647+0000 7fa7f4400640 1 -- 172.21.15.47:0/119110795 shutdown_connections 2024-04-20T10:31:08.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.647+0000 7fa7f4400640 1 -- 172.21.15.47:0/119110795 wait complete. 2024-04-20T10:31:08.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.649+0000 7fa7f4400640 1 Processor -- start 2024-04-20T10:31:08.650 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.649+0000 7fa7f4400640 1 -- start start 2024-04-20T10:31:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.650+0000 7fa7f4400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7d40039e0 0x7fa7ec15b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.650+0000 7fa7f4400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ec1f8870 0x7fa7ec072e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.652 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.650+0000 7fa7f4400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7ec0cdb00 0x7fa7ec0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.650+0000 7fa7f4400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec15d830 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.650+0000 7fa7f4400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7ec15d550 con 0x7fa7ec1f8870 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.650+0000 7fa7f4400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7ec0d07f0 con 0x7fa7d40039e0 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7ec0cdb00 0x7fa7ec0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7ec0cdb00 0x7fa7ec0cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38514/0 (socket says 172.21.15.47:38514) 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 -- 172.21.15.47:0/4031420017 learned_addr learned my addr 172.21.15.47:0/4031420017 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f1000640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ec1f8870 0x7fa7ec072e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f1a00640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7d40039e0 0x7fa7ec15b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 -- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7d40039e0 msgr2=0x7fa7ec15b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7d40039e0 0x7fa7ec15b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 -- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ec1f8870 msgr2=0x7fa7ec072e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:08.653 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ec1f8870 0x7fa7ec072e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f2400640 1 -- 172.21.15.47:0/4031420017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7ec0ccd40 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.651+0000 7fa7f1000640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ec1f8870 0x7fa7ec072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.652+0000 7fa7f2400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7ec0cdb00 0x7fa7ec0cdf60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae130 tx=0x7fa7e400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.652+0000 7fa7e9600640 1 -- 172.21.15.47:0/4031420017 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4018020 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.652+0000 7fa7e9600640 1 -- 172.21.15.47:0/4031420017 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa7e400bab0 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.652+0000 7fa7e9600640 1 -- 172.21.15.47:0/4031420017 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7e4014400 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.652+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7ec0cd020 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.654 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.652+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7ec073350 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.653+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7ec068980 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.654+0000 7fa7e9600640 1 -- 172.21.15.47:0/4031420017 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa7e4005d10 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.655+0000 7fa7e9600640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa7cc06c8f0 0x7fa7cc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:08.655 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.655+0000 7fa7e9600640 1 -- 172.21.15.47:0/4031420017 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6359+0+0 (secure 0 0 0) 0x7fa7e408c310 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.656 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.656+0000 7fa7f1a00640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa7cc06c8f0 0x7fa7cc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:08.657 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.657+0000 7fa7f1a00640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa7cc06c8f0 0x7fa7cc06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40057e0 tx=0x7fa7e0009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:08.660 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.660+0000 7fa7e9600640 1 -- 172.21.15.47:0/4031420017 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fa7e4022030 con 0x7fa7ec0cdb00 2024-04-20T10:31:08.905 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:08.905+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa7ec1f8c50 con 0x7fa7ec0cdb00 2024-04-20T10:31:09.435 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.435+0000 7fa7e9600640 1 -- 172.21.15.47:0/4031420017 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v43) ==== 104+0+0 (secure 0 0 0) 0x7fa7e405a410 con 0x7fa7ec0cdb00 2024-04-20T10:31:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa7cc06c8f0 msgr2=0x7fa7cc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa7cc06c8f0 0x7fa7cc06edb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa7d40057e0 tx=0x7fa7e0009030 comp rx=0 tx=0).stop 2024-04-20T10:31:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7ec0cdb00 msgr2=0x7fa7ec0cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7ec0cdb00 0x7fa7ec0cdf60 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fa7ec0ae130 tx=0x7fa7e400eea0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 shutdown_connections 2024-04-20T10:31:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa7cc06c8f0 0x7fa7cc06edb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.439 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa7ec0cdb00 0x7fa7ec0cdf60 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7ec1f8870 0x7fa7ec072e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 --2- 172.21.15.47:0/4031420017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7d40039e0 0x7fa7ec15b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.439+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 >> 172.21.15.47:0/4031420017 conn(0x7fa7ec0b3350 msgr2=0x7fa7ec150d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:09.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.440+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 shutdown_connections 2024-04-20T10:31:09.440 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.440+0000 7fa7f4400640 1 -- 172.21.15.47:0/4031420017 wait complete. 2024-04-20T10:31:09.461 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-20T10:31:09.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.723+0000 7f503f200640 1 Processor -- start 2024-04-20T10:31:09.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.723+0000 7f503f200640 1 -- start start 2024-04-20T10:31:09.724 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.724+0000 7f503f200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50400d1040 0x7f50400d3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.724+0000 7f503f200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f50400d14f0 0x7f50400d3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.724+0000 7f503f200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f50400d4b70 con 0x7f504015e6c0 2024-04-20T10:31:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.724+0000 7f503f200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f504006ef00 con 0x7f50400d1040 2024-04-20T10:31:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.724+0000 7f503f200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f50400d4e50 con 0x7f50400d14f0 2024-04-20T10:31:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50400d1040 0x7f50400d3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503d400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f50400d1040 0x7f50400d3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38528/0 (socket says 172.21.15.47:38528) 2024-04-20T10:31:09.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503d400640 1 -- 172.21.15.47:0/2881444665 learned_addr learned my addr 172.21.15.47:0/2881444665 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:09.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503d400640 1 -- 172.21.15.47:0/2881444665 >> v2:172.21.15.175:3300/0 conn(0x7f50400d14f0 msgr2=0x7f50400d3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:09.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503e800640 1 --2- 172.21.15.47:0/2881444665 >> v2:172.21.15.175:3300/0 conn(0x7f50400d14f0 0x7f50400d3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:09.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503d400640 1 --2- 172.21.15.47:0/2881444665 >> v2:172.21.15.175:3300/0 conn(0x7f50400d14f0 0x7f50400d3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503d400640 1 -- 172.21.15.47:0/2881444665 >> v1:172.21.15.175:6789/0 conn(0x7f504015e6c0 legacy=0x7f5040162b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:09.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503d400640 1 -- 172.21.15.47:0/2881444665 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5040162330 con 0x7f50400d1040 2024-04-20T10:31:09.726 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.725+0000 7f503e800640 1 --2- 172.21.15.47:0/2881444665 >> v2:172.21.15.175:3300/0 conn(0x7f50400d14f0 0x7f50400d3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.727+0000 7f503d400640 1 --2- 172.21.15.47:0/2881444665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f50400d3130 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f504008d7d0 tx=0x7f502c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7005e258e488b3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.727+0000 7f503ca00640 1 -- 172.21.15.47:0/2881444665 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c034070 con 0x7f50400d1040 2024-04-20T10:31:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.727+0000 7f503ca00640 1 -- 172.21.15.47:0/2881444665 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f502c030860 con 0x7f50400d1040 2024-04-20T10:31:09.727 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.727+0000 7f503ca00640 1 -- 172.21.15.47:0/2881444665 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f502c030c30 con 0x7f50400d1040 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 -- 172.21.15.47:0/2881444665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 msgr2=0x7f50400d3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 --2- 172.21.15.47:0/2881444665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f50400d3130 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f504008d7d0 tx=0x7f502c02b9a0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 -- 172.21.15.47:0/2881444665 shutdown_connections 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 --2- 172.21.15.47:0/2881444665 >> v2:172.21.15.175:3300/0 conn(0x7f50400d14f0 0x7f50400d3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 --2- 172.21.15.47:0/2881444665 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f50400d3130 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 -- 172.21.15.47:0/2881444665 >> 172.21.15.47:0/2881444665 conn(0x7f504009d6a0 msgr2=0x7f50400b9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 -- 172.21.15.47:0/2881444665 shutdown_connections 2024-04-20T10:31:09.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.728+0000 7f503f200640 1 -- 172.21.15.47:0/2881444665 wait complete. 2024-04-20T10:31:09.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.730+0000 7f503f200640 1 Processor -- start 2024-04-20T10:31:09.730 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.730+0000 7f503f200640 1 -- start start 2024-04-20T10:31:09.731 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503f200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f504015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50400d14f0 0x7f504015f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f504015e6c0 0x7f50400cc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503f200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f504006eb40 con 0x7f50400d1040 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f504015eaa0 con 0x7f50400d14f0 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f504006ecc0 con 0x7f504015e6c0 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f504015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503de00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f504015b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38536/0 (socket says 172.21.15.47:38536) 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503de00640 1 -- 172.21.15.47:0/419290455 learned_addr learned my addr 172.21.15.47:0/419290455 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503d400640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50400d14f0 0x7f504015f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503de00640 1 -- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f504015e6c0 msgr2=0x7f50400cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503e800640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f504015e6c0 0x7f50400cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:09.732 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.731+0000 7f503de00640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f504015e6c0 0x7f50400cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f503de00640 1 -- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50400d14f0 msgr2=0x7f504015f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f503de00640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50400d14f0 0x7f504015f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f503de00640 1 -- 172.21.15.47:0/419290455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f504015be20 con 0x7f50400d1040 2024-04-20T10:31:09.733 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f503de00640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f504015b8e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f503000b3e0 tx=0x7f503000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f5036000640 1 -- 172.21.15.47:0/419290455 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030015020 con 0x7f50400d1040 2024-04-20T10:31:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f5036000640 1 -- 172.21.15.47:0/419290455 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5030011850 con 0x7f50400d1040 2024-04-20T10:31:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f5036000640 1 -- 172.21.15.47:0/419290455 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5030011c40 con 0x7f50400d1040 2024-04-20T10:31:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f504015ff30 con 0x7f50400d1040 2024-04-20T10:31:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.732+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50401604f0 con 0x7f50400d1040 2024-04-20T10:31:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.734+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5040075b30 con 0x7f50400d1040 2024-04-20T10:31:09.734 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.734+0000 7f5036000640 1 -- 172.21.15.47:0/419290455 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f5030018020 con 0x7f50400d1040 2024-04-20T10:31:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.736+0000 7f5036000640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f502006c8f0 0x7f502006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.736+0000 7f5036000640 1 -- 172.21.15.47:0/419290455 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 6359+0+0 (secure 0 0 0) 0x7f503008bda0 con 0x7f50400d1040 2024-04-20T10:31:09.736 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.737+0000 7f503d400640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f502006c8f0 0x7f502006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:09.737 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.738+0000 7f503d400640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f502006c8f0 0x7f502006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f502c036000 tx=0x7f502c03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:09.741 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.741+0000 7f5036000640 1 -- 172.21.15.47:0/419290455 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f503001f020 con 0x7f50400d1040 2024-04-20T10:31:09.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:09.944+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f50400b7b60 con 0x7f50400d1040 2024-04-20T10:31:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.610+0000 7f5036000640 1 -- 172.21.15.47:0/419290455 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7f5030059f40 con 0x7f50400d1040 2024-04-20T10:31:10.611 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_metadata' removed 2024-04-20T10:31:10.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f502006c8f0 msgr2=0x7f502006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:10.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f502006c8f0 0x7f502006edb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f502c036000 tx=0x7f502c03b040 comp rx=0 tx=0).stop 2024-04-20T10:31:10.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 msgr2=0x7f504015b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:10.617 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f504015b8e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f503000b3e0 tx=0x7f503000b8a0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 shutdown_connections 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f504015e6c0 0x7f50400cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50400d14f0 0x7f504015f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f502006c8f0 0x7f502006edb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 --2- 172.21.15.47:0/419290455 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f50400d1040 0x7f504015b8e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.616+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 >> 172.21.15.47:0/419290455 conn(0x7f504009d6a0 msgr2=0x7f50400b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.617+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 shutdown_connections 2024-04-20T10:31:10.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.617+0000 7f503f200640 1 -- 172.21.15.47:0/419290455 wait complete. 2024-04-20T10:31:10.638 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-20T10:31:10.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.862+0000 7f8811600640 1 Processor -- start 2024-04-20T10:31:10.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.862+0000 7f8811600640 1 -- start start 2024-04-20T10:31:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.863+0000 7f8811600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f880c0d2840 0x7f880c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.864+0000 7f8811600640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f880c0d2e30 con 0x7f880c161d10 2024-04-20T10:31:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.864+0000 7f8811600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f880c0d3110 con 0x7f880c0d2460 2024-04-20T10:31:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.864+0000 7f8811600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f880c0d33f0 con 0x7f880c0d2840 2024-04-20T10:31:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.864+0000 7f8810c00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f880c161d10 0x7f880c0cfa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59624/0 (socket says 172.21.15.47:59624) 2024-04-20T10:31:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.864+0000 7f8810c00640 1 -- 172.21.15.47:0/3402402852 learned_addr learned my addr 172.21.15.47:0/3402402852 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:10.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.864+0000 7f880b400640 1 --2- 172.21.15.47:0/3402402852 >> v2:172.21.15.175:3300/0 conn(0x7f880c0d2840 0x7f880c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.864+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 513805772 0 0) 0x7f880c0d2e30 con 0x7f880c161d10 2024-04-20T10:31:10.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f87f0003520 con 0x7f880c161d10 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 3535646258 0 0) 0x7f87f0003520 con 0x7f880c161d10 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 >> v2:172.21.15.175:3300/0 conn(0x7f880c0d2840 msgr2=0x7f880c1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 --2- 172.21.15.47:0/3402402852 >> v2:172.21.15.175:3300/0 conn(0x7f880c0d2840 0x7f880c1617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 >> v1:172.21.15.175:6789/0 conn(0x7f880c0d2460 legacy=0x7f880c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c0d36d0 con 0x7f880c161d10 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f8804001670 con 0x7f880c161d10 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 >> v1:172.21.15.47:6789/0 conn(0x7f880c161d10 legacy=0x7f880c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0003a00 0x7f87f0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0004800 0x7f87f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f00051c0 0x7f87f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f880c0d33f0 con 0x7f87f0004800 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87f000c0a0 con 0x7f87f00051c0 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87f000c2a0 con 0x7f87f0003a00 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.865+0000 7f880b400640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0004800 0x7f87f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f8810c00640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f00051c0 0x7f87f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f880be00640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0003a00 0x7f87f0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f880b400640 1 -- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0003a00 msgr2=0x7f87f0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f880b400640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0003a00 0x7f87f0003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f880b400640 1 -- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f00051c0 msgr2=0x7f87f000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f880b400640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f00051c0 0x7f87f000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f880b400640 1 -- 172.21.15.47:0/3402402852 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87f0003520 con 0x7f87f0004800 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f8810c00640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87f00051c0 0x7f87f000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:10.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.866+0000 7f8810c00640 1 -- 172.21.15.47:0/3402402852 reap_dead start 2024-04-20T10:31:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.867+0000 7f880b400640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0004800 0x7f87f0004c80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009ac0 tx=0x7f87f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35110a0b8f11838e server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.867+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4039070 con 0x7f87f0004800 2024-04-20T10:31:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.867+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f402c6b0 con 0x7f87f0004800 2024-04-20T10:31:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.867+0000 7f880aa00640 1 -- 172.21.15.47:0/3402402852 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4009690 con 0x7f87f0004800 2024-04-20T10:31:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.868+0000 7f8811600640 1 -- 172.21.15.47:0/3402402852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0004800 msgr2=0x7f87f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:10.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.868+0000 7f8811600640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0004800 0x7f87f0004c80 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009ac0 tx=0x7f87f0005830 comp rx=0 tx=0).stop 2024-04-20T10:31:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.868+0000 7f8811600640 1 -- 172.21.15.47:0/3402402852 shutdown_connections 2024-04-20T10:31:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.868+0000 7f8811600640 1 --2- 172.21.15.47:0/3402402852 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f87f0004800 0x7f87f0004c80 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.868+0000 7f8811600640 1 -- 172.21.15.47:0/3402402852 >> 172.21.15.47:0/3402402852 conn(0x7f880c0b7570 msgr2=0x7f880c0b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.868+0000 7f8811600640 1 -- 172.21.15.47:0/3402402852 shutdown_connections 2024-04-20T10:31:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.868+0000 7f8811600640 1 -- 172.21.15.47:0/3402402852 wait complete. 2024-04-20T10:31:10.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.870+0000 7f8811600640 1 Processor -- start 2024-04-20T10:31:10.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.870+0000 7f8811600640 1 -- start start 2024-04-20T10:31:10.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f8811600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0004800 0x7f880c1f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f8811600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c06e1e0 0x7f880c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f8811600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0c8150 0x7f880c1f5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f8811600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f880c0d44d0 con 0x7f880c06e1e0 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f8811600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f880c162120 con 0x7f880c0c8150 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f8811600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f880c0d4650 con 0x7f87f0004800 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f880b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c06e1e0 0x7f880c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f880b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c06e1e0 0x7f880c0c9e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38556/0 (socket says 172.21.15.47:38556) 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.871+0000 7f880b400640 1 -- 172.21.15.47:0/1956575389 learned_addr learned my addr 172.21.15.47:0/1956575389 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880be00640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0004800 0x7f880c1f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f8810c00640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0c8150 0x7f880c1f5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880b400640 1 -- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0004800 msgr2=0x7f880c1f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880b400640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0004800 0x7f880c1f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880b400640 1 -- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0c8150 msgr2=0x7f880c1f5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880b400640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0c8150 0x7f880c1f5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880b400640 1 -- 172.21.15.47:0/1956575389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f880c1f8920 con 0x7f880c06e1e0 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880be00640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0004800 0x7f880c1f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:10.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.872+0000 7f880b400640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c06e1e0 0x7f880c0c9e80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009be0 tx=0x7f87f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.873+0000 7f8803e00640 1 -- 172.21.15.47:0/1956575389 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4042020 con 0x7f880c06e1e0 2024-04-20T10:31:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.873+0000 7f8803e00640 1 -- 172.21.15.47:0/1956575389 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f403a040 con 0x7f880c06e1e0 2024-04-20T10:31:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.873+0000 7f8803e00640 1 -- 172.21.15.47:0/1956575389 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87f4039070 con 0x7f880c06e1e0 2024-04-20T10:31:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.873+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f880c1f8c00 con 0x7f880c06e1e0 2024-04-20T10:31:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.873+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f880c1f91c0 con 0x7f880c06e1e0 2024-04-20T10:31:10.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.874+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f880c0d2e60 con 0x7f880c06e1e0 2024-04-20T10:31:10.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.875+0000 7f8803e00640 1 -- 172.21.15.47:0/1956575389 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f87f4045020 con 0x7f880c06e1e0 2024-04-20T10:31:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.876+0000 7f8803e00640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87e806c970 0x7f87e806ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:10.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.876+0000 7f8803e00640 1 -- 172.21.15.47:0/1956575389 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5921+0+0 (secure 0 0 0) 0x7f87f40acf20 con 0x7f880c06e1e0 2024-04-20T10:31:10.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.877+0000 7f880be00640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87e806c970 0x7f87e806ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:10.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.878+0000 7f880be00640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87e806c970 0x7f87e806ee30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87fc001d60 tx=0x7f87fc009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:10.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:10.881+0000 7f8803e00640 1 -- 172.21.15.47:0/1956575389 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f880c0d2e60 con 0x7f880c06e1e0 2024-04-20T10:31:11.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.086+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f880c0c7bc0 con 0x7f880c06e1e0 2024-04-20T10:31:11.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.621+0000 7f8803e00640 1 -- 172.21.15.47:0/1956575389 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7f87f404c070 con 0x7f880c06e1e0 2024-04-20T10:31:11.621 INFO:teuthology.orchestra.run.smithi047.stderr:pool 'cephfs_data' removed 2024-04-20T10:31:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87e806c970 msgr2=0x7f87e806ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87e806c970 0x7f87e806ee30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f87fc001d60 tx=0x7f87fc009040 comp rx=0 tx=0).stop 2024-04-20T10:31:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c06e1e0 msgr2=0x7f880c0c9e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.625 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c06e1e0 0x7f880c0c9e80 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009be0 tx=0x7f87f4004730 comp rx=0 tx=0).stop 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 shutdown_connections 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f880c0c8150 0x7f880c1f5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f87e806c970 0x7f87e806ee30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f880c06e1e0 0x7f880c0c9e80 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 --2- 172.21.15.47:0/1956575389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87f0004800 0x7f880c1f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 >> 172.21.15.47:0/1956575389 conn(0x7f880c0b7570 msgr2=0x7f880c1556a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 shutdown_connections 2024-04-20T10:31:11.626 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.625+0000 7f8811600640 1 -- 172.21.15.47:0/1956575389 wait complete. 2024-04-20T10:31:11.653 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-20T10:31:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.865+0000 7fe196400640 1 Processor -- start 2024-04-20T10:31:11.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.865+0000 7fe196400640 1 -- start start 2024-04-20T10:31:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.867+0000 7fe196400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 0x7fe1900b6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.867+0000 7fe196400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1900b5bd0 con 0x7fe19015e7d0 2024-04-20T10:31:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.867+0000 7fe196400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe1900b5eb0 con 0x7fe1900ca740 2024-04-20T10:31:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.867+0000 7fe196400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1900b5980 con 0x7fe1900cd930 2024-04-20T10:31:11.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.867+0000 7fe18f400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe19015e7d0 0x7fe1900ccd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59630/0 (socket says 172.21.15.47:59630) 2024-04-20T10:31:11.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.867+0000 7fe18f400640 1 -- 172.21.15.47:0/1787874911 learned_addr learned my addr 172.21.15.47:0/1787874911 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18fe00640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 0x7fe1900b6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2541330834 0 0) 0x7fe1900b5bd0 con 0x7fe19015e7d0 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe174003520 con 0x7fe19015e7d0 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 930927965 0 0) 0x7fe174003520 con 0x7fe19015e7d0 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 msgr2=0x7fe1900b6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 0x7fe1900b6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 >> v1:172.21.15.175:6789/0 conn(0x7fe1900ca740 legacy=0x7fe1900cf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1900cc530 con 0x7fe19015e7d0 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fe180001670 con 0x7fe19015e7d0 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 >> v1:172.21.15.47:6789/0 conn(0x7fe19015e7d0 legacy=0x7fe1900ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1740039e0 0x7fe174003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 0x7fe174004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe174004d40 0x7fe1740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe1900b5980 con 0x7fe1740039e0 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe17400bb50 con 0x7fe174004d40 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe17400bd50 con 0x7fe1900cd930 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.868+0000 7fe18f400640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1740039e0 0x7fe174003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.869+0000 7fe18ea00640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 0x7fe174004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.869+0000 7fe18f400640 1 -- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 msgr2=0x7fe174004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.869+0000 7fe18f400640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 0x7fe174004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.869+0000 7fe18f400640 1 -- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe174004d40 msgr2=0x7fe1740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:11.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.869+0000 7fe18f400640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe174004d40 0x7fe1740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.869+0000 7fe18f400640 1 -- 172.21.15.47:0/1787874911 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe174003520 con 0x7fe1740039e0 2024-04-20T10:31:11.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.869+0000 7fe18ea00640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1900cd930 0x7fe174004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:11.872 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.870+0000 7fe18f400640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1740039e0 0x7fe174003e00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c210 tx=0x7fe174005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18e57b57a0740aa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.870+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe180007d30 con 0x7fe1740039e0 2024-04-20T10:31:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.870+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe180030070 con 0x7fe1740039e0 2024-04-20T10:31:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.870+0000 7fe18e000640 1 -- 172.21.15.47:0/1787874911 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18002cd60 con 0x7fe1740039e0 2024-04-20T10:31:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.871+0000 7fe196400640 1 -- 172.21.15.47:0/1787874911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1740039e0 msgr2=0x7fe174003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.871+0000 7fe196400640 1 --2- 172.21.15.47:0/1787874911 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1740039e0 0x7fe174003e00 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c210 tx=0x7fe174005830 comp rx=0 tx=0).stop 2024-04-20T10:31:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.871+0000 7fe18fe00640 1 -- 172.21.15.47:0/1787874911 reap_dead start 2024-04-20T10:31:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.871+0000 7fe196400640 1 -- 172.21.15.47:0/1787874911 shutdown_connections 2024-04-20T10:31:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.871+0000 7fe196400640 1 -- 172.21.15.47:0/1787874911 >> 172.21.15.47:0/1787874911 conn(0x7fe1900704e0 msgr2=0x7fe1900708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.871+0000 7fe196400640 1 -- 172.21.15.47:0/1787874911 shutdown_connections 2024-04-20T10:31:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.871+0000 7fe196400640 1 -- 172.21.15.47:0/1787874911 wait complete. 2024-04-20T10:31:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.873+0000 7fe196400640 1 Processor -- start 2024-04-20T10:31:11.874 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.873+0000 7fe196400640 1 -- start start 2024-04-20T10:31:11.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe196400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1740039e0 0x7fe190146a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe196400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1901470e0 0x7fe190147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe196400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe190160ae0 0x7fe190160f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.876 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe196400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe19015ebb0 con 0x7fe1901470e0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe196400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1900cc8a0 con 0x7fe1740039e0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe196400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1900cc150 con 0x7fe190160ae0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe18ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1901470e0 0x7fe190147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe18ea00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1901470e0 0x7fe190147560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38564/0 (socket says 172.21.15.47:38564) 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.874+0000 7fe18ea00640 1 -- 172.21.15.47:0/1415966354 learned_addr learned my addr 172.21.15.47:0/1415966354 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18f400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1740039e0 0x7fe190146a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18fe00640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe190160ae0 0x7fe190160f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18ea00640 1 -- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe190160ae0 msgr2=0x7fe190160f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18ea00640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe190160ae0 0x7fe190160f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18ea00640 1 -- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1740039e0 msgr2=0x7fe190146a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18ea00640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1740039e0 0x7fe190146a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18ea00640 1 -- 172.21.15.47:0/1415966354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe190157480 con 0x7fe1901470e0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.875+0000 7fe18ea00640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1901470e0 0x7fe190147560 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00d7d0 tx=0x7fe17c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:11.877 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.876+0000 7fe187400640 1 -- 172.21.15.47:0/1415966354 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c005070 con 0x7fe1901470e0 2024-04-20T10:31:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.876+0000 7fe187400640 1 -- 172.21.15.47:0/1415966354 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe17c005230 con 0x7fe1901470e0 2024-04-20T10:31:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.876+0000 7fe187400640 1 -- 172.21.15.47:0/1415966354 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c007030 con 0x7fe1901470e0 2024-04-20T10:31:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.876+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe190147c50 con 0x7fe1901470e0 2024-04-20T10:31:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.876+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe190148210 con 0x7fe1901470e0 2024-04-20T10:31:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.877+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe190069090 con 0x7fe1901470e0 2024-04-20T10:31:11.878 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.878+0000 7fe187400640 1 -- 172.21.15.47:0/1415966354 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe17c01a020 con 0x7fe1901470e0 2024-04-20T10:31:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.879+0000 7fe187400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe17006c8f0 0x7fe17006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.879+0000 7fe187400640 1 -- 172.21.15.47:0/1415966354 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fe17c08b280 con 0x7fe1901470e0 2024-04-20T10:31:11.879 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.880+0000 7fe18f400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe17006c8f0 0x7fe17006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:11.881 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.881+0000 7fe18f400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe17006c8f0 0x7fe17006edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c210 tx=0x7fe180009030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:11.884 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:11.883+0000 7fe187400640 1 -- 172.21.15.47:0/1415966354 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe17c059830 con 0x7fe1901470e0 2024-04-20T10:31:12.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.087+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) -- 0x7fe19015f820 con 0x7fe1901470e0 2024-04-20T10:31:12.093 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.093+0000 7fe187400640 1 -- 172.21.15.47:0/1415966354 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) ==== 104+0+0 (secure 0 0 0) 0x7fe17c059130 con 0x7fe1901470e0 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.096+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe17006c8f0 msgr2=0x7fe17006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.096+0000 7fe196400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe17006c8f0 0x7fe17006edb0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe18000c210 tx=0x7fe180009030 comp rx=0 tx=0).stop 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.096+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1901470e0 msgr2=0x7fe190147560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.096+0000 7fe196400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1901470e0 0x7fe190147560 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fe17c00d7d0 tx=0x7fe17c00dc90 comp rx=0 tx=0).stop 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 shutdown_connections 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe190160ae0 0x7fe190160f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe17006c8f0 0x7fe17006edb0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.097 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe1901470e0 0x7fe190147560 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 --2- 172.21.15.47:0/1415966354 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1740039e0 0x7fe190146a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 >> 172.21.15.47:0/1415966354 conn(0x7fe1900704e0 msgr2=0x7fe1900cfb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 shutdown_connections 2024-04-20T10:31:12.098 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.097+0000 7fe196400640 1 -- 172.21.15.47:0/1415966354 wait complete. 2024-04-20T10:31:12.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly' 2024-04-20T10:31:12.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.319+0000 7ff87b200640 1 Processor -- start 2024-04-20T10:31:12.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.319+0000 7ff87b200640 1 -- start start 2024-04-20T10:31:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff87b200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7ff87c15b7e0 0x7ff87c15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff87b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 0x7ff87c06ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff87b200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff87c06ccc0 con 0x7ff87c15b400 2024-04-20T10:31:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff87b200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7ff87c06d280 con 0x7ff87c15b7e0 2024-04-20T10:31:12.325 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff87b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff87c06cfa0 con 0x7ff87c15c190 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff879e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7ff87c15b400 0x7ff87c072d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59638/0 (socket says 172.21.15.47:59638) 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff879e00640 1 -- 172.21.15.47:0/1255272462 learned_addr learned my addr 172.21.15.47:0/1255272462 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff87a800640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 0x7ff87c06ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff879400640 1 --2- 172.21.15.47:0/1255272462 >> v2:172.21.15.175:3300/0 conn(0x7ff87c15b7e0 0x7ff87c15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.321+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4167026315 0 0) 0x7ff87c06ccc0 con 0x7ff87c15b400 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7ff860003520 con 0x7ff87c15b400 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2237375246 0 0) 0x7ff860003520 con 0x7ff87c15b400 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 msgr2=0x7ff87c06ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 0x7ff87c06ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 >> v2:172.21.15.175:3300/0 conn(0x7ff87c15b7e0 msgr2=0x7ff87c15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 --2- 172.21.15.47:0/1255272462 >> v2:172.21.15.175:3300/0 conn(0x7ff87c15b7e0 0x7ff87c15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c06e440 con 0x7ff87c15b400 2024-04-20T10:31:12.326 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7ff86c001670 con 0x7ff87c15b400 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 >> v1:172.21.15.47:6789/0 conn(0x7ff87c15b400 legacy=0x7ff87c072d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 0x7ff8600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860003f30 0x7ff860004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c06d280 con 0x7ff860004d40 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c06cfa0 con 0x7ff860003f30 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff86000dbd0 con 0x7ff87c15c190 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.322+0000 7ff87a800640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff879e00640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 0x7ff8600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff87a800640 1 -- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 msgr2=0x7ff8600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff879400640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860003f30 0x7ff860004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff87a800640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 0x7ff8600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff87a800640 1 -- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860003f30 msgr2=0x7ff860004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff87a800640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860003f30 0x7ff860004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff87a800640 1 -- 172.21.15.47:0/1255272462 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff860003520 con 0x7ff860004d40 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff879e00640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c15c190 0x7ff8600039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.323+0000 7ff879400640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff860003f30 0x7ff860004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.324+0000 7ff87a800640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff874009fc0 tx=0x7ff860005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2afaa6ff2fe97218 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.324+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874039070 con 0x7ff860004d40 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.324+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff87402cc00 con 0x7ff860004d40 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.324+0000 7ff878a00640 1 -- 172.21.15.47:0/1255272462 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874009c10 con 0x7ff860004d40 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.325+0000 7ff87b200640 1 -- 172.21.15.47:0/1255272462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 msgr2=0x7ff8600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.325+0000 7ff87b200640 1 --2- 172.21.15.47:0/1255272462 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff8600051c0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff874009fc0 tx=0x7ff860005830 comp rx=0 tx=0).stop 2024-04-20T10:31:12.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.325+0000 7ff87a800640 1 -- 172.21.15.47:0/1255272462 reap_dead start 2024-04-20T10:31:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.325+0000 7ff87b200640 1 -- 172.21.15.47:0/1255272462 shutdown_connections 2024-04-20T10:31:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.325+0000 7ff87b200640 1 -- 172.21.15.47:0/1255272462 >> 172.21.15.47:0/1255272462 conn(0x7ff87c0d0040 msgr2=0x7ff87c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.325+0000 7ff87b200640 1 -- 172.21.15.47:0/1255272462 shutdown_connections 2024-04-20T10:31:12.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.325+0000 7ff87b200640 1 -- 172.21.15.47:0/1255272462 wait complete. 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.327+0000 7ff87b200640 1 Processor -- start 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.327+0000 7ff87b200640 1 -- start start 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff87b200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff87c15da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff87b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c1fc720 0x7ff87c1f8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff87b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87c154cb0 0x7ff87c0cdc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff87b200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c15c770 con 0x7ff860004d40 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff87b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff87c15c5f0 con 0x7ff87c154cb0 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff87b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff87c06b900 con 0x7ff87c1fc720 2024-04-20T10:31:12.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff879e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff87c15da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff879e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff87c15da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38582/0 (socket says 172.21.15.47:38582) 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.328+0000 7ff879e00640 1 -- 172.21.15.47:0/3298555040 learned_addr learned my addr 172.21.15.47:0/3298555040 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff879400640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c1fc720 0x7ff87c1f8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff87a800640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87c154cb0 0x7ff87c0cdc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff879e00640 1 -- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c1fc720 msgr2=0x7ff87c1f8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff879e00640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c1fc720 0x7ff87c1f8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff879e00640 1 -- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87c154cb0 msgr2=0x7ff87c0cdc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff879e00640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87c154cb0 0x7ff87c0cdc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff879e00640 1 -- 172.21.15.47:0/3298555040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff87c1fcba0 con 0x7ff860004d40 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff879e00640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff87c15da20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d8c0 tx=0x7ff86c00dd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.329+0000 7ff872000640 1 -- 172.21.15.47:0/3298555040 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c019070 con 0x7ff860004d40 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.330+0000 7ff872000640 1 -- 172.21.15.47:0/3298555040 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff86c0074e0 con 0x7ff860004d40 2024-04-20T10:31:12.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.330+0000 7ff872000640 1 -- 172.21.15.47:0/3298555040 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff86c007690 con 0x7ff860004d40 2024-04-20T10:31:12.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.330+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff87c1f8e30 con 0x7ff860004d40 2024-04-20T10:31:12.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.330+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff87c1f93f0 con 0x7ff860004d40 2024-04-20T10:31:12.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.331+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff87c075b60 con 0x7ff860004d40 2024-04-20T10:31:12.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.332+0000 7ff872000640 1 -- 172.21.15.47:0/3298555040 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7ff86c01c070 con 0x7ff860004d40 2024-04-20T10:31:12.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.333+0000 7ff872000640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff85806c920 0x7ff85806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:12.333 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.333+0000 7ff872000640 1 -- 172.21.15.47:0/3298555040 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7ff86c08b5c0 con 0x7ff860004d40 2024-04-20T10:31:12.334 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.334+0000 7ff879400640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff85806c920 0x7ff85806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:12.335 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.335+0000 7ff879400640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff85806c920 0x7ff85806ede0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff87c1587a0 tx=0x7ff864008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:12.337 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.337+0000 7ff872000640 1 -- 172.21.15.47:0/3298555040 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7ff86c059ad0 con 0x7ff860004d40 2024-04-20T10:31:12.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.610+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]} v 0) -- 0x7ff87c0b5220 con 0x7ff860004d40 2024-04-20T10:31:12.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.624+0000 7ff872000640 1 -- 172.21.15.47:0/3298555040 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_auto_repair.TestMDSAutoRepair.test_mds_readonly"]}]=0 v220) ==== 144+0+0 (secure 0 0 0) 0x7ff86c0593d0 con 0x7ff860004d40 2024-04-20T10:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff85806c920 msgr2=0x7ff85806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.627 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff85806c920 0x7ff85806ede0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff87c1587a0 tx=0x7ff864008040 comp rx=0 tx=0).stop 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 msgr2=0x7ff87c15da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff87c15da20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff86c00d8c0 tx=0x7ff86c00dd80 comp rx=0 tx=0).stop 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 shutdown_connections 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff87c154cb0 0x7ff87c0cdc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff87c1fc720 0x7ff87c1f8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7ff85806c920 0x7ff85806ede0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 --2- 172.21.15.47:0/3298555040 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7ff860004d40 0x7ff87c15da20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 >> 172.21.15.47:0/3298555040 conn(0x7ff87c0d0040 msgr2=0x7ff87c0afea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 shutdown_connections 2024-04-20T10:31:12.628 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:12.627+0000 7ff87b200640 1 -- 172.21.15.47:0/3298555040 wait complete. 2024-04-20T10:31:12.644 INFO:tasks.cephfs_test_runner:test if MDS behave correct when it's readonly ... ok 2024-04-20T10:31:12.644 INFO:tasks.cephfs_test_runner: 2024-04-20T10:31:12.644 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-20T10:31:12.644 INFO:tasks.cephfs_test_runner:Ran 2 tests in 160.291s 2024-04-20T10:31:12.645 INFO:tasks.cephfs_test_runner: 2024-04-20T10:31:12.645 INFO:tasks.cephfs_test_runner:OK 2024-04-20T10:31:12.645 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-20T10:31:12.654 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-04-20T10:31:12.663 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-04-20T10:31:12.663 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:31:12.663 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:12.715 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T10:31:12.715 DEBUG:teuthology.orchestra.run.smithi047:> set -ex 2024-04-20T10:31:12.715 DEBUG:teuthology.orchestra.run.smithi047:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:12.765 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053506,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249616k,nr_inodes=812404,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499240k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-20T10:31:12.765 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:31:12.765 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:12.823 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:31:12.823 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-04-20T10:31:12.823 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-20T10:31:12.877 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052721,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248988k,nr_inodes=812247,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497984k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-20T10:31:12.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:12.877 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-04-20T10:31:12.927 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-20T10:31:12.927 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-20T10:31:12.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:12.928 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-20T10:31:13.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:13.005 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-20T10:31:13.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:13.087 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-04-20T10:31:13.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:13.208 DEBUG:teuthology.orchestra.run.smithi047:> ip netns list 2024-04-20T10:31:13.268 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-20T10:31:13.268 INFO:teuthology.orchestra.run.smithi047.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-20T10:31:13.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:13.269 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-20T10:31:13.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:13.366 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-20T10:31:13.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-20T10:31:13.447 DEBUG:teuthology.orchestra.run.smithi047:> sudo ip link delete ceph-brx 2024-04-20T10:31:13.574 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-20T10:31:13.583 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-20T10:31:13.584 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:31:13.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.768+0000 7fc1ed800640 1 Processor -- start 2024-04-20T10:31:13.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.768+0000 7fc1ed800640 1 -- start start 2024-04-20T10:31:13.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.769+0000 7fc1ed800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1e815b3b0 0x7fc1e806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.769+0000 7fc1ed800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8071b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.769+0000 7fc1ed800640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc1e80ade40 con 0x7fc1e8072090 2024-04-20T10:31:13.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.769+0000 7fc1ed800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fc1e80ae0e0 con 0x7fc1e815b3b0 2024-04-20T10:31:13.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.769+0000 7fc1ed800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e815b790 con 0x7fc1e806dcb0 2024-04-20T10:31:13.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1e815b3b0 0x7fc1e806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e7400640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fc1e8072090 0x7fc1e80d0360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:59654/0 (socket says 172.21.15.47:59654) 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e7400640 1 -- 172.21.15.47:0/3672590768 learned_addr learned my addr 172.21.15.47:0/3672590768 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e6a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fc1e815b3b0 0x7fc1e806eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38588/0 (socket says 172.21.15.47:38588) 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e6000640 1 --2- 172.21.15.47:0/3672590768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8071b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e6a00640 1 -- 172.21.15.47:0/3672590768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 msgr2=0x7fc1e8071b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e6a00640 1 --2- 172.21.15.47:0/3672590768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8071b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e6a00640 1 -- 172.21.15.47:0/3672590768 >> v1:172.21.15.47:6789/0 conn(0x7fc1e8072090 legacy=0x7fc1e80d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.771+0000 7fc1e6a00640 1 -- 172.21.15.47:0/3672590768 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e80ae380 con 0x7fc1e815b3b0 2024-04-20T10:31:13.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.770+0000 7fc1e5600640 1 -- 172.21.15.47:0/3672590768 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1833828297 0 0) 0x7fc1e80ade40 con 0x7fc1e8072090 2024-04-20T10:31:13.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.772+0000 7fc1e6a00640 1 --2- 172.21.15.47:0/3672590768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e806eaa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097a0 tx=0x7fc1d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=867ea94f3142b356 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:13.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.772+0000 7fc1e5600640 1 -- 172.21.15.47:0/3672590768 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0035020 con 0x7fc1e815b3b0 2024-04-20T10:31:13.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.772+0000 7fc1e5600640 1 -- 172.21.15.47:0/3672590768 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d0031830 con 0x7fc1e815b3b0 2024-04-20T10:31:13.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.772+0000 7fc1e5600640 1 -- 172.21.15.47:0/3672590768 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0031c00 con 0x7fc1e815b3b0 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 -- 172.21.15.47:0/3672590768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 msgr2=0x7fc1e806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 --2- 172.21.15.47:0/3672590768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e806eaa0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00097a0 tx=0x7fc1d002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 -- 172.21.15.47:0/3672590768 shutdown_connections 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 --2- 172.21.15.47:0/3672590768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8071b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 --2- 172.21.15.47:0/3672590768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e806eaa0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 -- 172.21.15.47:0/3672590768 >> 172.21.15.47:0/3672590768 conn(0x7fc1e80b3350 msgr2=0x7fc1e80b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 -- 172.21.15.47:0/3672590768 shutdown_connections 2024-04-20T10:31:13.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.773+0000 7fc1ed800640 1 -- 172.21.15.47:0/3672590768 wait complete. 2024-04-20T10:31:13.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.775+0000 7fc1ed800640 1 Processor -- start 2024-04-20T10:31:13.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.775+0000 7fc1ed800640 1 -- start start 2024-04-20T10:31:13.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1ed800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1ed800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e81ef540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1ed800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e81f2a90 0x7fc1e81501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1ed800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e80aec20 con 0x7fc1e815b3b0 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1ed800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1e80ae940 con 0x7fc1e81f2a90 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1ed800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1e806efe0 con 0x7fc1e806dcb0 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e81ef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e81ef540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:38602/0 (socket says 172.21.15.47:38602) 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 -- 172.21.15.47:0/1122539609 learned_addr learned my addr 172.21.15.47:0/1122539609 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6a00640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 -- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 msgr2=0x7fc1e8158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 -- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e81f2a90 msgr2=0x7fc1e81501b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e81f2a90 0x7fc1e81501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.776+0000 7fc1e6000640 1 -- 172.21.15.47:0/1122539609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1e8159f10 con 0x7fc1e815b3b0 2024-04-20T10:31:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.777+0000 7fc1e6000640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e81ef540 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800efc0 tx=0x7fc1d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.777+0000 7fc1dea00640 1 -- 172.21.15.47:0/1122539609 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8018020 con 0x7fc1e815b3b0 2024-04-20T10:31:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.777+0000 7fc1dea00640 1 -- 172.21.15.47:0/1122539609 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc1d8014480 con 0x7fc1e815b3b0 2024-04-20T10:31:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.777+0000 7fc1dea00640 1 -- 172.21.15.47:0/1122539609 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8014870 con 0x7fc1e815b3b0 2024-04-20T10:31:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.777+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1e8159390 con 0x7fc1e815b3b0 2024-04-20T10:31:13.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.778+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1e8159950 con 0x7fc1e815b3b0 2024-04-20T10:31:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.779+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1e8068980 con 0x7fc1e815b3b0 2024-04-20T10:31:13.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.780+0000 7fc1dea00640 1 -- 172.21.15.47:0/1122539609 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fc1d8005d10 con 0x7fc1e815b3b0 2024-04-20T10:31:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.781+0000 7fc1dea00640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc1c406c970 0x7fc1c406ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.781+0000 7fc1dea00640 1 -- 172.21.15.47:0/1122539609 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fc1d808c800 con 0x7fc1e815b3b0 2024-04-20T10:31:13.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.781+0000 7fc1e6a00640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc1c406c970 0x7fc1c406ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:13.783 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.783+0000 7fc1e6a00640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc1c406c970 0x7fc1c406ee30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0036000 tx=0x7fc1d0008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:13.785 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.786+0000 7fc1dea00640 1 -- 172.21.15.47:0/1122539609 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fc1d8022030 con 0x7fc1e815b3b0 2024-04-20T10:31:13.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.986+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc1e815fa00 con 0x7fc1c406c970 2024-04-20T10:31:13.989 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.989+0000 7fc1dea00640 1 -- 172.21.15.47:0/1122539609 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7fc1e815fa00 con 0x7fc1c406c970 2024-04-20T10:31:13.989 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:13.990 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:31:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc1c406c970 msgr2=0x7fc1c406ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc1c406c970 0x7fc1c406ee30 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0036000 tx=0x7fc1d0008030 comp rx=0 tx=0).stop 2024-04-20T10:31:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 msgr2=0x7fc1e81ef540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e81ef540 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc1d800efc0 tx=0x7fc1d800c490 comp rx=0 tx=0).stop 2024-04-20T10:31:13.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 shutdown_connections 2024-04-20T10:31:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1e81f2a90 0x7fc1e81501b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fc1c406c970 0x7fc1c406ee30 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fc1e815b3b0 0x7fc1e81ef540 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 --2- 172.21.15.47:0/1122539609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1e806dcb0 0x7fc1e8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 >> 172.21.15.47:0/1122539609 conn(0x7fc1e80b3350 msgr2=0x7fc1e815a620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 shutdown_connections 2024-04-20T10:31:13.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:13.992+0000 7fc1ed800640 1 -- 172.21.15.47:0/1122539609 wait complete. 2024-04-20T10:31:14.011 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-20T10:31:13.148448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239488,"kb_used_data":6688,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735232,"statfs":{"total":773094113280,"available":772848877568,"internally_reserved":0,"allocated":6848512,"data_stored":3701810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12369,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-92,"num_write":-64,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001955"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":238,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T10:31:08.782096+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:31:08.782096+0000","last_peered":"2024-04-20T10:31:08.782096+0000","last_clean":"2024-04-20T10:31:08.782096+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:31:08.782096+0000","last_undegraded":"2024-04-20T10:31:08.782096+0000","last_fullsized":"2024-04-20T10:31:08.782096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":312140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":312624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":914071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":902480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":323663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:31:14.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:31:14.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.220+0000 7f68fbe00640 1 Processor -- start 2024-04-20T10:31:14.220 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.220+0000 7f68fbe00640 1 -- start start 2024-04-20T10:31:14.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.222+0000 7f68fbe00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f68fc159e10 0x7f68fc071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.222+0000 7f68fbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc15a1f0 0x7f68fc071fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.222+0000 7f68fbe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f68fc15cc90 con 0x7f68fc06da90 2024-04-20T10:31:14.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.222+0000 7f68fbe00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f68fc072900 con 0x7f68fc159e10 2024-04-20T10:31:14.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.222+0000 7f68fbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc15ca40 con 0x7f68fc15a1f0 2024-04-20T10:31:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.223+0000 7f68faa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f68fc159e10 0x7f68fc071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.223+0000 7f68f2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc15a1f0 0x7f68fc071fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.223+0000 7f68faa00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f68fc159e10 0x7f68fc071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.47:45786/0 (socket says 172.21.15.47:45786) 2024-04-20T10:31:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.223+0000 7f68faa00640 1 -- 172.21.15.47:0/3856384944 learned_addr learned my addr 172.21.15.47:0/3856384944 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:14.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.223+0000 7f68faa00640 1 -- 172.21.15.47:0/3856384944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc15a1f0 msgr2=0x7f68fc071fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:14.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.223+0000 7f68faa00640 1 --2- 172.21.15.47:0/3856384944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc15a1f0 0x7f68fc071fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.223+0000 7f68faa00640 1 -- 172.21.15.47:0/3856384944 >> v1:172.21.15.175:6789/0 conn(0x7f68fc06da90 legacy=0x7f68fc160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.224+0000 7f68faa00640 1 -- 172.21.15.47:0/3856384944 --> v2:172.21.15.175:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc15cf70 con 0x7f68fc159e10 2024-04-20T10:31:14.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.225+0000 7f68faa00640 1 --2- 172.21.15.47:0/3856384944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc159e10 0x7f68fc071a80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68fc15cbc0 tx=0x7f68ec02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44f2788801c5bca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:14.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.225+0000 7f68fa000640 1 -- 172.21.15.47:0/3856384944 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec034070 con 0x7f68fc159e10 2024-04-20T10:31:14.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.225+0000 7f68fa000640 1 -- 172.21.15.47:0/3856384944 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68ec030840 con 0x7f68fc159e10 2024-04-20T10:31:14.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.225+0000 7f68fa000640 1 -- 172.21.15.47:0/3856384944 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ec030c10 con 0x7f68fc159e10 2024-04-20T10:31:14.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 -- 172.21.15.47:0/3856384944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc159e10 msgr2=0x7f68fc071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 --2- 172.21.15.47:0/3856384944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc159e10 0x7f68fc071a80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f68fc15cbc0 tx=0x7f68ec02b980 comp rx=0 tx=0).stop 2024-04-20T10:31:14.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 -- 172.21.15.47:0/3856384944 shutdown_connections 2024-04-20T10:31:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 --2- 172.21.15.47:0/3856384944 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc15a1f0 0x7f68fc071fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 --2- 172.21.15.47:0/3856384944 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc159e10 0x7f68fc071a80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 -- 172.21.15.47:0/3856384944 >> 172.21.15.47:0/3856384944 conn(0x7f68fc0d0040 msgr2=0x7f68fc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 -- 172.21.15.47:0/3856384944 shutdown_connections 2024-04-20T10:31:14.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.226+0000 7f68fbe00640 1 -- 172.21.15.47:0/3856384944 wait complete. 2024-04-20T10:31:14.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.228+0000 7f68fbe00640 1 Processor -- start 2024-04-20T10:31:14.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.228+0000 7f68fbe00640 1 -- start start 2024-04-20T10:31:14.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.229+0000 7f68fbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc06da90 0x7f68fc15a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.229+0000 7f68fbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc159e10 0x7f68fc0b1020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.229+0000 7f68fbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68fc15a1f0 0x7f68fc0cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.229+0000 7f68fbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc15d810 con 0x7f68fc15a1f0 2024-04-20T10:31:14.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.229+0000 7f68fbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68fc15d530 con 0x7f68fc06da90 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.229+0000 7f68fbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68fc06b900 con 0x7f68fc159e10 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.229+0000 7f68fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68fc15a1f0 0x7f68fc0cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68fc15a1f0 0x7f68fc0cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53526/0 (socket says 172.21.15.47:53526) 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 -- 172.21.15.47:0/2307154635 learned_addr learned my addr 172.21.15.47:0/2307154635 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68faa00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc06da90 0x7f68fc15a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68f2000640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc159e10 0x7f68fc0b1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 -- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc159e10 msgr2=0x7f68fc0b1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc159e10 0x7f68fc0b1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 -- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc06da90 msgr2=0x7f68fc15a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc06da90 0x7f68fc15a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 -- 172.21.15.47:0/2307154635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68fc0ca150 con 0x7f68fc15a1f0 2024-04-20T10:31:14.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.230+0000 7f68fb400640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68fc15a1f0 0x7f68fc0cada0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f68f400c400 tx=0x7f68f400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:14.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.231+0000 7f68f3400640 1 -- 172.21.15.47:0/2307154635 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4017070 con 0x7f68fc15a1f0 2024-04-20T10:31:14.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.231+0000 7f68f3400640 1 -- 172.21.15.47:0/2307154635 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f68f40052d0 con 0x7f68fc15a1f0 2024-04-20T10:31:14.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.231+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68fc0ca430 con 0x7f68fc15a1f0 2024-04-20T10:31:14.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.231+0000 7f68f3400640 1 -- 172.21.15.47:0/2307154635 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f4003810 con 0x7f68fc15a1f0 2024-04-20T10:31:14.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.231+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68fc15ab10 con 0x7f68fc15a1f0 2024-04-20T10:31:14.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.232+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68fc0ca600 con 0x7f68fc15a1f0 2024-04-20T10:31:14.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.233+0000 7f68f3400640 1 -- 172.21.15.47:0/2307154635 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f68f401a070 con 0x7f68fc15a1f0 2024-04-20T10:31:14.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.234+0000 7f68f3400640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68d806c8f0 0x7f68d806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.234+0000 7f68f3400640 1 -- 172.21.15.47:0/2307154635 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f68f408ae80 con 0x7f68fc15a1f0 2024-04-20T10:31:14.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.235+0000 7f68faa00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68d806c8f0 0x7f68d806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.236+0000 7f68faa00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68d806c8f0 0x7f68d806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f68ec036000 tx=0x7f68ec03b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:14.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.239+0000 7f68f3400640 1 -- 172.21.15.47:0/2307154635 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f68fc0ca600 con 0x7f68fc15a1f0 2024-04-20T10:31:14.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.440+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68fc15c350 con 0x7f68d806c8f0 2024-04-20T10:31:14.443 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.443+0000 7f68f3400640 1 -- 172.21.15.47:0/2307154635 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7f68fc15c350 con 0x7f68d806c8f0 2024-04-20T10:31:14.443 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68d806c8f0 msgr2=0x7f68d806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68d806c8f0 0x7f68d806edb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f68ec036000 tx=0x7f68ec03b040 comp rx=0 tx=0).stop 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68fc15a1f0 msgr2=0x7f68fc0cada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68fc15a1f0 0x7f68fc0cada0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f68f400c400 tx=0x7f68f400c8c0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 shutdown_connections 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f68d806c8f0 0x7f68d806edb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f68fc15a1f0 0x7f68fc0cada0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68fc159e10 0x7f68fc0b1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 --2- 172.21.15.47:0/2307154635 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68fc06da90 0x7f68fc15a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.445+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 >> 172.21.15.47:0/2307154635 conn(0x7f68fc0d0040 msgr2=0x7f68fc072de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.446+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 shutdown_connections 2024-04-20T10:31:14.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.446+0000 7f68fbe00640 1 -- 172.21.15.47:0/2307154635 wait complete. 2024-04-20T10:31:14.466 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-20T10:31:13.148448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239488,"kb_used_data":6688,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735232,"statfs":{"total":773094113280,"available":772848877568,"internally_reserved":0,"allocated":6848512,"data_stored":3701810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12369,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-92,"num_write":-64,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001955"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":238,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T10:31:08.782096+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:31:08.782096+0000","last_peered":"2024-04-20T10:31:08.782096+0000","last_clean":"2024-04-20T10:31:08.782096+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:31:08.782096+0000","last_undegraded":"2024-04-20T10:31:08.782096+0000","last_fullsized":"2024-04-20T10:31:08.782096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":312140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":312624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":914071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":902480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":323663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:31:14.466 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-20T10:31:14.466 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:31:14.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.678+0000 7fee38400640 1 Processor -- start 2024-04-20T10:31:14.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.678+0000 7fee38400640 1 -- start start 2024-04-20T10:31:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee38400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee3006b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee38400640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fee3015ca40 0x7fee300ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee38400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee300ad8e0 0x7fee300d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee38400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fee30141c60 con 0x7fee300ad8e0 2024-04-20T10:31:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee38400640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fee3015cf90 con 0x7fee3015ca40 2024-04-20T10:31:14.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee38400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee3015d270 con 0x7fee30156c20 2024-04-20T10:31:14.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee36400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee300ad8e0 0x7fee300d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee36400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fee300ad8e0 0x7fee300d0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53538/0 (socket says 172.21.15.47:53538) 2024-04-20T10:31:14.680 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.679+0000 7fee36400640 1 -- 172.21.15.47:0/1895469841 learned_addr learned my addr 172.21.15.47:0/1895469841 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee35a00640 1 --2- 172.21.15.47:0/1895469841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee3006b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee35000640 1 --2- 172.21.15.47:0/1895469841 >> v2:172.21.15.175:3300/0 conn(0x7fee3015ca40 0x7fee300ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee36400640 1 -- 172.21.15.47:0/1895469841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 msgr2=0x7fee3006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee36400640 1 --2- 172.21.15.47:0/1895469841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee3006b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee36400640 1 -- 172.21.15.47:0/1895469841 >> v2:172.21.15.175:3300/0 conn(0x7fee3015ca40 msgr2=0x7fee300ad3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee36400640 1 --2- 172.21.15.47:0/1895469841 >> v2:172.21.15.175:3300/0 conn(0x7fee3015ca40 0x7fee300ad3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee36400640 1 -- 172.21.15.47:0/1895469841 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee3015d550 con 0x7fee300ad8e0 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee35a00640 1 --2- 172.21.15.47:0/1895469841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee3006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.680+0000 7fee35000640 1 --2- 172.21.15.47:0/1895469841 >> v2:172.21.15.175:3300/0 conn(0x7fee3015ca40 0x7fee300ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:14.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.681+0000 7fee36400640 1 --2- 172.21.15.47:0/1895469841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee300d0360 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fee3006fe70 tx=0x7fee2802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a781b0a1cc7fea server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:14.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.681+0000 7fee2fe00640 1 -- 172.21.15.47:0/1895469841 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2802ca80 con 0x7fee300ad8e0 2024-04-20T10:31:14.682 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.681+0000 7fee2fe00640 1 -- 172.21.15.47:0/1895469841 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee2802cc20 con 0x7fee300ad8e0 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.681+0000 7fee2fe00640 1 -- 172.21.15.47:0/1895469841 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280343f0 con 0x7fee300ad8e0 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 -- 172.21.15.47:0/1895469841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 msgr2=0x7fee300d0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 --2- 172.21.15.47:0/1895469841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee300d0360 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fee3006fe70 tx=0x7fee2802bee0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 -- 172.21.15.47:0/1895469841 shutdown_connections 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 --2- 172.21.15.47:0/1895469841 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee300d0360 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 --2- 172.21.15.47:0/1895469841 >> v2:172.21.15.175:3300/0 conn(0x7fee3015ca40 0x7fee300ad3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 --2- 172.21.15.47:0/1895469841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee3006b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 -- 172.21.15.47:0/1895469841 >> 172.21.15.47:0/1895469841 conn(0x7fee300b3350 msgr2=0x7fee300b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 -- 172.21.15.47:0/1895469841 shutdown_connections 2024-04-20T10:31:14.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.682+0000 7fee38400640 1 -- 172.21.15.47:0/1895469841 wait complete. 2024-04-20T10:31:14.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.684+0000 7fee38400640 1 Processor -- start 2024-04-20T10:31:14.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.684+0000 7fee38400640 1 -- start start 2024-04-20T10:31:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee38400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee301f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee38400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee30073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee38400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3015ca40 0x7fee30073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee38400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fee3006bfc0 con 0x7fee300ad8e0 2024-04-20T10:31:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee38400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee3006be40 con 0x7fee3015ca40 2024-04-20T10:31:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee38400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee300d08a0 con 0x7fee30156c20 2024-04-20T10:31:14.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee35a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee301f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee35a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee301f41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53552/0 (socket says 172.21.15.47:53552) 2024-04-20T10:31:14.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee35a00640 1 -- 172.21.15.47:0/1354066071 learned_addr learned my addr 172.21.15.47:0/1354066071 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee35a00640 1 -- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 msgr2=0x7fee30073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee35a00640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee30073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee35a00640 1 -- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3015ca40 msgr2=0x7fee30073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.685+0000 7fee35a00640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3015ca40 0x7fee30073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.686+0000 7fee35a00640 1 -- 172.21.15.47:0/1354066071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee300cf980 con 0x7fee300ad8e0 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.686+0000 7fee35a00640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee301f41a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c7f0 tx=0x7fee2400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.686+0000 7fee2d600640 1 -- 172.21.15.47:0/1354066071 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24015070 con 0x7fee300ad8e0 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.686+0000 7fee2d600640 1 -- 172.21.15.47:0/1354066071 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fee240079d0 con 0x7fee300ad8e0 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.686+0000 7fee2d600640 1 -- 172.21.15.47:0/1354066071 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee24007da0 con 0x7fee300ad8e0 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.686+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee301f46e0 con 0x7fee300ad8e0 2024-04-20T10:31:14.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.686+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301543a0 con 0x7fee300ad8e0 2024-04-20T10:31:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.688+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee30068980 con 0x7fee300ad8e0 2024-04-20T10:31:14.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.689+0000 7fee2d600640 1 -- 172.21.15.47:0/1354066071 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fee24018070 con 0x7fee300ad8e0 2024-04-20T10:31:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.690+0000 7fee2d600640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1006c940 0x7fee1006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.690+0000 7fee2d600640 1 -- 172.21.15.47:0/1354066071 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fee2408b8b0 con 0x7fee300ad8e0 2024-04-20T10:31:14.690 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.691+0000 7fee35000640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1006c940 0x7fee1006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:14.692 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.692+0000 7fee35000640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1006c940 0x7fee1006ee00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fee1c006fd0 tx=0x7fee1c008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:14.695 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.695+0000 7fee2d600640 1 -- 172.21.15.47:0/1354066071 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fee300cf980 con 0x7fee300ad8e0 2024-04-20T10:31:14.894 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.894+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fee30151550 con 0x7fee1006c940 2024-04-20T10:31:14.896 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.896+0000 7fee2d600640 1 -- 172.21.15.47:0/1354066071 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13495 (secure 0 0 0) 0x7fee30151550 con 0x7fee1006c940 2024-04-20T10:31:14.897 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:14.897 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:31:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1006c940 msgr2=0x7fee1006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.899 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1006c940 0x7fee1006ee00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fee1c006fd0 tx=0x7fee1c008040 comp rx=0 tx=0).stop 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 msgr2=0x7fee301f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee301f41a0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fee2400c7f0 tx=0x7fee2400ccb0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 shutdown_connections 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee3015ca40 0x7fee30073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee30156c20 0x7fee30073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fee1006c940 0x7fee1006ee00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 --2- 172.21.15.47:0/1354066071 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fee300ad8e0 0x7fee301f41a0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 >> 172.21.15.47:0/1354066071 conn(0x7fee300b3350 msgr2=0x7fee3015f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 shutdown_connections 2024-04-20T10:31:14.900 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:14.899+0000 7fee38400640 1 -- 172.21.15.47:0/1354066071 wait complete. 2024-04-20T10:31:14.916 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-04-20T10:31:13.148448+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239488,"kb_used_data":6688,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735232,"statfs":{"total":773094113280,"available":772848877568,"internally_reserved":0,"allocated":6848512,"data_stored":3701810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12369,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-79,"num_read_kb":-92,"num_write":-64,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001955"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":238,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-20T10:31:08.782096+0000","last_change":"2024-04-20T10:27:23.056080+0000","last_active":"2024-04-20T10:31:08.782096+0000","last_peered":"2024-04-20T10:31:08.782096+0000","last_clean":"2024-04-20T10:31:08.782096+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:31:08.782096+0000","last_undegraded":"2024-04-20T10:31:08.782096+0000","last_fullsized":"2024-04-20T10:31:08.782096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-20T10:27:21.800918+0000","last_clean_scrub_stamp":"2024-04-20T10:27:21.800918+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:20:36.607052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509489,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27712,"kb_used_data":692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344128,"statfs":{"total":96636764160,"available":96608387072,"internally_reserved":0,"allocated":708608,"data_stored":312140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":700416,"data_stored":312624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1288,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1318912,"data_stored":914071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509489,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509489,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509489,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32444,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339396,"statfs":{"total":96636764160,"available":96603541504,"internally_reserved":0,"allocated":1294336,"data_stored":902480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509489,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":700416,"data_stored":312624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542193,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27728,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344112,"statfs":{"total":96636764160,"available":96608370688,"internally_reserved":0,"allocated":724992,"data_stored":323663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:31:14.917 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-20T10:31:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.120+0000 7f88be000640 1 Processor -- start 2024-04-20T10:31:15.121 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.121+0000 7f88be000640 1 -- start start 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.122+0000 7f88be000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88b8161d50 0x7f88b80d38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.122+0000 7f88be000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f88b806cde0 0x7f88b80b3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.122+0000 7f88be000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f88b806c190 con 0x7f88b80b3330 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.122+0000 7f88be000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f88b8162130 con 0x7f88b8161d50 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.122+0000 7f88be000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f88b806c470 con 0x7f88b806cde0 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.123+0000 7f88bcc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88b8161d50 0x7f88b80d38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.123+0000 7f88bcc00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f88b8161d50 0x7f88b80d38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53564/0 (socket says 172.21.15.47:53564) 2024-04-20T10:31:15.123 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.123+0000 7f88bcc00640 1 -- 172.21.15.47:0/4184411206 learned_addr learned my addr 172.21.15.47:0/4184411206 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.123+0000 7f88bcc00640 1 -- 172.21.15.47:0/4184411206 >> v2:172.21.15.175:3300/0 conn(0x7f88b806cde0 msgr2=0x7f88b80b3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.123+0000 7f88bcc00640 1 --2- 172.21.15.47:0/4184411206 >> v2:172.21.15.175:3300/0 conn(0x7f88b806cde0 0x7f88b80b3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.123+0000 7f88bcc00640 1 -- 172.21.15.47:0/4184411206 >> v1:172.21.15.175:6789/0 conn(0x7f88b80b3330 legacy=0x7f88b806c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.123+0000 7f88bcc00640 1 -- 172.21.15.47:0/4184411206 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b80d2e00 con 0x7f88b8161d50 2024-04-20T10:31:15.124 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.124+0000 7f88bcc00640 1 --2- 172.21.15.47:0/4184411206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b8161d50 0x7f88b80d38e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f88b80cee50 tx=0x7f88ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b99173e7925dc2f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88b7400640 1 -- 172.21.15.47:0/4184411206 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac035020 con 0x7f88b8161d50 2024-04-20T10:31:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88b7400640 1 -- 172.21.15.47:0/4184411206 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ac031870 con 0x7f88b8161d50 2024-04-20T10:31:15.125 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88b7400640 1 -- 172.21.15.47:0/4184411206 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac031c40 con 0x7f88b8161d50 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88be000640 1 -- 172.21.15.47:0/4184411206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b8161d50 msgr2=0x7f88b80d38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88be000640 1 --2- 172.21.15.47:0/4184411206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b8161d50 0x7f88b80d38e0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f88b80cee50 tx=0x7f88ac02b910 comp rx=0 tx=0).stop 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88be000640 1 -- 172.21.15.47:0/4184411206 shutdown_connections 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88be000640 1 --2- 172.21.15.47:0/4184411206 >> v2:172.21.15.175:3300/0 conn(0x7f88b806cde0 0x7f88b80b3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88be000640 1 --2- 172.21.15.47:0/4184411206 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b8161d50 0x7f88b80d38e0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.125+0000 7f88be000640 1 -- 172.21.15.47:0/4184411206 >> 172.21.15.47:0/4184411206 conn(0x7f88b80d4450 msgr2=0x7f88b80d4860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.126+0000 7f88be000640 1 -- 172.21.15.47:0/4184411206 shutdown_connections 2024-04-20T10:31:15.126 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.126+0000 7f88be000640 1 -- 172.21.15.47:0/4184411206 wait complete. 2024-04-20T10:31:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.127+0000 7f88be000640 1 Processor -- start 2024-04-20T10:31:15.127 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.127+0000 7f88be000640 1 -- start start 2024-04-20T10:31:15.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.128+0000 7f88be000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b806cde0 0x7f88b80cb520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:15.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.128+0000 7f88be000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88b80b3330 0x7f88b814d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:15.129 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.128+0000 7f88be000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8161d50 0x7f88b80ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.128+0000 7f88be000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80b7660 con 0x7f88b806cde0 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.128+0000 7f88be000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88b80d33c0 con 0x7f88b8161d50 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.128+0000 7f88be000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88b80b77e0 con 0x7f88b80b3330 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b806cde0 0x7f88b80cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b806cde0 0x7f88b80cb520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53580/0 (socket says 172.21.15.47:53580) 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 -- 172.21.15.47:0/2661422844 learned_addr learned my addr 172.21.15.47:0/2661422844 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88b7e00640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88b80b3330 0x7f88b814d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bd600640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8161d50 0x7f88b80ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 -- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88b80b3330 msgr2=0x7f88b814d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88b80b3330 0x7f88b814d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 -- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8161d50 msgr2=0x7f88b80ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8161d50 0x7f88b80ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bcc00640 1 -- 172.21.15.47:0/2661422844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b80b49f0 con 0x7f88b806cde0 2024-04-20T10:31:15.130 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.129+0000 7f88bd600640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8161d50 0x7f88b80ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:15.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.130+0000 7f88bcc00640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b806cde0 0x7f88b80cb520 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f88ac02b950 tx=0x7f88ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:15.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.130+0000 7f88b4c00640 1 -- 172.21.15.47:0/2661422844 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac03ad60 con 0x7f88b806cde0 2024-04-20T10:31:15.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.130+0000 7f88b4c00640 1 -- 172.21.15.47:0/2661422844 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f88ac0443f0 con 0x7f88b806cde0 2024-04-20T10:31:15.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.130+0000 7f88b4c00640 1 -- 172.21.15.47:0/2661422844 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac0447e0 con 0x7f88b806cde0 2024-04-20T10:31:15.131 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.130+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b80b4cd0 con 0x7f88b806cde0 2024-04-20T10:31:15.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.130+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b80c80b0 con 0x7f88b806cde0 2024-04-20T10:31:15.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.131+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b80b4ea0 con 0x7f88b806cde0 2024-04-20T10:31:15.132 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.132+0000 7f88b4c00640 1 -- 172.21.15.47:0/2661422844 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f88ac038020 con 0x7f88b806cde0 2024-04-20T10:31:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.134+0000 7f88b4c00640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f889806c8f0 0x7f889806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.134+0000 7f88b4c00640 1 -- 172.21.15.47:0/2661422844 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f88ac0ade30 con 0x7f88b806cde0 2024-04-20T10:31:15.134 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.134+0000 7f88b7e00640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f889806c8f0 0x7f889806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:15.135 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.136+0000 7f88b7e00640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f889806c8f0 0x7f889806edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f88a4007fd0 tx=0x7f88a4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:15.138 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.138+0000 7f88b4c00640 1 -- 172.21.15.47:0/2661422844 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f88ac03f070 con 0x7f88b806cde0 2024-04-20T10:31:15.340 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.340+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88b80d2690 con 0x7f88b806cde0 2024-04-20T10:31:15.341 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.341+0000 7f88b4c00640 1 -- 172.21.15.47:0/2661422844 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+13561 (secure 0 0 0) 0x7f88ac07c1f0 con 0x7f88b806cde0 2024-04-20T10:31:15.341 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:15.342 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":61,"fsid":"775946e6-93c9-4843-aaf4-db758801eec1","created":"2024-04-20T10:27:07.754583+0000","modified":"2024-04-20T10:31:11.612488+0000","last_up_change":"2024-04-20T10:27:19.792108+0000","last_in_change":"2024-04-20T10:27:11.060578+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-20T10:27:21.096307+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"42adcc33-e207-4065-962d-c429043222a8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6800","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6801","nonce":2647753696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6802","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6804","nonce":2647753696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6811","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6814","nonce":2647753696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6807","nonce":2647753696},{"type":"v1","addr":"172.21.15.47:6809","nonce":2647753696}]},"public_addr":"172.21.15.47:6801/2647753696","cluster_addr":"172.21.15.47:6804/2647753696","heartbeat_back_addr":"172.21.15.47:6814/2647753696","heartbeat_front_addr":"172.21.15.47:6809/2647753696","state":["exists","up"]},{"osd":1,"uuid":"e74aaa9e-5db8-4f77-8235-5a57636e3b53","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6803","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6805","nonce":2415890922}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6806","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6808","nonce":2415890922}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6816","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6818","nonce":2415890922}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6810","nonce":2415890922},{"type":"v1","addr":"172.21.15.47:6813","nonce":2415890922}]},"public_addr":"172.21.15.47:6805/2415890922","cluster_addr":"172.21.15.47:6808/2415890922","heartbeat_back_addr":"172.21.15.47:6818/2415890922","heartbeat_front_addr":"172.21.15.47:6813/2415890922","state":["exists","up"]},{"osd":2,"uuid":"509d1c2b-3974-4e6e-bf6c-2b3861e20c72","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6822","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6824","nonce":2832687130}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6826","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6827","nonce":2832687130}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6830","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6831","nonce":2832687130}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6828","nonce":2832687130},{"type":"v1","addr":"172.21.15.47:6829","nonce":2832687130}]},"public_addr":"172.21.15.47:6824/2832687130","cluster_addr":"172.21.15.47:6827/2832687130","heartbeat_back_addr":"172.21.15.47:6831/2832687130","heartbeat_front_addr":"172.21.15.47:6829/2832687130","state":["exists","up"]},{"osd":3,"uuid":"5e63812e-7c05-4655-8d10-89aaccfc408c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6812","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6815","nonce":4183939241}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6817","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6819","nonce":4183939241}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6823","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6825","nonce":4183939241}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6820","nonce":4183939241},{"type":"v1","addr":"172.21.15.47:6821","nonce":4183939241}]},"public_addr":"172.21.15.47:6815/4183939241","cluster_addr":"172.21.15.47:6819/4183939241","heartbeat_back_addr":"172.21.15.47:6825/4183939241","heartbeat_front_addr":"172.21.15.47:6821/4183939241","state":["exists","up"]},{"osd":4,"uuid":"ead50013-a94f-44b0-9051-4e01a5858c4e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6817","nonce":1401363349}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6819","nonce":1401363349}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6823","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6825","nonce":1401363349}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":1401363349},{"type":"v1","addr":"172.21.15.175:6821","nonce":1401363349}]},"public_addr":"172.21.15.175:6817/1401363349","cluster_addr":"172.21.15.175:6819/1401363349","heartbeat_back_addr":"172.21.15.175:6825/1401363349","heartbeat_front_addr":"172.21.15.175:6821/1401363349","state":["exists","up"]},{"osd":5,"uuid":"5335fadb-0129-4fc5-a293-1c3a317097b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6801","nonce":1477827144}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6803","nonce":1477827144}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6807","nonce":1477827144}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1477827144},{"type":"v1","addr":"172.21.15.175:6805","nonce":1477827144}]},"public_addr":"172.21.15.175:6801/1477827144","cluster_addr":"172.21.15.175:6803/1477827144","heartbeat_back_addr":"172.21.15.175:6807/1477827144","heartbeat_front_addr":"172.21.15.175:6805/1477827144","state":["exists","up"]},{"osd":6,"uuid":"869d63f5-c8d5-485c-a632-d0405aef964d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6822","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6824","nonce":2500655665}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6826","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6827","nonce":2500655665}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6830","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6831","nonce":2500655665}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":2500655665},{"type":"v1","addr":"172.21.15.175:6829","nonce":2500655665}]},"public_addr":"172.21.15.175:6824/2500655665","cluster_addr":"172.21.15.175:6827/2500655665","heartbeat_back_addr":"172.21.15.175:6831/2500655665","heartbeat_front_addr":"172.21.15.175:6829/2500655665","state":["exists","up"]},{"osd":7,"uuid":"4bcce85d-9d35-460a-ae67-821f60715c81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6809","nonce":1688864180}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6811","nonce":1688864180}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6815","nonce":1688864180}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1688864180},{"type":"v1","addr":"172.21.15.175:6813","nonce":1688864180}]},"public_addr":"172.21.15.175:6809/1688864180","cluster_addr":"172.21.15.175:6811/1688864180","heartbeat_back_addr":"172.21.15.175:6815/1688864180","heartbeat_front_addr":"172.21.15.175:6813/1688864180","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.396306+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.376173+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.672235+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.712553+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.041019+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.236662+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.435949+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-20T10:27:16.490884+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.47:6833/566678338":"2024-04-21T10:31:08.428438+0000","172.21.15.47:6832/566678338":"2024-04-21T10:31:08.428438+0000","172.21.15.47:6833/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6832/793057653":"2024-04-21T10:30:54.470443+0000","172.21.15.47:6835/3814505199":"2024-04-21T10:30:53.464895+0000","172.21.15.47:6834/3814505199":"2024-04-21T10:30:53.464895+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-20T10:31:15.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f889806c8f0 msgr2=0x7f889806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:15.344 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f889806c8f0 0x7f889806edb0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f88a4007fd0 tx=0x7f88a4009040 comp rx=0 tx=0).stop 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b806cde0 msgr2=0x7f88b80cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b806cde0 0x7f88b80cb520 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f88ac02b950 tx=0x7f88ac0046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 shutdown_connections 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88b8161d50 0x7f88b80ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88b80b3330 0x7f88b814d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f889806c8f0 0x7f889806edb0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 --2- 172.21.15.47:0/2661422844 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f88b806cde0 0x7f88b80cb520 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 >> 172.21.15.47:0/2661422844 conn(0x7f88b80d4450 msgr2=0x7f88b80b7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 shutdown_connections 2024-04-20T10:31:15.345 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:15.344+0000 7f88be000640 1 -- 172.21.15.47:0/2661422844 wait complete. 2024-04-20T10:31:16.364 INFO:tasks.ceph:Scrubbing osd.0 2024-04-20T10:31:16.364 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.590+0000 7f6339200640 1 Processor -- start 2024-04-20T10:31:16.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.591+0000 7f6339200640 1 -- start start 2024-04-20T10:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.592+0000 7f6339200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f633415c400 0x7f633406f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.593+0000 7f6339200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f633406de90 con 0x7f633406d020 2024-04-20T10:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.593+0000 7f6339200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6334072130 con 0x7f633406d920 2024-04-20T10:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.593+0000 7f6339200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f633406e170 con 0x7f633415c400 2024-04-20T10:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.593+0000 7f6333400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f633415c400 0x7f633406f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.593+0000 7f6333e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f633406d020 0x7f633415c7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34030/0 (socket says 172.21.15.47:34030) 2024-04-20T10:31:16.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.593+0000 7f6333e00640 1 -- 172.21.15.47:0/745088318 learned_addr learned my addr 172.21.15.47:0/745088318 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.594+0000 7f6332000640 1 -- 172.21.15.47:0/745088318 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4076005245 0 0) 0x7f633406de90 con 0x7f633406d020 2024-04-20T10:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.594+0000 7f6332000640 1 -- 172.21.15.47:0/745088318 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6318003520 con 0x7f633406d020 2024-04-20T10:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.594+0000 7f6333400640 1 -- 172.21.15.47:0/745088318 >> v1:172.21.15.175:6789/0 conn(0x7f633406d920 legacy=0x7f633406c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.594+0000 7f6333400640 1 -- 172.21.15.47:0/745088318 >> v1:172.21.15.47:6789/0 conn(0x7f633406d020 legacy=0x7f633415c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.594+0000 7f6333400640 1 -- 172.21.15.47:0/745088318 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6334072410 con 0x7f633415c400 2024-04-20T10:31:16.595 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.595+0000 7f6333400640 1 --2- 172.21.15.47:0/745088318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633415c400 0x7f633406f570 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63240097a0 tx=0x7f632402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d63a4c15c1ccac98 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.596+0000 7f6332000640 1 -- 172.21.15.47:0/745088318 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6324034070 con 0x7f633415c400 2024-04-20T10:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.596+0000 7f6332000640 1 -- 172.21.15.47:0/745088318 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6324030610 con 0x7f633415c400 2024-04-20T10:31:16.596 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.596+0000 7f6332000640 1 -- 172.21.15.47:0/745088318 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63240309c0 con 0x7f633415c400 2024-04-20T10:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.596+0000 7f6339200640 1 -- 172.21.15.47:0/745088318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633415c400 msgr2=0x7f633406f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.596+0000 7f6339200640 1 --2- 172.21.15.47:0/745088318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633415c400 0x7f633406f570 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f63240097a0 tx=0x7f632402b960 comp rx=0 tx=0).stop 2024-04-20T10:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.597+0000 7f6339200640 1 -- 172.21.15.47:0/745088318 shutdown_connections 2024-04-20T10:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.597+0000 7f6339200640 1 --2- 172.21.15.47:0/745088318 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633415c400 0x7f633406f570 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.597+0000 7f6339200640 1 -- 172.21.15.47:0/745088318 >> 172.21.15.47:0/745088318 conn(0x7f63340cc9b0 msgr2=0x7f63340ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.597+0000 7f6339200640 1 -- 172.21.15.47:0/745088318 shutdown_connections 2024-04-20T10:31:16.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.597+0000 7f6339200640 1 -- 172.21.15.47:0/745088318 wait complete. 2024-04-20T10:31:16.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.598+0000 7f6339200640 1 Processor -- start 2024-04-20T10:31:16.598 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.599+0000 7f6339200640 1 -- start start 2024-04-20T10:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6339200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f633406d020 0x7f63341f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.601 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6339200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633406d920 0x7f633415f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6339200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f633415c400 0x7f63340c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6339200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6334072780 con 0x7f633406d920 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6339200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6334071ee0 con 0x7f633415c400 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6339200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6334072c10 con 0x7f633406d020 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6332a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633406d920 0x7f633415f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6332a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633406d920 0x7f633415f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53592/0 (socket says 172.21.15.47:53592) 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6332a00640 1 -- 172.21.15.47:0/3834756130 learned_addr learned my addr 172.21.15.47:0/3834756130 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6333400640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f633406d020 0x7f63341f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.600+0000 7f6333e00640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f633415c400 0x7f63340c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6332a00640 1 -- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f633406d020 msgr2=0x7f63341f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6332a00640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f633406d020 0x7f63341f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6332a00640 1 -- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f633415c400 msgr2=0x7f63340c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.602 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6332a00640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f633415c400 0x7f63340c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6332a00640 1 -- 172.21.15.47:0/3834756130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63340b38b0 con 0x7f633406d920 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6333e00640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f633415c400 0x7f63340c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6332a00640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633406d920 0x7f633415f4c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c480 tx=0x7f631c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c0052d0 con 0x7f633406d920 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f631c005470 con 0x7f633406d920 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63340b3b90 con 0x7f633406d920 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.601+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631c012870 con 0x7f633406d920 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.602+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63340b4150 con 0x7f633406d920 2024-04-20T10:31:16.603 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.602+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f63340b44c0 con 0x7f633406d920 2024-04-20T10:31:16.604 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.604+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f631c012a10 con 0x7f633406d920 2024-04-20T10:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.605+0000 7f632b400640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f631006c920 0x7f631006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.605+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f631c08b740 con 0x7f633406d920 2024-04-20T10:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.605+0000 7f632b400640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7f6310072f00 0x7f6310075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.605+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 --> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f631c004ec0 con 0x7f6310072f00 2024-04-20T10:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.605+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f631c092050 con 0x7f633406d920 2024-04-20T10:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.605+0000 7f6333400640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f631006c920 0x7f631006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.605 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.605+0000 7f6333e00640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7f6310072f00 0x7f6310075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.606 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.606+0000 7f6333e00640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7f6310072f00 0x7f6310075340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.607+0000 7f6333400640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f631006c920 0x7f631006ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6324036000 tx=0x7f6324008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.610 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.607+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== osd.0 v2:172.21.15.47:6800/2647753696 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f631c004ec0 con 0x7f6310072f00 2024-04-20T10:31:16.621 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.621+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 --> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6334075b30 con 0x7f6310072f00 2024-04-20T10:31:16.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.622+0000 7f632b400640 1 -- 172.21.15.47:0/3834756130 <== osd.0 v2:172.21.15.47:6800/2647753696 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6334075b30 con 0x7f6310072f00 2024-04-20T10:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:16.622 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:16.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.622+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7f6310072f00 msgr2=0x7f6310075340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.622 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.622+0000 7f6339200640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:6800/2647753696,v1:172.21.15.47:6801/2647753696] conn(0x7f6310072f00 0x7f6310075340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f631006c920 msgr2=0x7f631006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f631006c920 0x7f631006ede0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6324036000 tx=0x7f6324008030 comp rx=0 tx=0).stop 2024-04-20T10:31:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633406d920 msgr2=0x7f633415f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 --2- 172.21.15.47:0/3834756130 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f633406d920 0x7f633415f4c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f631c00c480 tx=0x7f631c00c940 comp rx=0 tx=0).stop 2024-04-20T10:31:16.623 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6333e00640 1 -- 172.21.15.47:0/3834756130 reap_dead start 2024-04-20T10:31:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 shutdown_connections 2024-04-20T10:31:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 >> 172.21.15.47:0/3834756130 conn(0x7f63340cc9b0 msgr2=0x7f63340ccd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 shutdown_connections 2024-04-20T10:31:16.624 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.623+0000 7f6339200640 1 -- 172.21.15.47:0/3834756130 wait complete. 2024-04-20T10:31:16.644 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-20T10:31:16.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.858+0000 7fd533200640 1 Processor -- start 2024-04-20T10:31:16.859 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.859+0000 7fd533200640 1 -- start start 2024-04-20T10:31:16.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.860+0000 7fd533200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd52c159e10 0x7fd52c071a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.860+0000 7fd533200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd52c160e50 con 0x7fd52c06da90 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.860+0000 7fd533200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd52c15cc90 con 0x7fd52c15a1f0 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.860+0000 7fd533200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fd52c072900 con 0x7fd52c159e10 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.861+0000 7fd531e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd52c159e10 0x7fd52c071a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.861+0000 7fd531e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fd52c159e10 0x7fd52c071a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53596/0 (socket says 172.21.15.47:53596) 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.861+0000 7fd531e00640 1 -- 172.21.15.47:0/2675135361 learned_addr learned my addr 172.21.15.47:0/2675135361 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.861+0000 7fd531e00640 1 -- 172.21.15.47:0/2675135361 >> v1:172.21.15.175:6789/0 conn(0x7fd52c15a1f0 legacy=0x7fd52c160030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.861 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.861+0000 7fd531e00640 1 -- 172.21.15.47:0/2675135361 >> v1:172.21.15.47:6789/0 conn(0x7fd52c06da90 legacy=0x7fd52c160740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.861+0000 7fd531e00640 1 -- 172.21.15.47:0/2675135361 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c15cf70 con 0x7fd52c159e10 2024-04-20T10:31:16.862 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.862+0000 7fd531e00640 1 --2- 172.21.15.47:0/2675135361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c071a80 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd520009870 tx=0x7fd52002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18a6c68177ca00df server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.863+0000 7fd530a00640 1 -- 172.21.15.47:0/2675135361 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52002f850 con 0x7fd52c159e10 2024-04-20T10:31:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.863+0000 7fd530a00640 1 -- 172.21.15.47:0/2675135361 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd52002f9f0 con 0x7fd52c159e10 2024-04-20T10:31:16.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.863+0000 7fd530a00640 1 -- 172.21.15.47:0/2675135361 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd52002fdc0 con 0x7fd52c159e10 2024-04-20T10:31:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.863+0000 7fd533200640 1 -- 172.21.15.47:0/2675135361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 msgr2=0x7fd52c071a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.863+0000 7fd533200640 1 --2- 172.21.15.47:0/2675135361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c071a80 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fd520009870 tx=0x7fd52002ba30 comp rx=0 tx=0).stop 2024-04-20T10:31:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.864+0000 7fd533200640 1 -- 172.21.15.47:0/2675135361 shutdown_connections 2024-04-20T10:31:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.864+0000 7fd533200640 1 --2- 172.21.15.47:0/2675135361 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c071a80 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.864+0000 7fd533200640 1 -- 172.21.15.47:0/2675135361 >> 172.21.15.47:0/2675135361 conn(0x7fd52c0d0040 msgr2=0x7fd52c0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.864+0000 7fd533200640 1 -- 172.21.15.47:0/2675135361 shutdown_connections 2024-04-20T10:31:16.864 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.864+0000 7fd533200640 1 -- 172.21.15.47:0/2675135361 wait complete. 2024-04-20T10:31:16.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.865+0000 7fd533200640 1 Processor -- start 2024-04-20T10:31:16.865 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.866+0000 7fd533200640 1 -- start start 2024-04-20T10:31:16.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.866+0000 7fd533200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52c06da90 0x7fd52c15c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.866 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.866+0000 7fd533200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c0b4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd533200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd52c15a1f0 0x7fd52c149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd533200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c15d530 con 0x7fd52c159e10 2024-04-20T10:31:16.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd533200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd52c15ca40 con 0x7fd52c15a1f0 2024-04-20T10:31:16.867 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd533200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd52c15d810 con 0x7fd52c06da90 2024-04-20T10:31:16.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c0b4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52c06da90 0x7fd52c15c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c0b4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53598/0 (socket says 172.21.15.47:53598) 2024-04-20T10:31:16.868 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531400640 1 -- 172.21.15.47:0/3930108956 learned_addr learned my addr 172.21.15.47:0/3930108956 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd532800640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd52c15a1f0 0x7fd52c149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531400640 1 -- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52c06da90 msgr2=0x7fd52c15c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531400640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52c06da90 0x7fd52c15c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531400640 1 -- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd52c15a1f0 msgr2=0x7fd52c149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.867+0000 7fd531400640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd52c15a1f0 0x7fd52c149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.868+0000 7fd531400640 1 -- 172.21.15.47:0/3930108956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd52c0b4ec0 con 0x7fd52c159e10 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.868+0000 7fd531400640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c0b4470 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c440 tx=0x7fd51c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.868+0000 7fd526000640 1 -- 172.21.15.47:0/3930108956 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c0052d0 con 0x7fd52c159e10 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.868+0000 7fd526000640 1 -- 172.21.15.47:0/3930108956 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd51c005470 con 0x7fd52c159e10 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.868+0000 7fd526000640 1 -- 172.21.15.47:0/3930108956 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51c007260 con 0x7fd52c159e10 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.868+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd52c1535d0 con 0x7fd52c159e10 2024-04-20T10:31:16.869 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.868+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd52c14c230 con 0x7fd52c159e10 2024-04-20T10:31:16.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.869+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd52c075b60 con 0x7fd52c159e10 2024-04-20T10:31:16.870 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.870+0000 7fd526000640 1 -- 172.21.15.47:0/3930108956 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fd51c007400 con 0x7fd52c159e10 2024-04-20T10:31:16.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.871+0000 7fd526000640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd51006c8f0 0x7fd51006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:16.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.871+0000 7fd526000640 1 -- 172.21.15.47:0/3930108956 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fd51c08aee0 con 0x7fd52c159e10 2024-04-20T10:31:16.871 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.872+0000 7fd531e00640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd51006c8f0 0x7fd51006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:16.873 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.873+0000 7fd531e00640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd51006c8f0 0x7fd51006edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd520036000 tx=0x7fd520008030 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:16.875 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:16.876+0000 7fd526000640 1 -- 172.21.15.47:0/3930108956 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fd51c058ce0 con 0x7fd52c159e10 2024-04-20T10:31:17.082 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.082+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7fd52c0b0b10 con 0x7fd51006c8f0 2024-04-20T10:31:17.083 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.083+0000 7fd526000640 1 -- 172.21.15.47:0/3930108956 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fd52c0b0b10 con 0x7fd51006c8f0 2024-04-20T10:31:17.083 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 0 to deep-scrub 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd51006c8f0 msgr2=0x7fd51006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd51006c8f0 0x7fd51006edb0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fd520036000 tx=0x7fd520008030 comp rx=0 tx=0).stop 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 msgr2=0x7fd52c0b4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c0b4470 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fd51c00c440 tx=0x7fd51c00c900 comp rx=0 tx=0).stop 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 shutdown_connections 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd52c15a1f0 0x7fd52c149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fd51006c8f0 0x7fd51006edb0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fd52c159e10 0x7fd52c0b4470 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.086 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.085+0000 7fd533200640 1 --2- 172.21.15.47:0/3930108956 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52c06da90 0x7fd52c15c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.086+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 >> 172.21.15.47:0/3930108956 conn(0x7fd52c0d0040 msgr2=0x7fd52c071ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.086+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 shutdown_connections 2024-04-20T10:31:17.087 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.086+0000 7fd533200640 1 -- 172.21.15.47:0/3930108956 wait complete. 2024-04-20T10:31:17.104 INFO:tasks.ceph:Scrubbing osd.1 2024-04-20T10:31:17.105 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:17.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.317+0000 7fbf59800640 1 Processor -- start 2024-04-20T10:31:17.318 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.318+0000 7fbf59800640 1 -- start start 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.319+0000 7fbf59800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf540d2460 0x7fbf540d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.319+0000 7fbf59800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fbf540d4e40 0x7fbf541617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.319+0000 7fbf59800640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbf540d2e30 con 0x7fbf540d2840 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.319+0000 7fbf59800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fbf540d3110 con 0x7fbf540d2460 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.319+0000 7fbf59800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fbf540d33f0 con 0x7fbf540d4e40 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf540d2460 0x7fbf540d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fbf540d2460 0x7fbf540d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53608/0 (socket says 172.21.15.47:53608) 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53e00640 1 -- 172.21.15.47:0/3234950038 learned_addr learned my addr 172.21.15.47:0/3234950038 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53400640 1 --2- 172.21.15.47:0/3234950038 >> v2:172.21.15.175:3300/0 conn(0x7fbf540d4e40 0x7fbf541617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53e00640 1 -- 172.21.15.47:0/3234950038 >> v2:172.21.15.175:3300/0 conn(0x7fbf540d4e40 msgr2=0x7fbf541617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:17.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53e00640 1 --2- 172.21.15.47:0/3234950038 >> v2:172.21.15.175:3300/0 conn(0x7fbf540d4e40 0x7fbf541617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53e00640 1 -- 172.21.15.47:0/3234950038 >> v1:172.21.15.175:6789/0 conn(0x7fbf540d2840 legacy=0x7fbf540b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:17.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.320+0000 7fbf53e00640 1 -- 172.21.15.47:0/3234950038 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf540d36d0 con 0x7fbf540d2460 2024-04-20T10:31:17.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.321+0000 7fbf53e00640 1 --2- 172.21.15.47:0/3234950038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2460 0x7fbf540d4900 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbf5408c9e0 tx=0x7fbf4402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5caa87e90cd8ea00 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf52a00640 1 -- 172.21.15.47:0/3234950038 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44034070 con 0x7fbf540d2460 2024-04-20T10:31:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf52a00640 1 -- 172.21.15.47:0/3234950038 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf44030860 con 0x7fbf540d2460 2024-04-20T10:31:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf52a00640 1 -- 172.21.15.47:0/3234950038 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44030c30 con 0x7fbf540d2460 2024-04-20T10:31:17.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 -- 172.21.15.47:0/3234950038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2460 msgr2=0x7fbf540d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 --2- 172.21.15.47:0/3234950038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2460 0x7fbf540d4900 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fbf5408c9e0 tx=0x7fbf4402b9a0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 -- 172.21.15.47:0/3234950038 shutdown_connections 2024-04-20T10:31:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 --2- 172.21.15.47:0/3234950038 >> v2:172.21.15.175:3300/0 conn(0x7fbf540d4e40 0x7fbf541617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 --2- 172.21.15.47:0/3234950038 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2460 0x7fbf540d4900 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 -- 172.21.15.47:0/3234950038 >> 172.21.15.47:0/3234950038 conn(0x7fbf540b7570 msgr2=0x7fbf540b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 -- 172.21.15.47:0/3234950038 shutdown_connections 2024-04-20T10:31:17.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.322+0000 7fbf59800640 1 -- 172.21.15.47:0/3234950038 wait complete. 2024-04-20T10:31:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.324+0000 7fbf59800640 1 Processor -- start 2024-04-20T10:31:17.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.324+0000 7fbf59800640 1 -- start start 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.325+0000 7fbf59800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf540d2460 0x7fbf54145b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.325+0000 7fbf59800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2840 0x7fbf54146080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.325+0000 7fbf59800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf540d4e40 0x7fbf540b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.325+0000 7fbf59800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fbf540d4650 con 0x7fbf540d2840 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.325+0000 7fbf59800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf540d44d0 con 0x7fbf540d2460 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.325+0000 7fbf59800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf540d0a50 con 0x7fbf540d4e40 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2840 0x7fbf54146080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2840 0x7fbf54146080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53610/0 (socket says 172.21.15.47:53610) 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 -- 172.21.15.47:0/1509643220 learned_addr learned my addr 172.21.15.47:0/1509643220 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53e00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf540d2460 0x7fbf54145b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.327 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf58e00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf540d4e40 0x7fbf540b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 -- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf540d4e40 msgr2=0x7fbf540b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf540d4e40 0x7fbf540b64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 -- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf540d2460 msgr2=0x7fbf54145b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf540d2460 0x7fbf54145b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 -- 172.21.15.47:0/1509643220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf5406cfd0 con 0x7fbf540d2840 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.326+0000 7fbf53400640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2840 0x7fbf54146080 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c007e40 tx=0x7fbf3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.327+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c014620 con 0x7fbf540d2840 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.327+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf3c019070 con 0x7fbf540d2840 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.327+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c014a00 con 0x7fbf540d2840 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.327+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf5406d2b0 con 0x7fbf540d2840 2024-04-20T10:31:17.328 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.327+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf5406d870 con 0x7fbf540d2840 2024-04-20T10:31:17.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.328+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fbf54075b30 con 0x7fbf540d2840 2024-04-20T10:31:17.329 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.329+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fbf3c014ba0 con 0x7fbf540d2840 2024-04-20T10:31:17.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.330+0000 7fbf4be00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbf3006c8f0 0x7fbf3006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.330 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.330+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fbf3c08c340 con 0x7fbf540d2840 2024-04-20T10:31:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.331+0000 7fbf4be00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7fbf30072ed0 0x7fbf30075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.331+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 --> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbf3c00ce50 con 0x7fbf30072ed0 2024-04-20T10:31:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.331+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fbf3c093050 con 0x7fbf540d2840 2024-04-20T10:31:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.331+0000 7fbf53e00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbf3006c8f0 0x7fbf3006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.331+0000 7fbf58e00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7fbf30072ed0 0x7fbf30075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.331 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.332+0000 7fbf58e00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7fbf30072ed0 0x7fbf30075310 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:17.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.332+0000 7fbf53e00640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbf3006c8f0 0x7fbf3006edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036000 tx=0x7fbf4403b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:17.332 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.332+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== osd.1 v2:172.21.15.47:6803/2415890922 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fbf3c00ce50 con 0x7fbf30072ed0 2024-04-20T10:31:17.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.347+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 --> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fbf54161260 con 0x7fbf30072ed0 2024-04-20T10:31:17.347 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.348+0000 7fbf4be00640 1 -- 172.21.15.47:0/1509643220 <== osd.1 v2:172.21.15.47:6803/2415890922 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fbf54161260 con 0x7fbf30072ed0 2024-04-20T10:31:17.348 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:17.348 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:17.348 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.348+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7fbf30072ed0 msgr2=0x7fbf30075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.348+0000 7fbf59800640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:6803/2415890922,v1:172.21.15.47:6805/2415890922] conn(0x7fbf30072ed0 0x7fbf30075310 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.348+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbf3006c8f0 msgr2=0x7fbf3006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.348+0000 7fbf59800640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fbf3006c8f0 0x7fbf3006edb0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fbf44036000 tx=0x7fbf4403b040 comp rx=0 tx=0).stop 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.348+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2840 msgr2=0x7fbf54146080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.349+0000 7fbf59800640 1 --2- 172.21.15.47:0/1509643220 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fbf540d2840 0x7fbf54146080 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c007e40 tx=0x7fbf3c00e4b0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.349+0000 7fbf58e00640 1 -- 172.21.15.47:0/1509643220 reap_dead start 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.349+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 shutdown_connections 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.349+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 >> 172.21.15.47:0/1509643220 conn(0x7fbf540b7570 msgr2=0x7fbf540cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.349+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 shutdown_connections 2024-04-20T10:31:17.349 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.349+0000 7fbf59800640 1 -- 172.21.15.47:0/1509643220 wait complete. 2024-04-20T10:31:17.367 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-20T10:31:17.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.580+0000 7f1334000640 1 Processor -- start 2024-04-20T10:31:17.580 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.580+0000 7f1334000640 1 -- start start 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1334000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c0d0a70 0x7f132c0ad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1334000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f132c06f090 0x7f132c0ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1334000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f132c06df70 con 0x7f132c162c00 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1334000640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f132c162fe0 con 0x7f132c06f090 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1334000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f132c06ec00 con 0x7f132c0d0a70 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1332000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f132c06f090 0x7f132c0ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1332000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f132c06f090 0x7f132c0ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53626/0 (socket says 172.21.15.47:53626) 2024-04-20T10:31:17.582 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1332000640 1 -- 172.21.15.47:0/1956612706 learned_addr learned my addr 172.21.15.47:0/1956612706 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1332000640 1 -- 172.21.15.47:0/1956612706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c0d0a70 msgr2=0x7f132c0ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1332000640 1 --2- 172.21.15.47:0/1956612706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c0d0a70 0x7f132c0ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1332000640 1 -- 172.21.15.47:0/1956612706 >> v1:172.21.15.175:6789/0 conn(0x7f132c162c00 legacy=0x7f132c0d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.582+0000 7f1332000640 1 -- 172.21.15.47:0/1956612706 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c06e310 con 0x7f132c06f090 2024-04-20T10:31:17.583 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.584+0000 7f1332000640 1 --2- 172.21.15.47:0/1956612706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c06f090 0x7f132c0ad590 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1324009e50 tx=0x7f132402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1aa26eaad7106 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.584+0000 7f132be00640 1 -- 172.21.15.47:0/1956612706 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132402c9f0 con 0x7f132c06f090 2024-04-20T10:31:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.584+0000 7f132be00640 1 -- 172.21.15.47:0/1956612706 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f132402cb90 con 0x7f132c06f090 2024-04-20T10:31:17.584 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.584+0000 7f132be00640 1 -- 172.21.15.47:0/1956612706 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13240343f0 con 0x7f132c06f090 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 -- 172.21.15.47:0/1956612706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c06f090 msgr2=0x7f132c0ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 --2- 172.21.15.47:0/1956612706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c06f090 0x7f132c0ad590 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f1324009e50 tx=0x7f132402be50 comp rx=0 tx=0).stop 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 -- 172.21.15.47:0/1956612706 shutdown_connections 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 --2- 172.21.15.47:0/1956612706 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c06f090 0x7f132c0ad590 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 --2- 172.21.15.47:0/1956612706 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c0d0a70 0x7f132c0ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 -- 172.21.15.47:0/1956612706 >> 172.21.15.47:0/1956612706 conn(0x7f132c0b3350 msgr2=0x7f132c0b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 -- 172.21.15.47:0/1956612706 shutdown_connections 2024-04-20T10:31:17.585 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.585+0000 7f1334000640 1 -- 172.21.15.47:0/1956612706 wait complete. 2024-04-20T10:31:17.586 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.586+0000 7f1334000640 1 Processor -- start 2024-04-20T10:31:17.587 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.587+0000 7f1334000640 1 -- start start 2024-04-20T10:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.587+0000 7f1334000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f132c06f090 0x7f132c1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.587+0000 7f1334000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c0d0a70 0x7f132c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1334000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c162c00 0x7f132c159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1334000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f132c06dbb0 con 0x7f132c0d0a70 2024-04-20T10:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1334000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f132c06e890 con 0x7f132c06f090 2024-04-20T10:31:17.588 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1334000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f132c06dd30 con 0x7f132c162c00 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c0d0a70 0x7f132c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c0d0a70 0x7f132c15c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53630/0 (socket says 172.21.15.47:53630) 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 -- 172.21.15.47:0/3141483277 learned_addr learned my addr 172.21.15.47:0/3141483277 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1332000640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c162c00 0x7f132c159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1331600640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f132c06f090 0x7f132c1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 -- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c162c00 msgr2=0x7f132c159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c162c00 0x7f132c159fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 -- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f132c06f090 msgr2=0x7f132c1557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f132c06f090 0x7f132c1557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1330c00640 1 -- 172.21.15.47:0/3141483277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c156980 con 0x7f132c0d0a70 2024-04-20T10:31:17.589 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.588+0000 7f1331600640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f132c06f090 0x7f132c1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.589+0000 7f1330c00640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c0d0a70 0x7f132c15c190 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f131800b4a0 tx=0x7f131800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.589+0000 7f1329600640 1 -- 172.21.15.47:0/3141483277 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318014070 con 0x7f132c0d0a70 2024-04-20T10:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.589+0000 7f1329600640 1 -- 172.21.15.47:0/3141483277 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1318010610 con 0x7f132c0d0a70 2024-04-20T10:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.589+0000 7f1329600640 1 -- 172.21.15.47:0/3141483277 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318010a00 con 0x7f132c0d0a70 2024-04-20T10:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.589+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c156c60 con 0x7f132c0d0a70 2024-04-20T10:31:17.590 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.589+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c157220 con 0x7f132c0d0a70 2024-04-20T10:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.590+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c068980 con 0x7f132c0d0a70 2024-04-20T10:31:17.591 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.591+0000 7f1329600640 1 -- 172.21.15.47:0/3141483277 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1318017070 con 0x7f132c0d0a70 2024-04-20T10:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.592+0000 7f1329600640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f130c06c8f0 0x7f130c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:17.592 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.593+0000 7f1329600640 1 -- 172.21.15.47:0/3141483277 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f131808b510 con 0x7f132c0d0a70 2024-04-20T10:31:17.593 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.593+0000 7f1331600640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f130c06c8f0 0x7f130c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:17.594 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.594+0000 7f1331600640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f130c06c8f0 0x7f130c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f132000a810 tx=0x7f1320008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:17.597 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.597+0000 7f1329600640 1 -- 172.21.15.47:0/3141483277 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f1318059a00 con 0x7f132c0d0a70 2024-04-20T10:31:17.806 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.806+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f132c15e7c0 con 0x7f130c06c8f0 2024-04-20T10:31:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.807+0000 7f1329600640 1 -- 172.21.15.47:0/3141483277 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f132c15e7c0 con 0x7f130c06c8f0 2024-04-20T10:31:17.807 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 1 to deep-scrub 2024-04-20T10:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f130c06c8f0 msgr2=0x7f130c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.810 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f130c06c8f0 0x7f130c06edb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f132000a810 tx=0x7f1320008040 comp rx=0 tx=0).stop 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c0d0a70 msgr2=0x7f132c15c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c0d0a70 0x7f132c15c190 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f131800b4a0 tx=0x7f131800b960 comp rx=0 tx=0).stop 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 shutdown_connections 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132c162c00 0x7f132c159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f130c06c8f0 0x7f130c06edb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f132c0d0a70 0x7f132c15c190 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 --2- 172.21.15.47:0/3141483277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f132c06f090 0x7f132c1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 >> 172.21.15.47:0/3141483277 conn(0x7f132c0b3350 msgr2=0x7f132c0aee90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 shutdown_connections 2024-04-20T10:31:17.811 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:17.810+0000 7f1334000640 1 -- 172.21.15.47:0/3141483277 wait complete. 2024-04-20T10:31:17.830 INFO:tasks.ceph:Scrubbing osd.2 2024-04-20T10:31:17.830 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:18.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.043+0000 7f9001200640 1 Processor -- start 2024-04-20T10:31:18.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.044+0000 7f9001200640 1 -- start start 2024-04-20T10:31:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f9001200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ffc0d2460 0x7f8ffc0d4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f9001200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d4e40 0x7f8ffc1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f9001200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8ffc0d2e30 con 0x7f8ffc0d2840 2024-04-20T10:31:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f9001200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8ffc0d3110 con 0x7f8ffc0d2460 2024-04-20T10:31:18.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f9001200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ffc0d33f0 con 0x7f8ffc0d4e40 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f8ffb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ffc0d2460 0x7f8ffc0d4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f8ffbe00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8ffc0d2840 0x7f8ffc0b60a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34050/0 (socket says 172.21.15.47:34050) 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f8ffb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8ffc0d2460 0x7f8ffc0d4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53636/0 (socket says 172.21.15.47:53636) 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.045+0000 7f8ffbe00640 1 -- 172.21.15.47:0/2048642251 learned_addr learned my addr 172.21.15.47:0/2048642251 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.046+0000 7f8ffaa00640 1 --2- 172.21.15.47:0/2048642251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d4e40 0x7f8ffc1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.046+0000 7f8ffb400640 1 -- 172.21.15.47:0/2048642251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d4e40 msgr2=0x7f8ffc1617d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.046+0000 7f8ffb400640 1 --2- 172.21.15.47:0/2048642251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d4e40 0x7f8ffc1617d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.046+0000 7f8ffb400640 1 -- 172.21.15.47:0/2048642251 >> v1:172.21.15.47:6789/0 conn(0x7f8ffc0d2840 legacy=0x7f8ffc0b60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.046+0000 7f8ffb400640 1 -- 172.21.15.47:0/2048642251 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc0d36d0 con 0x7f8ffc0d2460 2024-04-20T10:31:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.047+0000 7f8ffb400640 1 --2- 172.21.15.47:0/2048642251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc0d4900 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0097a0 tx=0x7f8fec02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab62ac1888990592 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.047+0000 7f8ffa000640 1 -- 172.21.15.47:0/2048642251 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec035020 con 0x7f8ffc0d2460 2024-04-20T10:31:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.047+0000 7f8ffa000640 1 -- 172.21.15.47:0/2048642251 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec031810 con 0x7f8ffc0d2460 2024-04-20T10:31:18.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.047+0000 7f8ffa000640 1 -- 172.21.15.47:0/2048642251 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec031be0 con 0x7f8ffc0d2460 2024-04-20T10:31:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 -- 172.21.15.47:0/2048642251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 msgr2=0x7f8ffc0d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 --2- 172.21.15.47:0/2048642251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc0d4900 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0097a0 tx=0x7f8fec02b8b0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 -- 172.21.15.47:0/2048642251 shutdown_connections 2024-04-20T10:31:18.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 --2- 172.21.15.47:0/2048642251 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d4e40 0x7f8ffc1617d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 --2- 172.21.15.47:0/2048642251 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc0d4900 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 -- 172.21.15.47:0/2048642251 >> 172.21.15.47:0/2048642251 conn(0x7f8ffc0b7570 msgr2=0x7f8ffc0b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 -- 172.21.15.47:0/2048642251 shutdown_connections 2024-04-20T10:31:18.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.048+0000 7f9001200640 1 -- 172.21.15.47:0/2048642251 wait complete. 2024-04-20T10:31:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.050+0000 7f9001200640 1 Processor -- start 2024-04-20T10:31:18.050 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.050+0000 7f9001200640 1 -- start start 2024-04-20T10:31:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f9001200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f9001200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d2840 0x7f8ffc06de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f9001200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0d4e40 0x7f8ffc06e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f9001200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc0d4650 con 0x7f8ffc0d2460 2024-04-20T10:31:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f9001200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ffc0d44d0 con 0x7f8ffc0d4e40 2024-04-20T10:31:18.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f9001200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ffc0d0a50 con 0x7f8ffc0d2840 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f8ffb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f8ffb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53640/0 (socket says 172.21.15.47:53640) 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.051+0000 7f8ffb400640 1 -- 172.21.15.47:0/3999467471 learned_addr learned my addr 172.21.15.47:0/3999467471 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffaa00640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d2840 0x7f8ffc06de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffb400640 1 -- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d2840 msgr2=0x7f8ffc06de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffbe00640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0d4e40 0x7f8ffc06e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffb400640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ffc0d2840 0x7f8ffc06de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffb400640 1 -- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0d4e40 msgr2=0x7f8ffc06e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffb400640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ffc0d4e40 0x7f8ffc06e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffb400640 1 -- 172.21.15.47:0/3999467471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ffc06c860 con 0x7f8ffc0d2460 2024-04-20T10:31:18.053 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.052+0000 7f8ffb400640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc155a60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009890 tx=0x7f8fec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.053+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec041070 con 0x7f8ffc0d2460 2024-04-20T10:31:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.053+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8fec03ae50 con 0x7f8ffc0d2460 2024-04-20T10:31:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.053+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec035040 con 0x7f8ffc0d2460 2024-04-20T10:31:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.053+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ffc06baa0 con 0x7f8ffc0d2460 2024-04-20T10:31:18.054 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.053+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ffc06c060 con 0x7f8ffc0d2460 2024-04-20T10:31:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.054+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ffc075b30 con 0x7f8ffc0d2460 2024-04-20T10:31:18.055 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.055+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8fec04c060 con 0x7f8ffc0d2460 2024-04-20T10:31:18.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.056+0000 7f8ff3400640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.056+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f8fec0ad7c0 con 0x7f8ffc0d2460 2024-04-20T10:31:18.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.056+0000 7f8ff3400640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f8fdc072ed0 0x7f8fdc075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.056+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 --> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8fec00af30 con 0x7f8fdc072ed0 2024-04-20T10:31:18.056 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.056+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f8fec0b4050 con 0x7f8ffc0d2460 2024-04-20T10:31:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.057+0000 7f8ffbe00640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f8fdc072ed0 0x7f8fdc075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.057+0000 7f8ffaa00640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.057 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.057+0000 7f8ffbe00640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f8fdc072ed0 0x7f8fdc075310 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.058+0000 7f8ffaa00640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc156fd0 tx=0x7f8fe800b040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.058 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.058+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== osd.2 v2:172.21.15.47:6822/2832687130 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8fec00af30 con 0x7f8fdc072ed0 2024-04-20T10:31:18.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.073+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 --> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8ffc0ce9b0 con 0x7f8fdc072ed0 2024-04-20T10:31:18.074 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.074+0000 7f8ff3400640 1 -- 172.21.15.47:0/3999467471 <== osd.2 v2:172.21.15.47:6822/2832687130 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8ffc0ce9b0 con 0x7f8fdc072ed0 2024-04-20T10:31:18.074 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:18.074 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:18.075 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:18.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.074+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f8fdc072ed0 msgr2=0x7f8fdc075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.074+0000 7f9001200640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:6822/2832687130,v1:172.21.15.47:6824/2832687130] conn(0x7f8fdc072ed0 0x7f8fdc075310 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8fdc06c8f0 msgr2=0x7f8fdc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8fdc06c8f0 0x7f8fdc06edb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc156fd0 tx=0x7f8fe800b040 comp rx=0 tx=0).stop 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 msgr2=0x7f8ffc155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 --2- 172.21.15.47:0/3999467471 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8ffc0d2460 0x7f8ffc155a60 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009890 tx=0x7f8fec0046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f8ffbe00640 1 -- 172.21.15.47:0/3999467471 reap_dead start 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 shutdown_connections 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 >> 172.21.15.47:0/3999467471 conn(0x7f8ffc0b7570 msgr2=0x7f8ffc0cb780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 shutdown_connections 2024-04-20T10:31:18.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.075+0000 7f9001200640 1 -- 172.21.15.47:0/3999467471 wait complete. 2024-04-20T10:31:18.092 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-20T10:31:18.299 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.299+0000 7f1374a00640 1 Processor -- start 2024-04-20T10:31:18.300 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.300+0000 7f1374a00640 1 -- start start 2024-04-20T10:31:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.301+0000 7f1374a00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f137015a1f0 0x7f1370072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.302 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.301+0000 7f1374a00640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f1370072190 0x7f1370073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.301+0000 7f1374a00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f137015cc90 con 0x7f1370159e10 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.301+0000 7f1374a00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f1370072900 con 0x7f137015a1f0 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.301+0000 7f1374a00640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f137015ca40 con 0x7f1370072190 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f1367e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f137015a1f0 0x7f1370072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f1367e00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f137015a1f0 0x7f1370072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53648/0 (socket says 172.21.15.47:53648) 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f1367e00640 1 -- 172.21.15.47:0/1147984651 learned_addr learned my addr 172.21.15.47:0/1147984651 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f136f200640 1 --2- 172.21.15.47:0/1147984651 >> v2:172.21.15.175:3300/0 conn(0x7f1370072190 0x7f1370073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f1367e00640 1 -- 172.21.15.47:0/1147984651 >> v2:172.21.15.175:3300/0 conn(0x7f1370072190 msgr2=0x7f1370073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f1367e00640 1 --2- 172.21.15.47:0/1147984651 >> v2:172.21.15.175:3300/0 conn(0x7f1370072190 0x7f1370073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f1367e00640 1 -- 172.21.15.47:0/1147984651 >> v1:172.21.15.175:6789/0 conn(0x7f1370159e10 legacy=0x7f1370071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:18.303 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.302+0000 7f1367e00640 1 -- 172.21.15.47:0/1147984651 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f137015cf70 con 0x7f137015a1f0 2024-04-20T10:31:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.303+0000 7f1367e00640 1 --2- 172.21.15.47:0/1147984651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f137015a1f0 0x7f1370072ee0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f137015daf0 tx=0x7f135802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e2393b96b062fc server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.304+0000 7f136de00640 1 -- 172.21.15.47:0/1147984651 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358035020 con 0x7f137015a1f0 2024-04-20T10:31:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.304+0000 7f136de00640 1 -- 172.21.15.47:0/1147984651 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1358031870 con 0x7f137015a1f0 2024-04-20T10:31:18.304 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.304+0000 7f136de00640 1 -- 172.21.15.47:0/1147984651 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358031c40 con 0x7f137015a1f0 2024-04-20T10:31:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 -- 172.21.15.47:0/1147984651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f137015a1f0 msgr2=0x7f1370072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 --2- 172.21.15.47:0/1147984651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f137015a1f0 0x7f1370072ee0 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f137015daf0 tx=0x7f135802b910 comp rx=0 tx=0).stop 2024-04-20T10:31:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 -- 172.21.15.47:0/1147984651 shutdown_connections 2024-04-20T10:31:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 --2- 172.21.15.47:0/1147984651 >> v2:172.21.15.175:3300/0 conn(0x7f1370072190 0x7f1370073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 --2- 172.21.15.47:0/1147984651 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f137015a1f0 0x7f1370072ee0 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.305 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 -- 172.21.15.47:0/1147984651 >> 172.21.15.47:0/1147984651 conn(0x7f13700d0040 msgr2=0x7f13700d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 -- 172.21.15.47:0/1147984651 shutdown_connections 2024-04-20T10:31:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.305+0000 7f1374a00640 1 -- 172.21.15.47:0/1147984651 wait complete. 2024-04-20T10:31:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.307+0000 7f1374a00640 1 Processor -- start 2024-04-20T10:31:18.306 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.307+0000 7f1374a00640 1 -- start start 2024-04-20T10:31:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1374a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1370072190 0x7f13700b3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.308 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1374a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1370159e10 0x7f137009abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1374a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015a1f0 0x7f137009cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1374a00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f137015d810 con 0x7f1370159e10 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1374a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f137015d530 con 0x7f137015a1f0 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1374a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f137006da90 con 0x7f1370072190 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1367e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1370159e10 0x7f137009abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1367e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1370159e10 0x7f137009abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53654/0 (socket says 172.21.15.47:53654) 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f1367e00640 1 -- 172.21.15.47:0/1106669768 learned_addr learned my addr 172.21.15.47:0/1106669768 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f136f200640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015a1f0 0x7f137009cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.308+0000 7f136e800640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1370072190 0x7f13700b3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1367e00640 1 -- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1370072190 msgr2=0x7f13700b3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1367e00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1370072190 0x7f13700b3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1367e00640 1 -- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015a1f0 msgr2=0x7f137009cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.309 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1367e00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015a1f0 0x7f137009cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1367e00640 1 -- 172.21.15.47:0/1106669768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13700b3970 con 0x7f1370159e10 2024-04-20T10:31:18.310 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f136e800640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1370072190 0x7f13700b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1367e00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1370159e10 0x7f137009abd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f135802b950 tx=0x7f13580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1366a00640 1 -- 172.21.15.47:0/1106669768 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358040070 con 0x7f1370159e10 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1366a00640 1 -- 172.21.15.47:0/1106669768 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1358030d00 con 0x7f1370159e10 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13700b3c50 con 0x7f1370159e10 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1366a00640 1 -- 172.21.15.47:0/1106669768 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358035040 con 0x7f1370159e10 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.309+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13700b4210 con 0x7f1370159e10 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.310+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1370075b60 con 0x7f1370159e10 2024-04-20T10:31:18.311 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.312+0000 7f1366a00640 1 -- 172.21.15.47:0/1106669768 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f1358038020 con 0x7f1370159e10 2024-04-20T10:31:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.313+0000 7f1366a00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f134c06c8f0 0x7f134c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.313+0000 7f1366a00640 1 -- 172.21.15.47:0/1106669768 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f13580ad9f0 con 0x7f1370159e10 2024-04-20T10:31:18.313 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.313+0000 7f136e800640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f134c06c8f0 0x7f134c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.314 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.315+0000 7f136e800640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f134c06c8f0 0x7f134c06edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1360003eb0 tx=0x7f1360009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.317 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.318+0000 7f1366a00640 1 -- 172.21.15.47:0/1106669768 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f135807bdb0 con 0x7f1370159e10 2024-04-20T10:31:18.527 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.527+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f13700b16e0 con 0x7f134c06c8f0 2024-04-20T10:31:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.528+0000 7f1366a00640 1 -- 172.21.15.47:0/1106669768 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f13700b16e0 con 0x7f134c06c8f0 2024-04-20T10:31:18.528 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 2 to deep-scrub 2024-04-20T10:31:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f134c06c8f0 msgr2=0x7f134c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f134c06c8f0 0x7f134c06edb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1360003eb0 tx=0x7f1360009040 comp rx=0 tx=0).stop 2024-04-20T10:31:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1370159e10 msgr2=0x7f137009abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1370159e10 0x7f137009abd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f135802b950 tx=0x7f13580046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.531 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 shutdown_connections 2024-04-20T10:31:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f137015a1f0 0x7f137009cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f134c06c8f0 0x7f134c06edb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f1370159e10 0x7f137009abd0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 --2- 172.21.15.47:0/1106669768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1370072190 0x7f13700b3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 >> 172.21.15.47:0/1106669768 conn(0x7f13700d0040 msgr2=0x7f1370071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 shutdown_connections 2024-04-20T10:31:18.532 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.531+0000 7f1374a00640 1 -- 172.21.15.47:0/1106669768 wait complete. 2024-04-20T10:31:18.551 INFO:tasks.ceph:Scrubbing osd.3 2024-04-20T10:31:18.551 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:18.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.764+0000 7fcdc1600640 1 Processor -- start 2024-04-20T10:31:18.764 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.764+0000 7fcdc1600640 1 -- start start 2024-04-20T10:31:18.765 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.765+0000 7fcdc1600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdbc15c340 0x7fcdbc0cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.765+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15ff70 0x7fcdbc0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.765+0000 7fcdc1600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fcdbc0cf0e0 0x7fcdbc0cfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.765+0000 7fcdc1600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fcdbc0cf550 con 0x7fcdbc15c340 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.765+0000 7fcdc1600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fcdbc160530 con 0x7fcdbc0cf0e0 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.765+0000 7fcdc1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc160810 con 0x7fcdbc15ff70 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdbc15c340 0x7fcdbc0cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fcdbc15c340 0x7fcdbc0cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53656/0 (socket says 172.21.15.47:53656) 2024-04-20T10:31:18.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 -- 172.21.15.47:0/423141850 learned_addr learned my addr 172.21.15.47:0/423141850 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdc0c00640 1 --2- 172.21.15.47:0/423141850 >> v2:172.21.15.175:3300/0 conn(0x7fcdbc0cf0e0 0x7fcdbc0cfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbb400640 1 --2- 172.21.15.47:0/423141850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15ff70 0x7fcdbc0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 -- 172.21.15.47:0/423141850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15ff70 msgr2=0x7fcdbc0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/423141850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15ff70 0x7fcdbc0ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 -- 172.21.15.47:0/423141850 >> v2:172.21.15.175:3300/0 conn(0x7fcdbc0cf0e0 msgr2=0x7fcdbc0cfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/423141850 >> v2:172.21.15.175:3300/0 conn(0x7fcdbc0cf0e0 0x7fcdbc0cfd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdbbe00640 1 -- 172.21.15.47:0/423141850 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc160af0 con 0x7fcdbc15c340 2024-04-20T10:31:18.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.766+0000 7fcdc0c00640 1 --2- 172.21.15.47:0/423141850 >> v2:172.21.15.175:3300/0 conn(0x7fcdbc0cf0e0 0x7fcdbc0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-20T10:31:18.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.768+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/423141850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15c340 0x7fcdbc0cc9e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0726f0 tx=0x7fcdac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e800c01772874d68 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.768+0000 7fcdbaa00640 1 -- 172.21.15.47:0/423141850 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac02cb00 con 0x7fcdbc15c340 2024-04-20T10:31:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdbaa00640 1 -- 172.21.15.47:0/423141850 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdac02cca0 con 0x7fcdbc15c340 2024-04-20T10:31:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdbaa00640 1 -- 172.21.15.47:0/423141850 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdac034610 con 0x7fcdbc15c340 2024-04-20T10:31:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 -- 172.21.15.47:0/423141850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15c340 msgr2=0x7fcdbc0cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 --2- 172.21.15.47:0/423141850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15c340 0x7fcdbc0cc9e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcdbc0726f0 tx=0x7fcdac02bf60 comp rx=0 tx=0).stop 2024-04-20T10:31:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 -- 172.21.15.47:0/423141850 shutdown_connections 2024-04-20T10:31:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 --2- 172.21.15.47:0/423141850 >> v2:172.21.15.175:3300/0 conn(0x7fcdbc0cf0e0 0x7fcdbc0cfd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 --2- 172.21.15.47:0/423141850 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15ff70 0x7fcdbc0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 --2- 172.21.15.47:0/423141850 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15c340 0x7fcdbc0cc9e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 -- 172.21.15.47:0/423141850 >> 172.21.15.47:0/423141850 conn(0x7fcdbc072140 msgr2=0x7fcdbc0b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 -- 172.21.15.47:0/423141850 shutdown_connections 2024-04-20T10:31:18.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.769+0000 7fcdc1600640 1 -- 172.21.15.47:0/423141850 wait complete. 2024-04-20T10:31:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.771+0000 7fcdc1600640 1 Processor -- start 2024-04-20T10:31:18.771 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.771+0000 7fcdc1600640 1 -- start start 2024-04-20T10:31:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.772+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc0cf0e0 0x7fcdbc159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.772+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15c340 0x7fcdbc070cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.772+0000 7fcdc1600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15ff70 0x7fcdbc0c7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.772+0000 7fcdc1600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc15c720 con 0x7fcdbc15ff70 2024-04-20T10:31:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.772+0000 7fcdc1600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcdbc160dd0 con 0x7fcdbc0cf0e0 2024-04-20T10:31:18.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.772+0000 7fcdc1600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcdbc15c8a0 con 0x7fcdbc15c340 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15ff70 0x7fcdbc0c7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15ff70 0x7fcdbc0c7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53664/0 (socket says 172.21.15.47:53664) 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 -- 172.21.15.47:0/3571414257 learned_addr learned my addr 172.21.15.47:0/3571414257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc0cf0e0 0x7fcdbc159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdbb400640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15c340 0x7fcdbc070cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 -- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15c340 msgr2=0x7fcdbc070cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdbc15c340 0x7fcdbc070cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 -- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc0cf0e0 msgr2=0x7fcdbc159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdbc0cf0e0 0x7fcdbc159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.773+0000 7fcdc0c00640 1 -- 172.21.15.47:0/3571414257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdbc06fca0 con 0x7fcdbc15ff70 2024-04-20T10:31:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.774+0000 7fcdc0c00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15ff70 0x7fcdbc0c7ec0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e7e0 tx=0x7fcdb400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.774+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400f040 con 0x7fcdbc15ff70 2024-04-20T10:31:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.774+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcdb4004030 con 0x7fcdbc15ff70 2024-04-20T10:31:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.774+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40043e0 con 0x7fcdbc15ff70 2024-04-20T10:31:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.774+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdbc06ff10 con 0x7fcdbc15ff70 2024-04-20T10:31:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.774+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdbc0704d0 con 0x7fcdbc15ff70 2024-04-20T10:31:18.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.775+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcdbc0708b0 con 0x7fcdbc15ff70 2024-04-20T10:31:18.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.776+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fcdb400c6d0 con 0x7fcdbc15ff70 2024-04-20T10:31:18.777 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.777+0000 7fcdb3e00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.777+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fcdb408ba20 con 0x7fcdbc15ff70 2024-04-20T10:31:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.777+0000 7fcdb3e00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7fcd9c072ed0 0x7fcd9c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.778+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 --> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd9c0759e0 con 0x7fcd9c072ed0 2024-04-20T10:31:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.778+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fcdb40902a0 con 0x7fcdbc15ff70 2024-04-20T10:31:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.778+0000 7fcdbb400640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7fcd9c072ed0 0x7fcd9c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.778+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:18.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.778+0000 7fcdbb400640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7fcd9c072ed0 0x7fcd9c075310 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.779+0000 7fcdbbe00640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcdac035000 tx=0x7fcdac030000 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:18.779 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.779+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== osd.3 v2:172.21.15.47:6812/4183939241 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd9c0759e0 con 0x7fcd9c072ed0 2024-04-20T10:31:18.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.794+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 --> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fcdbc075b30 con 0x7fcd9c072ed0 2024-04-20T10:31:18.794 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.794+0000 7fcdb3e00640 1 -- 172.21.15.47:0/3571414257 <== osd.3 v2:172.21.15.47:6812/4183939241 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fcdbc075b30 con 0x7fcd9c072ed0 2024-04-20T10:31:18.794 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:18.794 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:18.795 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:18.795 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7fcd9c072ed0 msgr2=0x7fcd9c075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:6812/4183939241,v1:172.21.15.47:6815/4183939241] conn(0x7fcd9c072ed0 0x7fcd9c075310 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd9c06c8f0 msgr2=0x7fcd9c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fcd9c06c8f0 0x7fcd9c06edb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcdac035000 tx=0x7fcdac030000 comp rx=0 tx=0).stop 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15ff70 msgr2=0x7fcdbc0c7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 --2- 172.21.15.47:0/3571414257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fcdbc15ff70 0x7fcdbc0c7ec0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400e7e0 tx=0x7fcdb400eca0 comp rx=0 tx=0).stop 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc0c00640 1 -- 172.21.15.47:0/3571414257 reap_dead start 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 shutdown_connections 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 >> 172.21.15.47:0/3571414257 conn(0x7fcdbc072140 msgr2=0x7fcdbc0b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.795+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 shutdown_connections 2024-04-20T10:31:18.796 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:18.796+0000 7fcdc1600640 1 -- 172.21.15.47:0/3571414257 wait complete. 2024-04-20T10:31:18.815 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-20T10:31:19.034 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.034+0000 7fe307400640 1 Processor -- start 2024-04-20T10:31:19.035 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.035+0000 7fe307400640 1 -- start start 2024-04-20T10:31:19.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.036+0000 7fe307400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3000b4960 0x7fe30006d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.036+0000 7fe307400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fe300162800 0x7fe300164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.036+0000 7fe307400640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe3000b4dc0 con 0x7fe30006e400 2024-04-20T10:31:19.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.036+0000 7fe307400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fe30006c010 con 0x7fe300162800 2024-04-20T10:31:19.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.036+0000 7fe307400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3000cf5c0 con 0x7fe3000b4960 2024-04-20T10:31:19.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.036+0000 7fe304a00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe30006e400 0x7fe30006b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34060/0 (socket says 172.21.15.47:34060) 2024-04-20T10:31:19.037 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe304a00640 1 -- 172.21.15.47:0/1173966100 learned_addr learned my addr 172.21.15.47:0/1173966100 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe305400640 1 --2- 172.21.15.47:0/1173966100 >> v2:172.21.15.47:3300/0 conn(0x7fe300162800 0x7fe300164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe2f7e00640 1 --2- 172.21.15.47:0/1173966100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3000b4960 0x7fe30006d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe2ffe00640 1 -- 172.21.15.47:0/1173966100 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2851914138 0 0) 0x7fe3000b4dc0 con 0x7fe30006e400 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe2ffe00640 1 -- 172.21.15.47:0/1173966100 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe2e4005730 con 0x7fe30006e400 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe305400640 1 -- 172.21.15.47:0/1173966100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3000b4960 msgr2=0x7fe30006d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe305400640 1 --2- 172.21.15.47:0/1173966100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3000b4960 0x7fe30006d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe305400640 1 -- 172.21.15.47:0/1173966100 >> v1:172.21.15.47:6789/0 conn(0x7fe30006e400 legacy=0x7fe30006b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.038 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.037+0000 7fe305400640 1 -- 172.21.15.47:0/1173966100 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30006d9f0 con 0x7fe300162800 2024-04-20T10:31:19.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.038+0000 7fe305400640 1 --2- 172.21.15.47:0/1173966100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe300162800 0x7fe300164bf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009d90 tx=0x7fe2f802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e763e2997fd7cc5b server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.038+0000 7fe2ffe00640 1 -- 172.21.15.47:0/1173966100 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8039070 con 0x7fe300162800 2024-04-20T10:31:19.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe2ffe00640 1 -- 172.21.15.47:0/1173966100 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f802cc50 con 0x7fe300162800 2024-04-20T10:31:19.039 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe2ffe00640 1 -- 172.21.15.47:0/1173966100 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8034440 con 0x7fe300162800 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 -- 172.21.15.47:0/1173966100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe300162800 msgr2=0x7fe300164bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 --2- 172.21.15.47:0/1173966100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe300162800 0x7fe300164bf0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009d90 tx=0x7fe2f802bea0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 -- 172.21.15.47:0/1173966100 shutdown_connections 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 --2- 172.21.15.47:0/1173966100 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe300162800 0x7fe300164bf0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 --2- 172.21.15.47:0/1173966100 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3000b4960 0x7fe30006d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 -- 172.21.15.47:0/1173966100 >> 172.21.15.47:0/1173966100 conn(0x7fe3000cd530 msgr2=0x7fe3000cd940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 -- 172.21.15.47:0/1173966100 shutdown_connections 2024-04-20T10:31:19.040 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.039+0000 7fe307400640 1 -- 172.21.15.47:0/1173966100 wait complete. 2024-04-20T10:31:19.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.041+0000 7fe307400640 1 Processor -- start 2024-04-20T10:31:19.041 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.041+0000 7fe307400640 1 -- start start 2024-04-20T10:31:19.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.042+0000 7fe307400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30006e400 0x7fe30013eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.042+0000 7fe307400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3000b4960 0x7fe300135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.042+0000 7fe307400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe300162800 0x7fe300136060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.043 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.042+0000 7fe307400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe3000cf250 con 0x7fe30006e400 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.042+0000 7fe307400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3000cef70 con 0x7fe3000b4960 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.042+0000 7fe307400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe30006c190 con 0x7fe300162800 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.042+0000 7fe304a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30006e400 0x7fe30013eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30006e400 0x7fe30013eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53682/0 (socket says 172.21.15.47:53682) 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 -- 172.21.15.47:0/1509497753 learned_addr learned my addr 172.21.15.47:0/1509497753 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe305400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe300162800 0x7fe300136060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 -- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe300162800 msgr2=0x7fe300136060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe2f7e00640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3000b4960 0x7fe300135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe300162800 0x7fe300136060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 -- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3000b4960 msgr2=0x7fe300135b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3000b4960 0x7fe300135b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 -- 172.21.15.47:0/1509497753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30013f0b0 con 0x7fe30006e400 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe305400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe300162800 0x7fe300136060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:19.044 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.043+0000 7fe304a00640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30006e400 0x7fe30013eb70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000c430 tx=0x7fe2f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.044+0000 7fe2fd600640 1 -- 172.21.15.47:0/1509497753 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f0017070 con 0x7fe30006e400 2024-04-20T10:31:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.044+0000 7fe2fd600640 1 -- 172.21.15.47:0/1509497753 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe2f0005ce0 con 0x7fe30006e400 2024-04-20T10:31:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.044+0000 7fe2fd600640 1 -- 172.21.15.47:0/1509497753 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00072f0 con 0x7fe30006e400 2024-04-20T10:31:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.044+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe300137970 con 0x7fe30006e400 2024-04-20T10:31:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.044+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe300137f30 con 0x7fe30006e400 2024-04-20T10:31:19.045 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.045+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe300068980 con 0x7fe30006e400 2024-04-20T10:31:19.046 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.046+0000 7fe2fd600640 1 -- 172.21.15.47:0/1509497753 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe2f001a070 con 0x7fe30006e400 2024-04-20T10:31:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.047+0000 7fe2fd600640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2dc06c8f0 0x7fe2dc06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.047 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.047+0000 7fe2fd600640 1 -- 172.21.15.47:0/1509497753 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fe2f008af50 con 0x7fe30006e400 2024-04-20T10:31:19.048 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.048+0000 7fe2f7e00640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2dc06c8f0 0x7fe2dc06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.049 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.049+0000 7fe2f7e00640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2dc06c8f0 0x7fe2dc06edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe30013d690 tx=0x7fe2e8008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.052 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.052+0000 7fe2fd600640 1 -- 172.21.15.47:0/1509497753 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe2f0059440 con 0x7fe30006e400 2024-04-20T10:31:19.258 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.258+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fe30015abe0 con 0x7fe2dc06c8f0 2024-04-20T10:31:19.259 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.259+0000 7fe2fd600640 1 -- 172.21.15.47:0/1509497753 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe30015abe0 con 0x7fe2dc06c8f0 2024-04-20T10:31:19.259 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 3 to deep-scrub 2024-04-20T10:31:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.261+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2dc06c8f0 msgr2=0x7fe2dc06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.261+0000 7fe307400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2dc06c8f0 0x7fe2dc06edb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe30013d690 tx=0x7fe2e8008040 comp rx=0 tx=0).stop 2024-04-20T10:31:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.261+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30006e400 msgr2=0x7fe30013eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.261+0000 7fe307400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30006e400 0x7fe30013eb70 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe2f000c430 tx=0x7fe2f000c8f0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.262 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 shutdown_connections 2024-04-20T10:31:19.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe300162800 0x7fe300136060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3000b4960 0x7fe300135b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe2dc06c8f0 0x7fe2dc06edb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 --2- 172.21.15.47:0/1509497753 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe30006e400 0x7fe30013eb70 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 >> 172.21.15.47:0/1509497753 conn(0x7fe3000cd530 msgr2=0x7fe30006b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:19.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 shutdown_connections 2024-04-20T10:31:19.265 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.262+0000 7fe307400640 1 -- 172.21.15.47:0/1509497753 wait complete. 2024-04-20T10:31:19.281 INFO:tasks.ceph:Scrubbing osd.4 2024-04-20T10:31:19.281 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.488+0000 7f0731c00640 1 Processor -- start 2024-04-20T10:31:19.488 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.488+0000 7f0731c00640 1 -- start start 2024-04-20T10:31:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f0731c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c06d020 0x7f072c06f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f0731c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f072c06d920 0x7f072c15c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f0731c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f072c06de90 con 0x7f072c15c400 2024-04-20T10:31:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f0731c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f072c06e170 con 0x7f072c06d920 2024-04-20T10:31:19.490 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f0731c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f072c071ee0 con 0x7f072c06d020 2024-04-20T10:31:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f072be00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f072c15c400 0x7f072c06c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34062/0 (socket says 172.21.15.47:34062) 2024-04-20T10:31:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f072be00640 1 -- 172.21.15.47:0/1490604619 learned_addr learned my addr 172.21.15.47:0/1490604619 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.490+0000 7f0731200640 1 --2- 172.21.15.47:0/1490604619 >> v2:172.21.15.47:3300/0 conn(0x7f072c06d920 0x7f072c15c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.491+0000 7f072b400640 1 --2- 172.21.15.47:0/1490604619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c06d020 0x7f072c06f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.491+0000 7f0731200640 1 -- 172.21.15.47:0/1490604619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c06d020 msgr2=0x7f072c06f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.491+0000 7f0731200640 1 --2- 172.21.15.47:0/1490604619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c06d020 0x7f072c06f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.491 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.491+0000 7f0731200640 1 -- 172.21.15.47:0/1490604619 >> v1:172.21.15.47:6789/0 conn(0x7f072c15c400 legacy=0x7f072c06c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.491+0000 7f0731200640 1 -- 172.21.15.47:0/1490604619 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c072130 con 0x7f072c06d920 2024-04-20T10:31:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.491+0000 7f072b400640 1 --2- 172.21.15.47:0/1490604619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c06d020 0x7f072c06f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:19.492 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.492+0000 7f0731200640 1 --2- 172.21.15.47:0/1490604619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c15c7e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f07240097a0 tx=0x7f072402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba01aeada0e25449 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.492+0000 7f072aa00640 1 -- 172.21.15.47:0/1490604619 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724035020 con 0x7f072c06d920 2024-04-20T10:31:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.492+0000 7f072aa00640 1 -- 172.21.15.47:0/1490604619 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0724031810 con 0x7f072c06d920 2024-04-20T10:31:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.492+0000 7f072aa00640 1 -- 172.21.15.47:0/1490604619 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0724031be0 con 0x7f072c06d920 2024-04-20T10:31:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 -- 172.21.15.47:0/1490604619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 msgr2=0x7f072c15c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 --2- 172.21.15.47:0/1490604619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c15c7e0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f07240097a0 tx=0x7f072402b8b0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.493 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 -- 172.21.15.47:0/1490604619 shutdown_connections 2024-04-20T10:31:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 --2- 172.21.15.47:0/1490604619 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c15c7e0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 --2- 172.21.15.47:0/1490604619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c06d020 0x7f072c06f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 -- 172.21.15.47:0/1490604619 >> 172.21.15.47:0/1490604619 conn(0x7f072c0cc9b0 msgr2=0x7f072c0ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 -- 172.21.15.47:0/1490604619 shutdown_connections 2024-04-20T10:31:19.494 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.493+0000 7f0731c00640 1 -- 172.21.15.47:0/1490604619 wait complete. 2024-04-20T10:31:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.495+0000 7f0731c00640 1 Processor -- start 2024-04-20T10:31:19.495 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.495+0000 7f0731c00640 1 -- start start 2024-04-20T10:31:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.496+0000 7f0731c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f072c06d020 0x7f072c0b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.496+0000 7f0731c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c0b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.496+0000 7f0731c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c15c400 0x7f072c0b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.496+0000 7f0731c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f072c071bd0 con 0x7f072c06d920 2024-04-20T10:31:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.496+0000 7f0731c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f072c072780 con 0x7f072c06d020 2024-04-20T10:31:19.497 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.496+0000 7f0731c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f072c072c10 con 0x7f072c15c400 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c0b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c0b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53696/0 (socket says 172.21.15.47:53696) 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 -- 172.21.15.47:0/4066666397 learned_addr learned my addr 172.21.15.47:0/4066666397 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072be00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f072c06d020 0x7f072c0b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f0731200640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c15c400 0x7f072c0b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 -- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c15c400 msgr2=0x7f072c0b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f072c15c400 0x7f072c0b5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 -- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f072c06d020 msgr2=0x7f072c0b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f072c06d020 0x7f072c0b2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.498 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.497+0000 7f072b400640 1 -- 172.21.15.47:0/4066666397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072c0b5720 con 0x7f072c06d920 2024-04-20T10:31:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.498+0000 7f072b400640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c0b4b40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f071800cf60 tx=0x7f071800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.498+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071801a070 con 0x7f072c06d920 2024-04-20T10:31:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.498+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f071800d840 con 0x7f072c06d920 2024-04-20T10:31:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.498+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f071800dc10 con 0x7f072c06d920 2024-04-20T10:31:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.498+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072c15eb50 con 0x7f072c06d920 2024-04-20T10:31:19.499 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.498+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f072c15b850 con 0x7f072c06d920 2024-04-20T10:31:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.499+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f072c0b5a00 con 0x7f072c06d920 2024-04-20T10:31:19.500 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.500+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f071801d070 con 0x7f072c06d920 2024-04-20T10:31:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.501+0000 7f0723e00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f070c06c8f0 0x7f070c06edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.501 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.501+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f071808c3b0 con 0x7f072c06d920 2024-04-20T10:31:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.501+0000 7f0723e00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f070c072ed0 0x7f070c075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.501+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 --> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0718012e50 con 0x7f070c072ed0 2024-04-20T10:31:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.501+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f0718093050 con 0x7f072c06d920 2024-04-20T10:31:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.502+0000 7f072be00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f070c06c8f0 0x7f070c06edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.502+0000 7f0731200640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f070c072ed0 0x7f070c075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.503+0000 7f072be00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f070c06c8f0 0x7f070c06edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f071c000f60 tx=0x7f071c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.503 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.504+0000 7f0731200640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f070c072ed0 0x7f070c075310 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.506+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== osd.4 v2:172.21.15.175:6816/1401363349 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0718012e50 con 0x7f070c072ed0 2024-04-20T10:31:19.521 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.521+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 --> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f072c0742b0 con 0x7f070c072ed0 2024-04-20T10:31:19.522 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.522+0000 7f0723e00640 1 -- 172.21.15.47:0/4066666397 <== osd.4 v2:172.21.15.175:6816/1401363349 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f072c0742b0 con 0x7f070c072ed0 2024-04-20T10:31:19.522 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:19.522 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:19.522 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.522+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f070c072ed0 msgr2=0x7f070c075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.522+0000 7f0731c00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6816/1401363349,v1:172.21.15.175:6817/1401363349] conn(0x7f070c072ed0 0x7f070c075310 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f070c06c8f0 msgr2=0x7f070c06edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f070c06c8f0 0x7f070c06edb0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f071c000f60 tx=0x7f071c009040 comp rx=0 tx=0).stop 2024-04-20T10:31:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 msgr2=0x7f072c0b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 --2- 172.21.15.47:0/4066666397 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f072c06d920 0x7f072c0b4b40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f071800cf60 tx=0x7f071800cf90 comp rx=0 tx=0).stop 2024-04-20T10:31:19.523 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731200640 1 -- 172.21.15.47:0/4066666397 reap_dead start 2024-04-20T10:31:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 shutdown_connections 2024-04-20T10:31:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 >> 172.21.15.47:0/4066666397 conn(0x7f072c0cc9b0 msgr2=0x7f072c0b0b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 shutdown_connections 2024-04-20T10:31:19.524 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.523+0000 7f0731c00640 1 -- 172.21.15.47:0/4066666397 wait complete. 2024-04-20T10:31:19.544 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-20T10:31:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.763+0000 7f13e6200640 1 Processor -- start 2024-04-20T10:31:19.763 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.763+0000 7f13e6200640 1 -- start start 2024-04-20T10:31:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.765+0000 7f13e6200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13e015a1f0 0x7f13e0072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.765+0000 7f13e6200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f13e0072190 0x7f13e0073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.765+0000 7f13e6200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13e015cc90 con 0x7f13e0159e10 2024-04-20T10:31:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.765+0000 7f13e6200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f13e0072900 con 0x7f13e015a1f0 2024-04-20T10:31:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.765+0000 7f13e6200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f13e015ca40 con 0x7f13e0072190 2024-04-20T10:31:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13dfe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13e015a1f0 0x7f13e0072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.766 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13dfe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f13e015a1f0 0x7f13e0072ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53710/0 (socket says 172.21.15.47:53710) 2024-04-20T10:31:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13dfe00640 1 -- 172.21.15.47:0/3570603482 learned_addr learned my addr 172.21.15.47:0/3570603482 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13e5800640 1 --2- 172.21.15.47:0/3570603482 >> v2:172.21.15.175:3300/0 conn(0x7f13e0072190 0x7f13e0073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13dfe00640 1 -- 172.21.15.47:0/3570603482 >> v2:172.21.15.175:3300/0 conn(0x7f13e0072190 msgr2=0x7f13e0073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13dfe00640 1 --2- 172.21.15.47:0/3570603482 >> v2:172.21.15.175:3300/0 conn(0x7f13e0072190 0x7f13e0073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13dfe00640 1 -- 172.21.15.47:0/3570603482 >> v1:172.21.15.175:6789/0 conn(0x7f13e0159e10 legacy=0x7f13e0071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13dfe00640 1 -- 172.21.15.47:0/3570603482 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e015cf70 con 0x7f13e015a1f0 2024-04-20T10:31:19.767 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.766+0000 7f13e5800640 1 --2- 172.21.15.47:0/3570603482 >> v2:172.21.15.175:3300/0 conn(0x7f13e0072190 0x7f13e0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.767+0000 7f13dfe00640 1 --2- 172.21.15.47:0/3570603482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0072ee0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f13e015daf0 tx=0x7f13cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d92b10b7d0046c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13df400640 1 -- 172.21.15.47:0/3570603482 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc035020 con 0x7f13e015a1f0 2024-04-20T10:31:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13df400640 1 -- 172.21.15.47:0/3570603482 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13cc031870 con 0x7f13e015a1f0 2024-04-20T10:31:19.768 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13df400640 1 -- 172.21.15.47:0/3570603482 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc031c40 con 0x7f13e015a1f0 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13e6200640 1 -- 172.21.15.47:0/3570603482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 msgr2=0x7f13e0072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13e6200640 1 --2- 172.21.15.47:0/3570603482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0072ee0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f13e015daf0 tx=0x7f13cc02b910 comp rx=0 tx=0).stop 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13e6200640 1 -- 172.21.15.47:0/3570603482 shutdown_connections 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13e6200640 1 --2- 172.21.15.47:0/3570603482 >> v2:172.21.15.175:3300/0 conn(0x7f13e0072190 0x7f13e0073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13e6200640 1 --2- 172.21.15.47:0/3570603482 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0072ee0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13e6200640 1 -- 172.21.15.47:0/3570603482 >> 172.21.15.47:0/3570603482 conn(0x7f13e00d0040 msgr2=0x7f13e00d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.768+0000 7f13e6200640 1 -- 172.21.15.47:0/3570603482 shutdown_connections 2024-04-20T10:31:19.769 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.769+0000 7f13e6200640 1 -- 172.21.15.47:0/3570603482 wait complete. 2024-04-20T10:31:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.770+0000 7f13e6200640 1 Processor -- start 2024-04-20T10:31:19.770 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.770+0000 7f13e6200640 1 -- start start 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.771+0000 7f13e6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e0072190 0x7f13e00b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.771+0000 7f13e6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13e0159e10 0x7f13e0158450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.771+0000 7f13e6200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0158990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.771+0000 7f13e6200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f13e015d810 con 0x7f13e015a1f0 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.771+0000 7f13e6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f13e015d530 con 0x7f13e0072190 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.771+0000 7f13e6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f13e006da90 con 0x7f13e0159e10 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0158990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53722/0 (socket says 172.21.15.47:53722) 2024-04-20T10:31:19.772 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 -- 172.21.15.47:0/339474794 learned_addr learned my addr 172.21.15.47:0/339474794 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e4e00640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e0072190 0x7f13e00b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13dfe00640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13e0159e10 0x7f13e0158450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 -- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13e0159e10 msgr2=0x7f13e0158450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13e0159e10 0x7f13e0158450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 -- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e0072190 msgr2=0x7f13e00b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e0072190 0x7f13e00b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e5800640 1 -- 172.21.15.47:0/339474794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e00c9540 con 0x7f13e015a1f0 2024-04-20T10:31:19.773 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.772+0000 7f13e4e00640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e0072190 0x7f13e00b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.773+0000 7f13e5800640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0158990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f13d800ce80 tx=0x7f13d800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.773+0000 7f13dcc00640 1 -- 172.21.15.47:0/339474794 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d801a070 con 0x7f13e015a1f0 2024-04-20T10:31:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.773+0000 7f13dcc00640 1 -- 172.21.15.47:0/339474794 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13d800d880 con 0x7f13e015a1f0 2024-04-20T10:31:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.773+0000 7f13dcc00640 1 -- 172.21.15.47:0/339474794 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d800dc70 con 0x7f13e015a1f0 2024-04-20T10:31:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.773+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e00c9820 con 0x7f13e015a1f0 2024-04-20T10:31:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.773+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e00c9b00 con 0x7f13e015a1f0 2024-04-20T10:31:19.774 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.774+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e0075b60 con 0x7f13e015a1f0 2024-04-20T10:31:19.775 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.775+0000 7f13dcc00640 1 -- 172.21.15.47:0/339474794 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f13d801d070 con 0x7f13e015a1f0 2024-04-20T10:31:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.776+0000 7f13dcc00640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f13c006c940 0x7f13c006ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.776+0000 7f13dcc00640 1 -- 172.21.15.47:0/339474794 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f13d808c1a0 con 0x7f13e015a1f0 2024-04-20T10:31:19.776 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.777+0000 7f13e4e00640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f13c006c940 0x7f13c006ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:19.778 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.778+0000 7f13e4e00640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f13c006c940 0x7f13c006ee00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f13d4007fd0 tx=0x7f13d4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:19.781 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.781+0000 7f13dcc00640 1 -- 172.21.15.47:0/339474794 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f13d805a0b0 con 0x7f13e015a1f0 2024-04-20T10:31:19.987 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.987+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f13e00c92f0 con 0x7f13c006c940 2024-04-20T10:31:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.988+0000 7f13dcc00640 1 -- 172.21.15.47:0/339474794 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f13e00c92f0 con 0x7f13c006c940 2024-04-20T10:31:19.988 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 4 to deep-scrub 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f13c006c940 msgr2=0x7f13c006ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f13c006c940 0x7f13c006ee00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f13d4007fd0 tx=0x7f13d4009040 comp rx=0 tx=0).stop 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 msgr2=0x7f13e0158990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0158990 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f13d800ce80 tx=0x7f13d800a710 comp rx=0 tx=0).stop 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 shutdown_connections 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f13c006c940 0x7f13c006ee00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f13e015a1f0 0x7f13e0158990 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13e0159e10 0x7f13e0158450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 --2- 172.21.15.47:0/339474794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13e0072190 0x7f13e00b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.990+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 >> 172.21.15.47:0/339474794 conn(0x7f13e00d0040 msgr2=0x7f13e0071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:19.991 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.991+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 shutdown_connections 2024-04-20T10:31:19.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:19.991+0000 7f13e6200640 1 -- 172.21.15.47:0/339474794 wait complete. 2024-04-20T10:31:20.009 INFO:tasks.ceph:Scrubbing osd.5 2024-04-20T10:31:20.010 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:20.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.234+0000 7f6d6e600640 1 Processor -- start 2024-04-20T10:31:20.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.234+0000 7f6d6e600640 1 -- start start 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.235+0000 7f6d6e600640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d680cf210 0x7f6d680b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.235+0000 7f6d6e600640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f6d6815fe90 0x7f6d680cfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.235+0000 7f6d6e600640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d680b2af0 con 0x7f6d6815ecf0 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.235+0000 7f6d6e600640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f6d680b36d0 con 0x7f6d680cf210 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.235+0000 7f6d6e600640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f6d680b2250 con 0x7f6d6815fe90 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d6d200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d680cf210 0x7f6d680b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d6d200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f6d680cf210 0x7f6d680b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53732/0 (socket says 172.21.15.47:53732) 2024-04-20T10:31:20.236 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d6d200640 1 -- 172.21.15.47:0/794476257 learned_addr learned my addr 172.21.15.47:0/794476257 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:20.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d67e00640 1 --2- 172.21.15.47:0/794476257 >> v2:172.21.15.175:3300/0 conn(0x7f6d6815fe90 0x7f6d680cfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d6d200640 1 -- 172.21.15.47:0/794476257 >> v2:172.21.15.175:3300/0 conn(0x7f6d6815fe90 msgr2=0x7f6d680cfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:20.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d6d200640 1 --2- 172.21.15.47:0/794476257 >> v2:172.21.15.175:3300/0 conn(0x7f6d6815fe90 0x7f6d680cfd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d6d200640 1 -- 172.21.15.47:0/794476257 >> v1:172.21.15.175:6789/0 conn(0x7f6d6815ecf0 legacy=0x7f6d681612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:20.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.236+0000 7f6d6d200640 1 -- 172.21.15.47:0/794476257 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d680b2530 con 0x7f6d680cf210 2024-04-20T10:31:20.237 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.238+0000 7f6d6d200640 1 --2- 172.21.15.47:0/794476257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d680cf210 0x7f6d680b1b50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6d680b4c60 tx=0x7f6d5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30abc561f7a8e98d server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.238+0000 7f6d67400640 1 -- 172.21.15.47:0/794476257 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58035020 con 0x7f6d680cf210 2024-04-20T10:31:20.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.238+0000 7f6d67400640 1 -- 172.21.15.47:0/794476257 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d58031870 con 0x7f6d680cf210 2024-04-20T10:31:20.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.238+0000 7f6d67400640 1 -- 172.21.15.47:0/794476257 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58031c40 con 0x7f6d680cf210 2024-04-20T10:31:20.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 -- 172.21.15.47:0/794476257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d680cf210 msgr2=0x7f6d680b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 --2- 172.21.15.47:0/794476257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d680cf210 0x7f6d680b1b50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f6d680b4c60 tx=0x7f6d5802b910 comp rx=0 tx=0).stop 2024-04-20T10:31:20.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 -- 172.21.15.47:0/794476257 shutdown_connections 2024-04-20T10:31:20.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 --2- 172.21.15.47:0/794476257 >> v2:172.21.15.175:3300/0 conn(0x7f6d6815fe90 0x7f6d680cfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 --2- 172.21.15.47:0/794476257 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d680cf210 0x7f6d680b1b50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 -- 172.21.15.47:0/794476257 >> 172.21.15.47:0/794476257 conn(0x7f6d680b3ce0 msgr2=0x7f6d680b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:20.239 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 -- 172.21.15.47:0/794476257 shutdown_connections 2024-04-20T10:31:20.240 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.239+0000 7f6d6e600640 1 -- 172.21.15.47:0/794476257 wait complete. 2024-04-20T10:31:20.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.241+0000 7f6d6e600640 1 Processor -- start 2024-04-20T10:31:20.241 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.241+0000 7f6d6e600640 1 -- start start 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.242+0000 7f6d6e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d680cf210 0x7f6d680b7370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.242+0000 7f6d6e600640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6815ecf0 0x7f6d680c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.242+0000 7f6d6e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6815fe90 0x7f6d680c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.242+0000 7f6d6e600640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f6d680d02c0 con 0x7f6d6815ecf0 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.242+0000 7f6d6e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6806d340 con 0x7f6d680cf210 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.242+0000 7f6d6e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d680d0440 con 0x7f6d6815fe90 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.242+0000 7f6d67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6815ecf0 0x7f6d680c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.243 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6815ecf0 0x7f6d680c93d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53738/0 (socket says 172.21.15.47:53738) 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 -- 172.21.15.47:0/3464228481 learned_addr learned my addr 172.21.15.47:0/3464228481 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d6d200640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d680cf210 0x7f6d680b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d6dc00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6815fe90 0x7f6d680c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 -- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6815fe90 msgr2=0x7f6d680c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d6815fe90 0x7f6d680c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 -- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d680cf210 msgr2=0x7f6d680b7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d680cf210 0x7f6d680b7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 -- 172.21.15.47:0/3464228481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d680b78b0 con 0x7f6d6815ecf0 2024-04-20T10:31:20.244 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.243+0000 7f6d67e00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6815ecf0 0x7f6d680c93d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400c7c0 tx=0x7f6d5400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.244+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54015070 con 0x7f6d6815ecf0 2024-04-20T10:31:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.244+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d54007820 con 0x7f6d6815ecf0 2024-04-20T10:31:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.244+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54007bd0 con 0x7f6d6815ecf0 2024-04-20T10:31:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.244+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d680b62d0 con 0x7f6d6815ecf0 2024-04-20T10:31:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.244+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d680b6890 con 0x7f6d6815ecf0 2024-04-20T10:31:20.245 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.245+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d68075b30 con 0x7f6d6815ecf0 2024-04-20T10:31:20.246 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.246+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f6d54018070 con 0x7f6d6815ecf0 2024-04-20T10:31:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.247+0000 7f6d64c00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d4806c920 0x7f6d4806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.247+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f6d5408b610 con 0x7f6d6815ecf0 2024-04-20T10:31:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.247+0000 7f6d64c00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7f6d48072f00 0x7f6d48075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.247+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 --> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d48075a10 con 0x7f6d48072f00 2024-04-20T10:31:20.247 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.247+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f6d54092050 con 0x7f6d6815ecf0 2024-04-20T10:31:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.248+0000 7f6d6d200640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d4806c920 0x7f6d4806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.248 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.248+0000 7f6d6dc00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7f6d48072f00 0x7f6d48075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.249+0000 7f6d6d200640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d4806c920 0x7f6d4806ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6d58036000 tx=0x7f6d5803c040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.249 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.250+0000 7f6d6dc00640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7f6d48072f00 0x7f6d48075340 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.251 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.252+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== osd.5 v2:172.21.15.175:6800/1477827144 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d48075a10 con 0x7f6d48072f00 2024-04-20T10:31:20.267 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.267+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 --> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6d680b6ff0 con 0x7f6d48072f00 2024-04-20T10:31:20.268 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.268+0000 7f6d64c00640 1 -- 172.21.15.47:0/3464228481 <== osd.5 v2:172.21.15.175:6800/1477827144 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6d680b6ff0 con 0x7f6d48072f00 2024-04-20T10:31:20.268 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:20.268 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:20.268 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7f6d48072f00 msgr2=0x7f6d48075340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6800/1477827144,v1:172.21.15.175:6801/1477827144] conn(0x7f6d48072f00 0x7f6d48075340 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.269 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d4806c920 msgr2=0x7f6d4806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f6d4806c920 0x7f6d4806ede0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f6d58036000 tx=0x7f6d5803c040 comp rx=0 tx=0).stop 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6815ecf0 msgr2=0x7f6d680c93d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 --2- 172.21.15.47:0/3464228481 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f6d6815ecf0 0x7f6d680c93d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f6d5400c7c0 tx=0x7f6d5400cc80 comp rx=0 tx=0).stop 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6dc00640 1 -- 172.21.15.47:0/3464228481 reap_dead start 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 shutdown_connections 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 >> 172.21.15.47:0/3464228481 conn(0x7f6d680b3ce0 msgr2=0x7f6d680cd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 shutdown_connections 2024-04-20T10:31:20.270 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.269+0000 7f6d6e600640 1 -- 172.21.15.47:0/3464228481 wait complete. 2024-04-20T10:31:20.288 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-20T10:31:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.502+0000 7f8e99c00640 1 Processor -- start 2024-04-20T10:31:20.502 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.502+0000 7f8e99c00640 1 -- start start 2024-04-20T10:31:20.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e99c00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e9415a1f0 0x7f8e94072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.504 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e99c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94073420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e99c00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8e9415cc90 con 0x7f8e94159e10 2024-04-20T10:31:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e99c00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f8e94072900 con 0x7f8e9415a1f0 2024-04-20T10:31:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e99c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9415ca40 con 0x7f8e94072190 2024-04-20T10:31:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e93400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f8e9415a1f0 0x7f8e94072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e93e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f8e94159e10 0x7f8e94071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34068/0 (socket says 172.21.15.47:34068) 2024-04-20T10:31:20.505 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.504+0000 7f8e93e00640 1 -- 172.21.15.47:0/4291529814 learned_addr learned my addr 172.21.15.47:0/4291529814 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.505+0000 7f8e99200640 1 --2- 172.21.15.47:0/4291529814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.505+0000 7f8e93400640 1 -- 172.21.15.47:0/4291529814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 msgr2=0x7f8e94073420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.505+0000 7f8e93400640 1 --2- 172.21.15.47:0/4291529814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94073420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.505+0000 7f8e93400640 1 -- 172.21.15.47:0/4291529814 >> v1:172.21.15.47:6789/0 conn(0x7f8e94159e10 legacy=0x7f8e94071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.505+0000 7f8e93400640 1 -- 172.21.15.47:0/4291529814 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9415cf70 con 0x7f8e9415a1f0 2024-04-20T10:31:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.505+0000 7f8e92a00640 1 -- 172.21.15.47:0/4291529814 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 697097552 0 0) 0x7f8e9415cc90 con 0x7f8e94159e10 2024-04-20T10:31:20.506 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.505+0000 7f8e99200640 1 --2- 172.21.15.47:0/4291529814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:20.507 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.506+0000 7f8e93400640 1 --2- 172.21.15.47:0/4291529814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e9415a1f0 0x7f8e94072ee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e1f6814a76f403 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e92a00640 1 -- 172.21.15.47:0/4291529814 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80035020 con 0x7f8e9415a1f0 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e92a00640 1 -- 172.21.15.47:0/4291529814 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80031830 con 0x7f8e9415a1f0 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e92a00640 1 -- 172.21.15.47:0/4291529814 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80031c00 con 0x7f8e9415a1f0 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 -- 172.21.15.47:0/4291529814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e9415a1f0 msgr2=0x7f8e94072ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 --2- 172.21.15.47:0/4291529814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e9415a1f0 0x7f8e94072ee0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f8e800097a0 tx=0x7f8e8002b8d0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 -- 172.21.15.47:0/4291529814 shutdown_connections 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 --2- 172.21.15.47:0/4291529814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94073420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 --2- 172.21.15.47:0/4291529814 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e9415a1f0 0x7f8e94072ee0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 -- 172.21.15.47:0/4291529814 >> 172.21.15.47:0/4291529814 conn(0x7f8e940d0040 msgr2=0x7f8e940d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:20.508 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 -- 172.21.15.47:0/4291529814 shutdown_connections 2024-04-20T10:31:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.507+0000 7f8e99c00640 1 -- 172.21.15.47:0/4291529814 wait complete. 2024-04-20T10:31:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.509+0000 7f8e99c00640 1 Processor -- start 2024-04-20T10:31:20.509 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.509+0000 7f8e99c00640 1 -- start start 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e99c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e99c00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159e10 0x7f8e940b3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e99c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e9415a1f0 0x7f8e940b43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e99c00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9415d810 con 0x7f8e94159e10 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e99c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9415d530 con 0x7f8e9415a1f0 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e99c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8e9406da90 con 0x7f8e94072190 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159e10 0x7f8e940b3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.512 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e93400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159e10 0x7f8e940b3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53756/0 (socket says 172.21.15.47:53756) 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.510+0000 7f8e93400640 1 -- 172.21.15.47:0/3232827200 learned_addr learned my addr 172.21.15.47:0/3232827200 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93e00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e99200640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e9415a1f0 0x7f8e940b43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93400640 1 -- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 msgr2=0x7f8e94152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93400640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93400640 1 -- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e9415a1f0 msgr2=0x7f8e940b43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93400640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e9415a1f0 0x7f8e940b43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93400640 1 -- 172.21.15.47:0/3232827200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e940ca540 con 0x7f8e94159e10 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93e00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e99200640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e9415a1f0 0x7f8e940b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.511+0000 7f8e93400640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159e10 0x7f8e940b3eb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8e80034040 tx=0x7f8e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.513 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.512+0000 7f8e8be00640 1 -- 172.21.15.47:0/3232827200 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80040070 con 0x7f8e94159e10 2024-04-20T10:31:20.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.512+0000 7f8e8be00640 1 -- 172.21.15.47:0/3232827200 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80039a20 con 0x7f8e94159e10 2024-04-20T10:31:20.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.512+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e940ca820 con 0x7f8e94159e10 2024-04-20T10:31:20.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.512+0000 7f8e8be00640 1 -- 172.21.15.47:0/3232827200 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80035040 con 0x7f8e94159e10 2024-04-20T10:31:20.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.512+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e940cade0 con 0x7f8e94159e10 2024-04-20T10:31:20.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.513+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e94075b60 con 0x7f8e94159e10 2024-04-20T10:31:20.514 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.514+0000 7f8e8be00640 1 -- 172.21.15.47:0/3232827200 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f8e80038020 con 0x7f8e94159e10 2024-04-20T10:31:20.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.515+0000 7f8e8be00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8e7406c8f0 0x7f8e7406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.515+0000 7f8e8be00640 1 -- 172.21.15.47:0/3232827200 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f8e800ad530 con 0x7f8e94159e10 2024-04-20T10:31:20.515 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.516+0000 7f8e93e00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8e7406c8f0 0x7f8e7406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.516 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.517+0000 7f8e93e00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8e7406c8f0 0x7f8e7406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8e84000f60 tx=0x7f8e84009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.519 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.519+0000 7f8e8be00640 1 -- 172.21.15.47:0/3232827200 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f8e8007b980 con 0x7f8e94159e10 2024-04-20T10:31:20.723 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.723+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f8e940cb5e0 con 0x7f8e7406c8f0 2024-04-20T10:31:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.725+0000 7f8e8be00640 1 -- 172.21.15.47:0/3232827200 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8e840013d0 con 0x7f8e7406c8f0 2024-04-20T10:31:20.725 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 5 to deep-scrub 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.727+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8e7406c8f0 msgr2=0x7f8e7406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.727+0000 7f8e99c00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8e7406c8f0 0x7f8e7406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8e84000f60 tx=0x7f8e84009040 comp rx=0 tx=0).stop 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.727+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159e10 msgr2=0x7f8e940b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.727+0000 7f8e99c00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159e10 0x7f8e940b3eb0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f8e80034040 tx=0x7f8e800046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 shutdown_connections 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8e9415a1f0 0x7f8e940b43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f8e7406c8f0 0x7f8e7406edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f8e94159e10 0x7f8e940b3eb0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 --2- 172.21.15.47:0/3232827200 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8e94072190 0x7f8e94152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.728 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 >> 172.21.15.47:0/3232827200 conn(0x7f8e940d0040 msgr2=0x7f8e940b06b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 shutdown_connections 2024-04-20T10:31:20.729 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.728+0000 7f8e99c00640 1 -- 172.21.15.47:0/3232827200 wait complete. 2024-04-20T10:31:20.745 INFO:tasks.ceph:Scrubbing osd.6 2024-04-20T10:31:20.745 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:20.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.960+0000 7f542e200640 1 Processor -- start 2024-04-20T10:31:20.960 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.960+0000 7f542e200640 1 -- start start 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542e200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f542815b0c0 0x7f54280727d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 0x7f54280cc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542e200640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f542806c910 0x7f542806cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542e200640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f542806d2c0 con 0x7f542806c910 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542e200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f5428072e90 con 0x7f542815b0c0 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5428073170 con 0x7f5428071ba0 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f542806c910 0x7f542806cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542d800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f542806c910 0x7f542806cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53772/0 (socket says 172.21.15.47:53772) 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.962+0000 7f542d800640 1 -- 172.21.15.47:0/1183700080 learned_addr learned my addr 172.21.15.47:0/1183700080 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.963+0000 7f5427e00640 1 --2- 172.21.15.47:0/1183700080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 0x7f54280cc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.963+0000 7f542ce00640 1 --2- 172.21.15.47:0/1183700080 >> v2:172.21.15.175:3300/0 conn(0x7f542815b0c0 0x7f54280727d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.963 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.963+0000 7f542d800640 1 -- 172.21.15.47:0/1183700080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 msgr2=0x7f54280cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.963+0000 7f542d800640 1 --2- 172.21.15.47:0/1183700080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 0x7f54280cc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.963+0000 7f542d800640 1 -- 172.21.15.47:0/1183700080 >> v2:172.21.15.175:3300/0 conn(0x7f542815b0c0 msgr2=0x7f54280727d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.963+0000 7f542d800640 1 --2- 172.21.15.47:0/1183700080 >> v2:172.21.15.175:3300/0 conn(0x7f542815b0c0 0x7f54280727d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.963+0000 7f542d800640 1 -- 172.21.15.47:0/1183700080 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5428073450 con 0x7f542806c910 2024-04-20T10:31:20.964 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.964+0000 7f542d800640 1 --2- 172.21.15.47:0/1183700080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542806cd80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f54280cdab0 tx=0x7f542002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c54ce90d63bbdf server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.964+0000 7f5427400640 1 -- 172.21.15.47:0/1183700080 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f542002ca70 con 0x7f542806c910 2024-04-20T10:31:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.964+0000 7f5427400640 1 -- 172.21.15.47:0/1183700080 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f542002cc10 con 0x7f542806c910 2024-04-20T10:31:20.965 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.964+0000 7f5427400640 1 -- 172.21.15.47:0/1183700080 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54200343f0 con 0x7f542806c910 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 -- 172.21.15.47:0/1183700080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 msgr2=0x7f542806cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 --2- 172.21.15.47:0/1183700080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542806cd80 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f54280cdab0 tx=0x7f542002bed0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 -- 172.21.15.47:0/1183700080 shutdown_connections 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 --2- 172.21.15.47:0/1183700080 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542806cd80 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 --2- 172.21.15.47:0/1183700080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 0x7f54280cc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 --2- 172.21.15.47:0/1183700080 >> v2:172.21.15.175:3300/0 conn(0x7f542815b0c0 0x7f54280727d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 -- 172.21.15.47:0/1183700080 >> 172.21.15.47:0/1183700080 conn(0x7f54280cc9b0 msgr2=0x7f54280ccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 -- 172.21.15.47:0/1183700080 shutdown_connections 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.965+0000 7f542e200640 1 -- 172.21.15.47:0/1183700080 wait complete. 2024-04-20T10:31:20.966 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.967+0000 7f542e200640 1 Processor -- start 2024-04-20T10:31:20.967 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.967+0000 7f542e200640 1 -- start start 2024-04-20T10:31:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542e200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542814b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 0x7f54280c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f542815b0c0 0x7f542809d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542e200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f5428072000 con 0x7f542806c910 2024-04-20T10:31:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54280cbb10 con 0x7f542815b0c0 2024-04-20T10:31:20.968 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54280cc570 con 0x7f5428071ba0 2024-04-20T10:31:20.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542814b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542ce00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542814b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53780/0 (socket says 172.21.15.47:53780) 2024-04-20T10:31:20.969 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.968+0000 7f542ce00640 1 -- 172.21.15.47:0/2616690198 learned_addr learned my addr 172.21.15.47:0/2616690198 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f5427e00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 0x7f54280c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f542ce00640 1 -- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 msgr2=0x7f54280c9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f542ce00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5428071ba0 0x7f54280c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f542ce00640 1 -- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f542815b0c0 msgr2=0x7f542809d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f542ce00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f542815b0c0 0x7f542809d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f542ce00640 1 -- 172.21.15.47:0/2616690198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542814b8c0 con 0x7f542806c910 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f542ce00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542814b380 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f541c00d5f0 tx=0x7f541c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c0125f0 con 0x7f542806c910 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f541c019070 con 0x7f542806c910 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c0129d0 con 0x7f542806c910 2024-04-20T10:31:20.970 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.969+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542809c350 con 0x7f542806c910 2024-04-20T10:31:20.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.970+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542809c910 con 0x7f542806c910 2024-04-20T10:31:20.971 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.970+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5428075b30 con 0x7f542806c910 2024-04-20T10:31:20.972 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.972+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f541c012b70 con 0x7f542806c910 2024-04-20T10:31:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.973+0000 7f5424c00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f540806c920 0x7f540806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.973+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f541c08c840 con 0x7f542806c910 2024-04-20T10:31:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.973+0000 7f5424c00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f5408072f00 0x7f5408075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.973+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 --> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f541c024e50 con 0x7f5408072f00 2024-04-20T10:31:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.973+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7f541c08ccd0 con 0x7f542806c910 2024-04-20T10:31:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.974+0000 7f5427e00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f540806c920 0x7f540806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.973 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.974+0000 7f542d800640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f5408072f00 0x7f5408075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:20.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.975+0000 7f5427e00640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f540806c920 0x7f540806ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f54280c6330 tx=0x7f5414008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.975 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.975+0000 7f542d800640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f5408072f00 0x7f5408075340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:20.977 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.977+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== osd.6 v2:172.21.15.175:6822/2500655665 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f541c024e50 con 0x7f5408072f00 2024-04-20T10:31:20.992 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.992+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 --> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f542809dee0 con 0x7f5408072f00 2024-04-20T10:31:20.993 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.993+0000 7f5424c00640 1 -- 172.21.15.47:0/2616690198 <== osd.6 v2:172.21.15.175:6822/2500655665 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f542809dee0 con 0x7f5408072f00 2024-04-20T10:31:20.993 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:20.993 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:20.993 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f5408072f00 msgr2=0x7f5408075340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6822/2500655665,v1:172.21.15.175:6824/2500655665] conn(0x7f5408072f00 0x7f5408075340 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f540806c920 msgr2=0x7f540806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f540806c920 0x7f540806ede0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f54280c6330 tx=0x7f5414008040 comp rx=0 tx=0).stop 2024-04-20T10:31:20.994 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 msgr2=0x7f542814b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 --2- 172.21.15.47:0/2616690198 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f542806c910 0x7f542814b380 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f541c00d5f0 tx=0x7f541c00dab0 comp rx=0 tx=0).stop 2024-04-20T10:31:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542d800640 1 -- 172.21.15.47:0/2616690198 reap_dead start 2024-04-20T10:31:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 shutdown_connections 2024-04-20T10:31:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 >> 172.21.15.47:0/2616690198 conn(0x7f54280cc9b0 msgr2=0x7f54280b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 shutdown_connections 2024-04-20T10:31:20.995 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:20.994+0000 7f542e200640 1 -- 172.21.15.47:0/2616690198 wait complete. 2024-04-20T10:31:21.012 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-20T10:31:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.222+0000 7f47bbe00640 1 Processor -- start 2024-04-20T10:31:21.222 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.222+0000 7f47bbe00640 1 -- start start 2024-04-20T10:31:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.223+0000 7f47bbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47bc072190 0x7f47bc0735f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.223 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.223+0000 7f47bbe00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc15cc90 con 0x7f47bc159e10 2024-04-20T10:31:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.223+0000 7f47bbe00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f47bc15cf70 con 0x7f47bc15a1f0 2024-04-20T10:31:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.223+0000 7f47bbe00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f47bc072900 con 0x7f47bc072190 2024-04-20T10:31:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.223+0000 7f47bb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47bc072190 0x7f47bc0735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.224+0000 7f47bb400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f47bc072190 0x7f47bc0735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53786/0 (socket says 172.21.15.47:53786) 2024-04-20T10:31:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.224+0000 7f47bb400640 1 -- 172.21.15.47:0/1214603132 learned_addr learned my addr 172.21.15.47:0/1214603132 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.224+0000 7f47baa00640 1 --1- 172.21.15.47:0/1214603132 >> v1:172.21.15.47:6789/0 conn(0x7f47bc159e10 0x7f47bc071a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34070/0 (socket says 172.21.15.47:34070) 2024-04-20T10:31:21.224 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.224+0000 7f47bb400640 1 -- 172.21.15.47:0/1214603132 >> v1:172.21.15.175:6789/0 conn(0x7f47bc15a1f0 legacy=0x7f47bc072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.224+0000 7f47bb400640 1 -- 172.21.15.47:0/1214603132 >> v1:172.21.15.47:6789/0 conn(0x7f47bc159e10 legacy=0x7f47bc071a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.224+0000 7f47bb400640 1 -- 172.21.15.47:0/1214603132 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc15d250 con 0x7f47bc072190 2024-04-20T10:31:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.225+0000 7f47bb400640 1 --2- 172.21.15.47:0/1214603132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc072190 0x7f47bc0735f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009e50 tx=0x7f47b402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fd4151dda24159 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.225+0000 7f47ba000640 1 -- 172.21.15.47:0/1214603132 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b402cb10 con 0x7f47bc072190 2024-04-20T10:31:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47ba000640 1 -- 172.21.15.47:0/1214603132 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b402ccb0 con 0x7f47bc072190 2024-04-20T10:31:21.225 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47ba000640 1 -- 172.21.15.47:0/1214603132 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b402ce60 con 0x7f47bc072190 2024-04-20T10:31:21.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47bbe00640 1 -- 172.21.15.47:0/1214603132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc072190 msgr2=0x7f47bc0735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47bbe00640 1 --2- 172.21.15.47:0/1214603132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc072190 0x7f47bc0735f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009e50 tx=0x7f47b402bf70 comp rx=0 tx=0).stop 2024-04-20T10:31:21.226 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47bbe00640 1 -- 172.21.15.47:0/1214603132 shutdown_connections 2024-04-20T10:31:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47bbe00640 1 --2- 172.21.15.47:0/1214603132 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc072190 0x7f47bc0735f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47bbe00640 1 -- 172.21.15.47:0/1214603132 >> 172.21.15.47:0/1214603132 conn(0x7f47bc0d0040 msgr2=0x7f47bc0d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47bbe00640 1 -- 172.21.15.47:0/1214603132 shutdown_connections 2024-04-20T10:31:21.227 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.226+0000 7f47bbe00640 1 -- 172.21.15.47:0/1214603132 wait complete. 2024-04-20T10:31:21.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.228+0000 7f47bbe00640 1 Processor -- start 2024-04-20T10:31:21.228 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.228+0000 7f47bbe00640 1 -- start start 2024-04-20T10:31:21.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47bc072190 0x7f47bc14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47bc159e10 0x7f47bc15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc15a1f0 0x7f47bc0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.229 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bbe00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc15d810 con 0x7f47bc15a1f0 2024-04-20T10:31:21.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47bc15ca40 con 0x7f47bc159e10 2024-04-20T10:31:21.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47bc073b30 con 0x7f47bc072190 2024-04-20T10:31:21.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc15a1f0 0x7f47bc0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bb400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc15a1f0 0x7f47bc0b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53790/0 (socket says 172.21.15.47:53790) 2024-04-20T10:31:21.230 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.229+0000 7f47bb400640 1 -- 172.21.15.47:0/984487363 learned_addr learned my addr 172.21.15.47:0/984487363 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47b2000640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47bc159e10 0x7f47bc15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47bb400640 1 -- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47bc072190 msgr2=0x7f47bc14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47bb400640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47bc072190 0x7f47bc14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47bb400640 1 -- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47bc159e10 msgr2=0x7f47bc15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47bb400640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47bc159e10 0x7f47bc15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47bb400640 1 -- 172.21.15.47:0/984487363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47bc0b3070 con 0x7f47bc15a1f0 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47b2000640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47bc159e10 0x7f47bc15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:21.231 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47bb400640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc15a1f0 0x7f47bc0b09e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009e50 tx=0x7f47b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.230+0000 7f47b3400640 1 -- 172.21.15.47:0/984487363 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b4043020 con 0x7f47bc15a1f0 2024-04-20T10:31:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.231+0000 7f47b3400640 1 -- 172.21.15.47:0/984487363 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f47b403e450 con 0x7f47bc15a1f0 2024-04-20T10:31:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.231+0000 7f47b3400640 1 -- 172.21.15.47:0/984487363 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b402ce60 con 0x7f47bc15a1f0 2024-04-20T10:31:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.231+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47bc0b3350 con 0x7f47bc15a1f0 2024-04-20T10:31:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.231+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47bc0b3910 con 0x7f47bc15a1f0 2024-04-20T10:31:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.232+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47bc075b60 con 0x7f47bc15a1f0 2024-04-20T10:31:21.232 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.233+0000 7f47b3400640 1 -- 172.21.15.47:0/984487363 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f47b4046020 con 0x7f47bc15a1f0 2024-04-20T10:31:21.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.234+0000 7f47b3400640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479806c920 0x7f479806ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.233 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.234+0000 7f47b3400640 1 -- 172.21.15.47:0/984487363 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f47b40adfc0 con 0x7f47bc15a1f0 2024-04-20T10:31:21.234 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.234+0000 7f47baa00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479806c920 0x7f479806ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.235 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.236+0000 7f47baa00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479806c920 0x7f479806ede0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000f60 tx=0x7f47ac007a00 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.238 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.238+0000 7f47b3400640 1 -- 172.21.15.47:0/984487363 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f47b404d070 con 0x7f47bc15a1f0 2024-04-20T10:31:21.441 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.441+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f47bc1561a0 con 0x7f479806c920 2024-04-20T10:31:21.442 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.442+0000 7f47b3400640 1 -- 172.21.15.47:0/984487363 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f47bc1561a0 con 0x7f479806c920 2024-04-20T10:31:21.442 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 6 to deep-scrub 2024-04-20T10:31:21.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.444+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479806c920 msgr2=0x7f479806ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.444 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.444+0000 7f47bbe00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479806c920 0x7f479806ede0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f47ac000f60 tx=0x7f47ac007a00 comp rx=0 tx=0).stop 2024-04-20T10:31:21.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.444+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc15a1f0 msgr2=0x7f47bc0b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.444+0000 7f47bbe00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc15a1f0 0x7f47bc0b09e0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009e50 tx=0x7f47b4004a40 comp rx=0 tx=0).stop 2024-04-20T10:31:21.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 shutdown_connections 2024-04-20T10:31:21.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f479806c920 0x7f479806ede0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f47bc15a1f0 0x7f47bc0b09e0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47bc159e10 0x7f47bc15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.445 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 --2- 172.21.15.47:0/984487363 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f47bc072190 0x7f47bc14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 >> 172.21.15.47:0/984487363 conn(0x7f47bc0d0040 msgr2=0x7f47bc156c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 shutdown_connections 2024-04-20T10:31:21.446 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.445+0000 7f47bbe00640 1 -- 172.21.15.47:0/984487363 wait complete. 2024-04-20T10:31:21.464 INFO:tasks.ceph:Scrubbing osd.7 2024-04-20T10:31:21.464 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-20T10:31:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.674+0000 7fa04be00640 1 Processor -- start 2024-04-20T10:31:21.674 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.674+0000 7fa04be00640 1 -- start start 2024-04-20T10:31:21.675 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.675+0000 7fa04be00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa04c0d2840 0x7fa04c1617d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.675+0000 7fa04be00640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c0d2e30 con 0x7fa04c0d2460 2024-04-20T10:31:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.675+0000 7fa04be00640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa04c0d3110 con 0x7fa04c161d10 2024-04-20T10:31:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.675+0000 7fa04be00640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fa04c0d33f0 con 0x7fa04c0d2840 2024-04-20T10:31:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.676+0000 7fa042000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa04c0d2840 0x7fa04c1617d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.676+0000 7fa042000640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fa04c0d2840 0x7fa04c1617d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53802/0 (socket says 172.21.15.47:53802) 2024-04-20T10:31:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.676+0000 7fa042000640 1 -- 172.21.15.47:0/1114911629 learned_addr learned my addr 172.21.15.47:0/1114911629 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:21.676 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.676+0000 7fa042000640 1 -- 172.21.15.47:0/1114911629 >> v1:172.21.15.175:6789/0 conn(0x7fa04c161d10 legacy=0x7fa04c0cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:21.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.676+0000 7fa042000640 1 -- 172.21.15.47:0/1114911629 >> v1:172.21.15.47:6789/0 conn(0x7fa04c0d2460 legacy=0x7fa04c0d4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.676+0000 7fa04a000640 1 -- 172.21.15.47:0/1114911629 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1847012124 0 0) 0x7fa04c0d2e30 con 0x7fa04c0d2460 2024-04-20T10:31:21.677 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.676+0000 7fa042000640 1 -- 172.21.15.47:0/1114911629 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0d36d0 con 0x7fa04c0d2840 2024-04-20T10:31:21.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.678+0000 7fa042000640 1 --2- 172.21.15.47:0/1114911629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c1617d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa034009870 tx=0x7fa03402ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1949d86d9918f3be server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.678+0000 7fa04a000640 1 -- 172.21.15.47:0/1114911629 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03402f850 con 0x7fa04c0d2840 2024-04-20T10:31:21.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.678+0000 7fa04a000640 1 -- 172.21.15.47:0/1114911629 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa03402f9f0 con 0x7fa04c0d2840 2024-04-20T10:31:21.678 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.678+0000 7fa04a000640 1 -- 172.21.15.47:0/1114911629 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03402fdc0 con 0x7fa04c0d2840 2024-04-20T10:31:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.679+0000 7fa04be00640 1 -- 172.21.15.47:0/1114911629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 msgr2=0x7fa04c1617d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.679+0000 7fa04be00640 1 --2- 172.21.15.47:0/1114911629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c1617d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fa034009870 tx=0x7fa03402ba30 comp rx=0 tx=0).stop 2024-04-20T10:31:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.679+0000 7fa04be00640 1 -- 172.21.15.47:0/1114911629 shutdown_connections 2024-04-20T10:31:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.679+0000 7fa04be00640 1 --2- 172.21.15.47:0/1114911629 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c1617d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.679+0000 7fa04be00640 1 -- 172.21.15.47:0/1114911629 >> 172.21.15.47:0/1114911629 conn(0x7fa04c0b7570 msgr2=0x7fa04c0b7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.679+0000 7fa04be00640 1 -- 172.21.15.47:0/1114911629 shutdown_connections 2024-04-20T10:31:21.679 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.679+0000 7fa04be00640 1 -- 172.21.15.47:0/1114911629 wait complete. 2024-04-20T10:31:21.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.681+0000 7fa04be00640 1 Processor -- start 2024-04-20T10:31:21.681 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.681+0000 7fa04be00640 1 -- start start 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.682+0000 7fa04be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c0d2460 0x7fa04c145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.682+0000 7fa04be00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c0b1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.682+0000 7fa04be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa04c161d10 0x7fa04c0b2280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.682+0000 7fa04be00640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c0d5010 con 0x7fa04c0d2840 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.682+0000 7fa04be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa04c162120 con 0x7fa04c0d2460 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.682+0000 7fa04be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa04c0d44d0 con 0x7fa04c161d10 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c0b1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c0b1520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53818/0 (socket says 172.21.15.47:53818) 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 -- 172.21.15.47:0/1743028023 learned_addr learned my addr 172.21.15.47:0/1743028023 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:21.683 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa04aa00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c0d2460 0x7fa04c145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.684 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 -- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa04c161d10 msgr2=0x7fa04c0b2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa04b400640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa04c161d10 0x7fa04c0b2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa04c161d10 0x7fa04c0b2280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 -- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c0d2460 msgr2=0x7fa04c145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa04c0d2460 0x7fa04c145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.683+0000 7fa042000640 1 -- 172.21.15.47:0/1743028023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04c0ce5f0 con 0x7fa04c0d2840 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.684+0000 7fa042000640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c0b1520 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa034009870 tx=0x7fa034004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.684+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034037d10 con 0x7fa04c0d2840 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.684+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa034046070 con 0x7fa04c0d2840 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.684+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa034041640 con 0x7fa04c0d2840 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.684+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04c0b6db0 con 0x7fa04c0d2840 2024-04-20T10:31:21.685 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.684+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa04c145f50 con 0x7fa04c0d2840 2024-04-20T10:31:21.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.685+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa04c0b1e60 con 0x7fa04c0d2840 2024-04-20T10:31:21.686 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.686+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fa034040070 con 0x7fa04c0d2840 2024-04-20T10:31:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.687+0000 7fa043e00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa02806c8f0 0x7fa02806edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.687 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.687+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fa0340ae340 con 0x7fa04c0d2840 2024-04-20T10:31:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.688+0000 7fa043e00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fa028072ed0 0x7fa028075310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.688+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 --> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0280759e0 con 0x7fa028072ed0 2024-04-20T10:31:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.688+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) ==== 24+0+0 (secure 0 0 0) 0x7fa0340b5050 con 0x7fa04c0d2840 2024-04-20T10:31:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.688+0000 7fa04aa00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa02806c8f0 0x7fa02806edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.688 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.688+0000 7fa04b400640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fa028072ed0 0x7fa028075310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.689+0000 7fa04b400640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fa028072ed0 0x7fa028075310 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.689 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.690+0000 7fa04aa00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa02806c8f0 0x7fa02806edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000f60 tx=0x7fa03c009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.691 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.692+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== osd.7 v2:172.21.15.175:6808/1688864180 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0280759e0 con 0x7fa028072ed0 2024-04-20T10:31:21.706 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.706+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 --> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fa04c075b30 con 0x7fa028072ed0 2024-04-20T10:31:21.708 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.708+0000 7fa043e00640 1 -- 172.21.15.47:0/1743028023 <== osd.7 v2:172.21.15.175:6808/1688864180 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fa04c075b30 con 0x7fa028072ed0 2024-04-20T10:31:21.708 INFO:teuthology.orchestra.run.smithi047.stdout:{ 2024-04-20T10:31:21.708 INFO:teuthology.orchestra.run.smithi047.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-20T10:31:21.708 INFO:teuthology.orchestra.run.smithi047.stdout:} 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.708+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fa028072ed0 msgr2=0x7fa028075310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.708+0000 7fa04be00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6808/1688864180,v1:172.21.15.175:6809/1688864180] conn(0x7fa028072ed0 0x7fa028075310 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.708+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa02806c8f0 msgr2=0x7fa02806edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.708+0000 7fa04be00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fa02806c8f0 0x7fa02806edb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa03c000f60 tx=0x7fa03c009040 comp rx=0 tx=0).stop 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.708+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 msgr2=0x7fa04c0b1520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.708+0000 7fa04be00640 1 --2- 172.21.15.47:0/1743028023 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fa04c0d2840 0x7fa04c0b1520 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa034009870 tx=0x7fa034004730 comp rx=0 tx=0).stop 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.709+0000 7fa04b400640 1 -- 172.21.15.47:0/1743028023 reap_dead start 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.709+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 shutdown_connections 2024-04-20T10:31:21.709 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.709+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 >> 172.21.15.47:0/1743028023 conn(0x7fa04c0b7570 msgr2=0x7fa04c15d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.709+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 shutdown_connections 2024-04-20T10:31:21.710 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.709+0000 7fa04be00640 1 -- 172.21.15.47:0/1743028023 wait complete. 2024-04-20T10:31:21.726 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-20T10:31:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.932+0000 7fe529000640 1 Processor -- start 2024-04-20T10:31:21.932 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.932+0000 7fe529000640 1 -- start start 2024-04-20T10:31:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.934+0000 7fe529000640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fe5240b2f70 0x7fe5240b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.934+0000 7fe529000640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe524071a80 con 0x7fe52406b900 2024-04-20T10:31:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.934+0000 7fe529000640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe5240ae3f0 con 0x7fe5240adf60 2024-04-20T10:31:21.934 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.934+0000 7fe529000640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fe524071cf0 con 0x7fe5240b2f70 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.934+0000 7fe523e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7fe52406b900 0x7fe52406bd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34086/0 (socket says 172.21.15.47:34086) 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.934+0000 7fe523e00640 1 -- 172.21.15.47:0/2271676219 learned_addr learned my addr 172.21.15.47:0/2271676219 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe523400640 1 --2- 172.21.15.47:0/2271676219 >> v2:172.21.15.175:3300/0 conn(0x7fe5240b2f70 0x7fe5240b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 96756529 0 0) 0x7fe524071a80 con 0x7fe52406b900 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe508003520 con 0x7fe52406b900 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 4190232794 0 0) 0x7fe508003520 con 0x7fe52406b900 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 >> v2:172.21.15.175:3300/0 conn(0x7fe5240b2f70 msgr2=0x7fe5240b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 --2- 172.21.15.47:0/2271676219 >> v2:172.21.15.175:3300/0 conn(0x7fe5240b2f70 0x7fe5240b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 >> v1:172.21.15.175:6789/0 conn(0x7fe5240adf60 legacy=0x7fe52415f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe524071fd0 con 0x7fe52406b900 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7fe51c0019f0 con 0x7fe52406b900 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 >> v1:172.21.15.47:6789/0 conn(0x7fe52406b900 legacy=0x7fe52406bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.936 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080039b0 0x7fe508003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe508004800 0x7fe508004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.935+0000 7fe522000640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5080051c0 0x7fe50800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe524071cf0 con 0x7fe5080051c0 2024-04-20T10:31:21.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe50800c060 con 0x7fe508004800 2024-04-20T10:31:21.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe50800c260 con 0x7fe5080039b0 2024-04-20T10:31:21.937 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523e00640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5080051c0 0x7fe50800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523400640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080039b0 0x7fe508003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe522a00640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe508004800 0x7fe508004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523e00640 1 -- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080039b0 msgr2=0x7fe508003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523e00640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080039b0 0x7fe508003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523e00640 1 -- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe508004800 msgr2=0x7fe508004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523e00640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe508004800 0x7fe508004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523e00640 1 -- 172.21.15.47:0/2271676219 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe508003520 con 0x7fe5080051c0 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.936+0000 7fe523400640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080039b0 0x7fe508003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.937+0000 7fe523e00640 1 -- 172.21.15.47:0/2271676219 reap_dead start 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.938+0000 7fe523e00640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5080051c0 0x7fe50800b9c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00a860 tx=0x7fe508005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ab26997a3a09ac server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.938+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c00a480 con 0x7fe5080051c0 2024-04-20T10:31:21.939 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.938+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c034630 con 0x7fe5080051c0 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.938+0000 7fe522000640 1 -- 172.21.15.47:0/2271676219 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c034a00 con 0x7fe5080051c0 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.938+0000 7fe529000640 1 -- 172.21.15.47:0/2271676219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5080051c0 msgr2=0x7fe50800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.938+0000 7fe529000640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5080051c0 0x7fe50800b9c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00a860 tx=0x7fe508005830 comp rx=0 tx=0).stop 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.939+0000 7fe529000640 1 -- 172.21.15.47:0/2271676219 shutdown_connections 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.939+0000 7fe529000640 1 --2- 172.21.15.47:0/2271676219 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5080051c0 0x7fe50800b9c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.939+0000 7fe529000640 1 -- 172.21.15.47:0/2271676219 >> 172.21.15.47:0/2271676219 conn(0x7fe5240d0040 msgr2=0x7fe5240d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.939+0000 7fe529000640 1 -- 172.21.15.47:0/2271676219 shutdown_connections 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.939+0000 7fe529000640 1 -- 172.21.15.47:0/2271676219 wait complete. 2024-04-20T10:31:21.940 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.940+0000 7fe529000640 1 Processor -- start 2024-04-20T10:31:21.941 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.941+0000 7fe529000640 1 -- start start 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.941+0000 7fe529000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080051c0 0x7fe524154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.941+0000 7fe529000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52409b800 0x7fe524151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe529000640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5241521d0 0x7fe5241f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe529000640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fe5240b4320 con 0x7fe5241521d0 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe529000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5240736c0 con 0x7fe52409b800 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe529000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5240b44a0 con 0x7fe5080051c0 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5241521d0 0x7fe5241f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5241521d0 0x7fe5241f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53846/0 (socket says 172.21.15.47:53846) 2024-04-20T10:31:21.942 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 -- 172.21.15.47:0/2343017646 learned_addr learned my addr 172.21.15.47:0/2343017646 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe522a00640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52409b800 0x7fe524151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 -- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080051c0 msgr2=0x7fe524154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080051c0 0x7fe524154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.943 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 -- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52409b800 msgr2=0x7fe524151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52409b800 0x7fe524151c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.942+0000 7fe523e00640 1 -- 172.21.15.47:0/2343017646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe524158af0 con 0x7fe5241521d0 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.943+0000 7fe523e00640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5241521d0 0x7fe5241f7e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00a980 tx=0x7fe5240b4860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.943+0000 7fe51b400640 1 -- 172.21.15.47:0/2343017646 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c039040 con 0x7fe5241521d0 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.943+0000 7fe51b400640 1 -- 172.21.15.47:0/2343017646 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe51c043070 con 0x7fe5241521d0 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.943+0000 7fe51b400640 1 -- 172.21.15.47:0/2343017646 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c00a480 con 0x7fe5241521d0 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.943+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5241f8b30 con 0x7fe5241521d0 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.943+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe524154650 con 0x7fe5241521d0 2024-04-20T10:31:21.944 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.944+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe524075b60 con 0x7fe5241521d0 2024-04-20T10:31:21.945 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.945+0000 7fe51b400640 1 -- 172.21.15.47:0/2343017646 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fe51c034630 con 0x7fe5241521d0 2024-04-20T10:31:21.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.946+0000 7fe51b400640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe50006c920 0x7fe50006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:21.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.946+0000 7fe51b400640 1 -- 172.21.15.47:0/2343017646 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fe51c0ad670 con 0x7fe5241521d0 2024-04-20T10:31:21.946 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.947+0000 7fe523400640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe50006c920 0x7fe50006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:21.948 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.948+0000 7fe523400640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe50006c920 0x7fe50006ede0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe5140099e0 tx=0x7fe514008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:21.950 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:21.950+0000 7fe51b400640 1 -- 172.21.15.47:0/2343017646 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fe51c07bac0 con 0x7fe5241521d0 2024-04-20T10:31:22.155 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.154+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fe52406d220 con 0x7fe50006c920 2024-04-20T10:31:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.156+0000 7fe51b400640 1 -- 172.21.15.47:0/2343017646 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe52406d220 con 0x7fe50006c920 2024-04-20T10:31:22.156 INFO:teuthology.orchestra.run.smithi047.stderr:instructed osd(s) 7 to deep-scrub 2024-04-20T10:31:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe50006c920 msgr2=0x7fe50006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe50006c920 0x7fe50006ede0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe5140099e0 tx=0x7fe514008040 comp rx=0 tx=0).stop 2024-04-20T10:31:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5241521d0 msgr2=0x7fe5241f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.158 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5241521d0 0x7fe5241f7e70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7fe51c00a980 tx=0x7fe5240b4860 comp rx=0 tx=0).stop 2024-04-20T10:31:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 shutdown_connections 2024-04-20T10:31:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fe50006c920 0x7fe50006ede0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fe5241521d0 0x7fe5241f7e70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe52409b800 0x7fe524151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 --2- 172.21.15.47:0/2343017646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5080051c0 0x7fe524154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.158+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 >> 172.21.15.47:0/2343017646 conn(0x7fe5240d0040 msgr2=0x7fe52406db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:22.159 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.159+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 shutdown_connections 2024-04-20T10:31:22.160 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.159+0000 7fe529000640 1 -- 172.21.15.47:0/2343017646 wait complete. 2024-04-20T10:31:22.178 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-20T10:31:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.393+0000 7f3027200640 1 Processor -- start 2024-04-20T10:31:22.393 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.393+0000 7f3027200640 1 -- start start 2024-04-20T10:31:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.394+0000 7f3027200640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7f30200cf350 0x7f30200711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.394+0000 7f3027200640 1 -- --> v1:172.21.15.47:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3020160250 con 0x7f30200ccd50 2024-04-20T10:31:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.395+0000 7f3027200640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3020160530 con 0x7f30200ce790 2024-04-20T10:31:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.395+0000 7f3027200640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7f3020160dd0 con 0x7f30200cf350 2024-04-20T10:31:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.395+0000 7f3025e00640 1 --1- >> v1:172.21.15.47:6789/0 conn(0x7f30200ccd50 0x7f30200cec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.47:6789/0 says I am v1:172.21.15.47:34098/0 (socket says 172.21.15.47:34098) 2024-04-20T10:31:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.395+0000 7f3025e00640 1 -- 172.21.15.47:0/4166577411 learned_addr learned my addr 172.21.15.47:0/4166577411 (peer_addr_for_me v1:172.21.15.47:0/0) 2024-04-20T10:31:22.395 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.395+0000 7f3025400640 1 --2- 172.21.15.47:0/4166577411 >> v2:172.21.15.175:3300/0 conn(0x7f30200cf350 0x7f30200711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.395+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 <== mon.0 v1:172.21.15.47:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4241401374 0 0) 0x7f3020160250 con 0x7f30200ccd50 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.395+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 --> v1:172.21.15.47:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3008003520 con 0x7f30200ccd50 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 <== mon.0 v1:172.21.15.47:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) ==== 214+0+0 (unknown 2270676681 0 0) 0x7f3008003520 con 0x7f30200ccd50 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 >> v2:172.21.15.175:3300/0 conn(0x7f30200cf350 msgr2=0x7f30200711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 --2- 172.21.15.47:0/4166577411 >> v2:172.21.15.175:3300/0 conn(0x7f30200cf350 0x7f30200711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 >> v1:172.21.15.175:6789/0 conn(0x7f30200ce790 legacy=0x7f30200d0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 --> v1:172.21.15.47:6789/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3020160810 con 0x7f30200ccd50 2024-04-20T10:31:22.397 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 <== mon.0 v1:172.21.15.47:6789/0 3 ==== mon_map magic: 0 ==== 413+0+0 (unknown 903488452 0 0) 0x7f3014001670 con 0x7f30200ccd50 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 >> v1:172.21.15.47:6789/0 conn(0x7f30200ccd50 legacy=0x7f30200cec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008003a00 0x7f3008003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30080051c0 0x7f300800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f3020160dd0 con 0x7f3008004800 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f300800c0c0 con 0x7f3008003a00 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f300800c2c0 con 0x7f30080051c0 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.396+0000 7f3025400640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.397+0000 7f3025e00640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008003a00 0x7f3008003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.397+0000 7f3025400640 1 -- 172.21.15.47:0/4166577411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30080051c0 msgr2=0x7f300800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.397+0000 7f3025400640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30080051c0 0x7f300800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.397+0000 7f3025400640 1 -- 172.21.15.47:0/4166577411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008003a00 msgr2=0x7f3008003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:22.398 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.397+0000 7f3025400640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008003a00 0x7f3008003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.397+0000 7f3025400640 1 -- 172.21.15.47:0/4166577411 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3008003520 con 0x7f3008004800 2024-04-20T10:31:22.399 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.397+0000 7f3026800640 1 -- 172.21.15.47:0/4166577411 reap_dead start 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.398+0000 7f3025400640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3008004800 0x7f3008004c80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f300c009ac0 tx=0x7f3008005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91aa2d223079369 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.398+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c039070 con 0x7f3008004800 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.398+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300c02c6d0 con 0x7f3008004800 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.398+0000 7f3024a00640 1 -- 172.21.15.47:0/4166577411 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c009690 con 0x7f3008004800 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.399+0000 7f3027200640 1 -- 172.21.15.47:0/4166577411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3008004800 msgr2=0x7f3008004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.399+0000 7f3027200640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3008004800 0x7f3008004c80 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f300c009ac0 tx=0x7f3008005830 comp rx=0 tx=0).stop 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.399+0000 7f3027200640 1 -- 172.21.15.47:0/4166577411 shutdown_connections 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.399+0000 7f3027200640 1 --2- 172.21.15.47:0/4166577411 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f3008004800 0x7f3008004c80 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.399+0000 7f3027200640 1 -- 172.21.15.47:0/4166577411 >> 172.21.15.47:0/4166577411 conn(0x7f30200b3df0 msgr2=0x7f30200b41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:22.400 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.399+0000 7f3027200640 1 -- 172.21.15.47:0/4166577411 shutdown_connections 2024-04-20T10:31:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.399+0000 7f3027200640 1 -- 172.21.15.47:0/4166577411 wait complete. 2024-04-20T10:31:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.401+0000 7f3027200640 1 Processor -- start 2024-04-20T10:31:22.401 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.401+0000 7f3027200640 1 -- start start 2024-04-20T10:31:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3027200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008004800 0x7f30200b01b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3027200640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f302015b3e0 0x7f30201e81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3027200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30201e8730 0x7f30200ae3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3027200640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f302015feb0 con 0x7f302015b3e0 2024-04-20T10:31:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3027200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30200cd130 con 0x7f3008004800 2024-04-20T10:31:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3027200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3020160030 con 0x7f30201e8730 2024-04-20T10:31:22.403 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f302015b3e0 0x7f30201e81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f302015b3e0 0x7f30201e81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53864/0 (socket says 172.21.15.47:53864) 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 -- 172.21.15.47:0/1621613337 learned_addr learned my addr 172.21.15.47:0/1621613337 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 -- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30201e8730 msgr2=0x7f30200ae3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30201e8730 0x7f30200ae3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 -- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008004800 msgr2=0x7f30200b01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008004800 0x7f30200b01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.402+0000 7f3025400640 1 -- 172.21.15.47:0/1621613337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30200b06f0 con 0x7f302015b3e0 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.403+0000 7f3025400640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f302015b3e0 0x7f30201e81f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f300c009be0 tx=0x7f300c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.403+0000 7f301a000640 1 -- 172.21.15.47:0/1621613337 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c042020 con 0x7f302015b3e0 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.403+0000 7f301a000640 1 -- 172.21.15.47:0/1621613337 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f300c03a040 con 0x7f302015b3e0 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.403+0000 7f301a000640 1 -- 172.21.15.47:0/1621613337 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c039070 con 0x7f302015b3e0 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.403+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30200af110 con 0x7f302015b3e0 2024-04-20T10:31:22.404 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.403+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30200af6d0 con 0x7f302015b3e0 2024-04-20T10:31:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.405+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3020075b30 con 0x7f302015b3e0 2024-04-20T10:31:22.405 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.405+0000 7f301a000640 1 -- 172.21.15.47:0/1621613337 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f300c045020 con 0x7f302015b3e0 2024-04-20T10:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.406+0000 7f301a000640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f300006c8f0 0x7f300006edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.406 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.407+0000 7f301a000640 1 -- 172.21.15.47:0/1621613337 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f300c0acfe0 con 0x7f302015b3e0 2024-04-20T10:31:22.407 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.407+0000 7f3025e00640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f300006c8f0 0x7f300006edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.408 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.409+0000 7f3025e00640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f300006c8f0 0x7f300006edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3014000ff0 tx=0x7f3014009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:22.411 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.411+0000 7f301a000640 1 -- 172.21.15.47:0/1621613337 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f300c04c070 con 0x7f302015b3e0 2024-04-20T10:31:22.613 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.613+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 --> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3020070f90 con 0x7f300006c8f0 2024-04-20T10:31:22.615 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.615+0000 7f301a000640 1 -- 172.21.15.47:0/1621613337 <== mgr.4112 v2:172.21.15.175:6832/1546709378 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13390 (secure 0 0 0) 0x7f3020070f90 con 0x7f300006c8f0 2024-04-20T10:31:22.616 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:22.616 INFO:teuthology.orchestra.run.smithi047.stderr:dumped all 2024-04-20T10:31:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f300006c8f0 msgr2=0x7f300006edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f300006c8f0 0x7f300006edb0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3014000ff0 tx=0x7f3014009040 comp rx=0 tx=0).stop 2024-04-20T10:31:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f302015b3e0 msgr2=0x7f30201e81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f302015b3e0 0x7f30201e81f0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f300c009be0 tx=0x7f300c004730 comp rx=0 tx=0).stop 2024-04-20T10:31:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 shutdown_connections 2024-04-20T10:31:22.618 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30201e8730 0x7f30200ae3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f300006c8f0 0x7f300006edb0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f302015b3e0 0x7f30201e81f0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 --2- 172.21.15.47:0/1621613337 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3008004800 0x7f30200b01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 >> 172.21.15.47:0/1621613337 conn(0x7f30200b3df0 msgr2=0x7f30200d1300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 shutdown_connections 2024-04-20T10:31:22.619 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.618+0000 7f3027200640 1 -- 172.21.15.47:0/1621613337 wait complete. 2024-04-20T10:31:22.636 INFO:teuthology.orchestra.run.smithi047.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-04-20T10:31:21.150430+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239660,"kb_used_data":6856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735060,"statfs":{"total":773094113280,"available":772848701440,"internally_reserved":0,"allocated":7020544,"data_stored":3778640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-42,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-1,"num_write_kb":-1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002493"},"pg_stats":[{"pgid":"1.0","version":"61'119","reported_seq":252,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-04-20T10:31:18.582345+0000","last_change":"2024-04-20T10:31:18.582389+0000","last_active":"2024-04-20T10:31:18.582345+0000","last_peered":"2024-04-20T10:31:18.582345+0000","last_clean":"2024-04-20T10:31:18.582345+0000","last_became_active":"2024-04-20T10:27:23.053430+0000","last_became_peered":"2024-04-20T10:27:23.053430+0000","last_unstale":"2024-04-20T10:31:18.582345+0000","last_undegraded":"2024-04-20T10:31:18.582345+0000","last_fullsized":"2024-04-20T10:31:18.582345+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"61'119","last_scrub_stamp":"2024-04-20T10:31:18.582255+0000","last_deep_scrub":"61'119","last_deep_scrub_stamp":"2024-04-20T10:31:18.582255+0000","last_clean_scrub_stamp":"2024-04-20T10:31:18.582255+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-21T14:35:21.318731+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27732,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":729088,"data_stored":324738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":324738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509491,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28308,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343532,"statfs":{"total":96636764160,"available":96607776768,"internally_reserved":0,"allocated":1323008,"data_stored":915106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":729088,"data_stored":324738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31892,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":729088,"data_stored":324738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509491,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1292,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1323008,"data_stored":915106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509491,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31896,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":729088,"data_stored":324738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542195,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27736,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344104,"statfs":{"total":96636764160,"available":96608362496,"internally_reserved":0,"allocated":729088,"data_stored":324738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-20T10:31:22.640 DEBUG:teuthology.orchestra.run.smithi047:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-20T10:31:22.845 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.845+0000 7f2dc9400640 1 Processor -- start 2024-04-20T10:31:22.846 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.846+0000 7f2dc9400640 1 -- start start 2024-04-20T10:31:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dc9400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2dc415ca40 0x7f2dc40ad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dc9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc40d0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.847 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dc9400640 1 -- --> v1:172.21.15.175:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2dc4141c60 con 0x7f2dc4156c20 2024-04-20T10:31:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dc9400640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7f2dc406c010 con 0x7f2dc415ca40 2024-04-20T10:31:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dc9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc415cf90 con 0x7f2dc40ad8e0 2024-04-20T10:31:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dbbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2dc415ca40 0x7f2dc40ad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dbbe00640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7f2dc415ca40 0x7f2dc40ad3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53872/0 (socket says 172.21.15.47:53872) 2024-04-20T10:31:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.847+0000 7f2dbbe00640 1 -- 172.21.15.47:0/3867531236 learned_addr learned my addr 172.21.15.47:0/3867531236 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.848+0000 7f2dc3200640 1 --2- 172.21.15.47:0/3867531236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc40d0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.848 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.848+0000 7f2dbbe00640 1 -- 172.21.15.47:0/3867531236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 msgr2=0x7f2dc40d0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.848+0000 7f2dbbe00640 1 --2- 172.21.15.47:0/3867531236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc40d0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.848+0000 7f2dbbe00640 1 -- 172.21.15.47:0/3867531236 >> v1:172.21.15.175:6789/0 conn(0x7f2dc4156c20 legacy=0x7f2dc406b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.848+0000 7f2dbbe00640 1 -- 172.21.15.47:0/3867531236 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc415d270 con 0x7f2dc415ca40 2024-04-20T10:31:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.849+0000 7f2dbbe00640 1 --2- 172.21.15.47:0/3867531236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc415ca40 0x7f2dc40ad3a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baf4121cefeaee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:22.849 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.849+0000 7f2dc1e00640 1 -- 172.21.15.47:0/3867531236 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac035020 con 0x7f2dc415ca40 2024-04-20T10:31:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.849+0000 7f2dc1e00640 1 -- 172.21.15.47:0/3867531236 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac031640 con 0x7f2dc415ca40 2024-04-20T10:31:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc1e00640 1 -- 172.21.15.47:0/3867531236 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac02d040 con 0x7f2dc415ca40 2024-04-20T10:31:22.850 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 -- 172.21.15.47:0/3867531236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc415ca40 msgr2=0x7f2dc40ad3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 --2- 172.21.15.47:0/3867531236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc415ca40 0x7f2dc40ad3a0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2dac009870 tx=0x7f2dac02b8f0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 -- 172.21.15.47:0/3867531236 shutdown_connections 2024-04-20T10:31:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 --2- 172.21.15.47:0/3867531236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc40d0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 --2- 172.21.15.47:0/3867531236 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc415ca40 0x7f2dc40ad3a0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 -- 172.21.15.47:0/3867531236 >> 172.21.15.47:0/3867531236 conn(0x7f2dc40b3350 msgr2=0x7f2dc40b3760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 -- 172.21.15.47:0/3867531236 shutdown_connections 2024-04-20T10:31:22.851 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.850+0000 7f2dc9400640 1 -- 172.21.15.47:0/3867531236 wait complete. 2024-04-20T10:31:22.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.852+0000 7f2dc9400640 1 Processor -- start 2024-04-20T10:31:22.852 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.852+0000 7f2dc9400640 1 -- start start 2024-04-20T10:31:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dc9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc414ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dc9400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc4156c20 0x7f2dc40b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dc9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415ca40 0x7f2dc41f4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dc9400640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc40d08a0 con 0x7f2dc4156c20 2024-04-20T10:31:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dc9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc415d830 con 0x7f2dc415ca40 2024-04-20T10:31:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dc9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2dc40d0a20 con 0x7f2dc40ad8e0 2024-04-20T10:31:22.854 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc4156c20 0x7f2dc40b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.853+0000 7f2dbbe00640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc4156c20 0x7f2dc40b3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53882/0 (socket says 172.21.15.47:53882) 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dbbe00640 1 -- 172.21.15.47:0/784854912 learned_addr learned my addr 172.21.15.47:0/784854912 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dc2800640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc414ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dc3200640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415ca40 0x7f2dc41f4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dbbe00640 1 -- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 msgr2=0x7f2dc414ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dbbe00640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc414ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dbbe00640 1 -- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415ca40 msgr2=0x7f2dc41f4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dbbe00640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415ca40 0x7f2dc41f4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dbbe00640 1 -- 172.21.15.47:0/784854912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc40cf5b0 con 0x7f2dc4156c20 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dc3200640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415ca40 0x7f2dc41f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:22.855 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.854+0000 7f2dbbe00640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc4156c20 0x7f2dc40b3790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000c00 tx=0x7f2dac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:22.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.855+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac03aa60 con 0x7f2dc4156c20 2024-04-20T10:31:22.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.855+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dac03ac20 con 0x7f2dc4156c20 2024-04-20T10:31:22.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.855+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dac0443f0 con 0x7f2dc4156c20 2024-04-20T10:31:22.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.855+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc41f4da0 con 0x7f2dc4156c20 2024-04-20T10:31:22.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.855+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc41555d0 con 0x7f2dc4156c20 2024-04-20T10:31:22.856 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.856+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2dc4068980 con 0x7f2dc4156c20 2024-04-20T10:31:22.857 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.857+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7f2dac038020 con 0x7f2dc4156c20 2024-04-20T10:31:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.858+0000 7f2dbaa00640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2da006c920 0x7f2da006ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.858+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7f2dac0ad7c0 con 0x7f2dc4156c20 2024-04-20T10:31:22.858 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.859+0000 7f2dc2800640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2da006c920 0x7f2da006ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:22.860 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.860+0000 7f2dc2800640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2da006c920 0x7f2da006ede0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2db4007fd0 tx=0x7f2db4009040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:22.863 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:22.863+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7f2dac03f070 con 0x7f2dc4156c20 2024-04-20T10:31:23.067 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.067+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f2dc40cfd90 con 0x7f2dc4156c20 2024-04-20T10:31:23.073 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.073+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f2dac07bc10 con 0x7f2dc4156c20 2024-04-20T10:31:23.075 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.074+0000 7f2dbaa00640 1 -- 172.21.15.47:0/784854912 <== mon.0 v2:172.21.15.47:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f2dac07b510 con 0x7f2dc4156c20 2024-04-20T10:31:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2da006c920 msgr2=0x7f2da006ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.076 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2da006c920 0x7f2da006ede0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2db4007fd0 tx=0x7f2db4009040 comp rx=0 tx=0).stop 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc4156c20 msgr2=0x7f2dc40b3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc4156c20 0x7f2dc40b3790 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f2dac000c00 tx=0x7f2dac0046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 shutdown_connections 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2dc415ca40 0x7f2dc41f4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7f2da006c920 0x7f2da006ede0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7f2dc4156c20 0x7f2dc40b3790 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 --2- 172.21.15.47:0/784854912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2dc40ad8e0 0x7f2dc414ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 >> 172.21.15.47:0/784854912 conn(0x7f2dc40b3350 msgr2=0x7f2dc415db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 shutdown_connections 2024-04-20T10:31:23.077 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.076+0000 7f2dc9400640 1 -- 172.21.15.47:0/784854912 wait complete. 2024-04-20T10:31:23.095 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-20T10:31:23.095 DEBUG:teuthology.orchestra.run.smithi047:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-20T10:31:23.301 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.301+0000 7fea3a800640 1 Processor -- start 2024-04-20T10:31:23.315 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.301+0000 7fea3a800640 1 -- start start 2024-04-20T10:31:23.319 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea3a800640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3415b400 0x7fea34072d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea3a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415eec0 0x7fea3415f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea3a800640 1 --2- >> v2:172.21.15.175:3300/0 conn(0x7fea3415f870 0x7fea3415b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea3a800640 1 -- --> v2:172.21.15.47:3300/0 -- mon_getmap magic: 0 -- 0x7fea3406d280 con 0x7fea3415b400 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea3a800640 1 -- --> v2:172.21.15.175:3300/0 -- mon_getmap magic: 0 -- 0x7fea3406ccc0 con 0x7fea3415f870 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea3a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea3406cfa0 con 0x7fea3415eec0 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea39400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3415b400 0x7fea34072d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea39400640 1 --2- >> v2:172.21.15.47:3300/0 conn(0x7fea3415b400 0x7fea34072d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53892/0 (socket says 172.21.15.47:53892) 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.303+0000 7fea39400640 1 -- 172.21.15.47:0/4234106158 learned_addr learned my addr 172.21.15.47:0/4234106158 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea38a00640 1 --2- 172.21.15.47:0/4234106158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415eec0 0x7fea3415f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea39e00640 1 --2- 172.21.15.47:0/4234106158 >> v2:172.21.15.175:3300/0 conn(0x7fea3415f870 0x7fea3415b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea39400640 1 -- 172.21.15.47:0/4234106158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415eec0 msgr2=0x7fea3415f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea39400640 1 --2- 172.21.15.47:0/4234106158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415eec0 0x7fea3415f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea39400640 1 -- 172.21.15.47:0/4234106158 >> v2:172.21.15.175:3300/0 conn(0x7fea3415f870 msgr2=0x7fea3415b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-20T10:31:23.320 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea39400640 1 --2- 172.21.15.47:0/4234106158 >> v2:172.21.15.175:3300/0 conn(0x7fea3415f870 0x7fea3415b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea39400640 1 -- 172.21.15.47:0/4234106158 --> v2:172.21.15.47:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3406e440 con 0x7fea3415b400 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.304+0000 7fea39e00640 1 --2- 172.21.15.47:0/4234106158 >> v2:172.21.15.175:3300/0 conn(0x7fea3415f870 0x7fea3415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.305+0000 7fea39400640 1 --2- 172.21.15.47:0/4234106158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea34072d00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fea34074a10 tx=0x7fea2802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac6a9f08e63651a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.305+0000 7fea33e00640 1 -- 172.21.15.47:0/4234106158 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28035020 con 0x7fea3415b400 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.305+0000 7fea33e00640 1 -- 172.21.15.47:0/4234106158 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fea28031870 con 0x7fea3415b400 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.305+0000 7fea33e00640 1 -- 172.21.15.47:0/4234106158 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28031c60 con 0x7fea3415b400 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 -- 172.21.15.47:0/4234106158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 msgr2=0x7fea34072d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 --2- 172.21.15.47:0/4234106158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea34072d00 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fea34074a10 tx=0x7fea2802b910 comp rx=0 tx=0).stop 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 -- 172.21.15.47:0/4234106158 shutdown_connections 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 --2- 172.21.15.47:0/4234106158 >> v2:172.21.15.175:3300/0 conn(0x7fea3415f870 0x7fea3415b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 --2- 172.21.15.47:0/4234106158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415eec0 0x7fea3415f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 --2- 172.21.15.47:0/4234106158 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea34072d00 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.321 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 -- 172.21.15.47:0/4234106158 >> 172.21.15.47:0/4234106158 conn(0x7fea340d0040 msgr2=0x7fea340d0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 -- 172.21.15.47:0/4234106158 shutdown_connections 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.306+0000 7fea3a800640 1 -- 172.21.15.47:0/4234106158 wait complete. 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.308+0000 7fea3a800640 1 Processor -- start 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.308+0000 7fea3a800640 1 -- start start 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea3a800640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea3414faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea3a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415eec0 0x7fea3409a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea3a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415f870 0x7fea3409af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea3a800640 1 -- --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_getmap magic: 0 -- 0x7fea3415fc50 con 0x7fea3415b400 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea3a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea34073240 con 0x7fea3415eec0 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea3a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea3415bd20 con 0x7fea3415f870 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea3414faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea39400640 1 --2- >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea3414faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.47:3300/0 says I am v2:172.21.15.47:53908/0 (socket says 172.21.15.47:53908) 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.309+0000 7fea39400640 1 -- 172.21.15.47:0/836411450 learned_addr learned my addr 172.21.15.47:0/836411450 (peer_addr_for_me v2:172.21.15.47:0/0) 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.310+0000 7fea39400640 1 -- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415f870 msgr2=0x7fea3409af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.322 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.310+0000 7fea39400640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415f870 0x7fea3409af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.310+0000 7fea39400640 1 -- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415eec0 msgr2=0x7fea3409a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.310+0000 7fea38a00640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415eec0 0x7fea3409a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.310+0000 7fea39400640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415eec0 0x7fea3409a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.310+0000 7fea39400640 1 -- 172.21.15.47:0/836411450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea340cd3a0 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.310+0000 7fea39400640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea3414faa0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fea2802b950 tx=0x7fea280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.311+0000 7fea31600640 1 -- 172.21.15.47:0/836411450 <== mon.0 v2:172.21.15.47:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28040070 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.311+0000 7fea31600640 1 -- 172.21.15.47:0/836411450 <== mon.0 v2:172.21.15.47:3300/0 2 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7fea28030c50 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.311+0000 7fea31600640 1 -- 172.21.15.47:0/836411450 <== mon.0 v2:172.21.15.47:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28035040 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.311+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea340cd680 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.311+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea340cdc40 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.312+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea34075b60 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.313+0000 7fea31600640 1 -- 172.21.15.47:0/836411450 <== mon.0 v2:172.21.15.47:3300/0 4 ==== mgrmap(e 5) ==== 91040+0+0 (secure 0 0 0) 0x7fea28038020 con 0x7fea3415b400 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.314+0000 7fea31600640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea1406c8f0 0x7fea1406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-20T10:31:23.323 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.314+0000 7fea31600640 1 -- 172.21.15.47:0/836411450 <== mon.0 v2:172.21.15.47:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5507+0+0 (secure 0 0 0) 0x7fea280ad980 con 0x7fea3415b400 2024-04-20T10:31:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.315+0000 7fea38a00640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea1406c8f0 0x7fea1406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-20T10:31:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.316+0000 7fea38a00640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea1406c8f0 0x7fea1406edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea341f5e70 tx=0x7fea20008040 comp rx=0 tx=0).ready entity=mgr.4112 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-20T10:31:23.324 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.319+0000 7fea31600640 1 -- 172.21.15.47:0/836411450 <== mon.0 v2:172.21.15.47:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187257 (secure 0 0 0) 0x7fea2807be60 con 0x7fea3415b400 2024-04-20T10:31:23.566 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.566+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 --> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fea340b5220 con 0x7fea3415b400 2024-04-20T10:31:23.567 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.567+0000 7fea31600640 1 -- 172.21.15.47:0/836411450 <== mon.0 v2:172.21.15.47:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+3872 (secure 0 0 0) 0x7fea2807b760 con 0x7fea3415b400 2024-04-20T10:31:23.567 INFO:teuthology.orchestra.run.smithi047.stdout: 2024-04-20T10:31:23.568 INFO:teuthology.orchestra.run.smithi047.stdout:{"epoch":43,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5120,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4213997352","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4213997352},{"type":"v1","addr":"172.21.15.175:6835","nonce":4213997352}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":5524,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/1377196750","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6836","nonce":1377196750},{"type":"v1","addr":"172.21.15.175:6837","nonce":1377196750}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":37},{"gid":6270,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6835/3774977814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6834","nonce":3774977814},{"type":"v1","addr":"172.21.15.47:6835","nonce":3774977814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":6423,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.47:6833/698977162","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.47:6832","nonce":698977162},{"type":"v1","addr":"172.21.15.47:6833","nonce":698977162}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":43}],"filesystems":[]} 2024-04-20T10:31:23.568 INFO:teuthology.orchestra.run.smithi047.stderr:dumped fsmap epoch 43 2024-04-20T10:31:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea1406c8f0 msgr2=0x7fea1406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea1406c8f0 0x7fea1406edb0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fea341f5e70 tx=0x7fea20008040 comp rx=0 tx=0).stop 2024-04-20T10:31:23.570 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 msgr2=0x7fea3414faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea3414faa0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fea2802b950 tx=0x7fea280046c0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 shutdown_connections 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fea3415f870 0x7fea3409af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea3415eec0 0x7fea3409a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.175:6832/1546709378,v1:172.21.15.175:6833/1546709378] conn(0x7fea1406c8f0 0x7fea1406edb0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 --2- 172.21.15.47:0/836411450 >> [v2:172.21.15.47:3300/0,v1:172.21.15.47:6789/0] conn(0x7fea3415b400 0x7fea3414faa0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 >> 172.21.15.47:0/836411450 conn(0x7fea340d0040 msgr2=0x7fea3406f750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 shutdown_connections 2024-04-20T10:31:23.571 INFO:teuthology.orchestra.run.smithi047.stderr:2024-04-20T10:31:23.570+0000 7fea3a800640 1 -- 172.21.15.47:0/836411450 wait complete. 2024-04-20T10:31:23.590 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-20T10:31:23.590 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-20T10:31:23.590 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-20T10:31:23.590 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:23.609 INFO:tasks.ceph.mds.a:Stopped 2024-04-20T10:31:23.610 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-20T10:31:23.610 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:23.680 INFO:tasks.ceph.mds.c:Stopped 2024-04-20T10:31:23.680 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-20T10:31:23.680 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:23.707 INFO:tasks.ceph.mds.b:Stopped 2024-04-20T10:31:23.707 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-20T10:31:23.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:23.771 INFO:tasks.ceph.mds.d:Stopped 2024-04-20T10:31:23.772 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-20T10:31:23.772 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-20T10:31:23.772 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:23.831 INFO:tasks.ceph.osd.0:Stopped 2024-04-20T10:31:23.831 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-20T10:31:23.831 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:23.894 INFO:tasks.ceph.osd.1:Stopped 2024-04-20T10:31:23.894 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-20T10:31:23.894 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:23.957 INFO:tasks.ceph.osd.2:Stopped 2024-04-20T10:31:23.957 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-20T10:31:23.957 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.028 INFO:tasks.ceph.osd.3:Stopped 2024-04-20T10:31:24.029 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-20T10:31:24.029 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.103 INFO:tasks.ceph.osd.4:Stopped 2024-04-20T10:31:24.103 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-20T10:31:24.103 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.161 INFO:tasks.ceph.osd.5:Stopped 2024-04-20T10:31:24.161 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-20T10:31:24.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.215 INFO:tasks.ceph.osd.6:Stopped 2024-04-20T10:31:24.215 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-20T10:31:24.215 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.298 INFO:tasks.ceph.osd.7:Stopped 2024-04-20T10:31:24.299 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-20T10:31:24.299 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-20T10:31:24.299 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.379 INFO:tasks.ceph.mgr.y:Stopped 2024-04-20T10:31:24.379 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-20T10:31:24.379 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.463 INFO:tasks.ceph.mgr.x:Stopped 2024-04-20T10:31:24.463 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-20T10:31:24.463 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-20T10:31:24.464 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.505 INFO:tasks.ceph.mon.a:Stopped 2024-04-20T10:31:24.505 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-20T10:31:24.505 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.548 INFO:tasks.ceph.mon.b:Stopped 2024-04-20T10:31:24.548 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-20T10:31:24.548 INFO:teuthology.orchestra.run:waiting for 300 2024-04-20T10:31:24.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-04-20T10:31:24.585 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-04-20T10:31:24.585 INFO:tasks.ceph.mon.c:Stopped 2024-04-20T10:31:24.585 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-20T10:31:24.586 DEBUG:teuthology.orchestra.run.smithi047:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'force file system read-only' | egrep -v 'bad backtrace' | egrep -v 'MDS in read-only mode' | egrep -v '\(MDS_READ_ONLY\)' | head -n 1 2024-04-20T10:31:24.698 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:31:24.698 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-20T10:31:24.924 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:31:24.924 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-20T10:31:25.056 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:31:25.057 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-20T10:31:25.207 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:31:25.207 DEBUG:teuthology.orchestra.run.smithi047:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-20T10:31:25.340 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:31:25.341 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-20T10:31:25.688 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:31:25.688 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-20T10:31:25.808 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:31:25.808 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-20T10:31:25.954 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:31:25.955 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-20T10:31:26.101 INFO:tasks.ceph:Archiving mon data... 2024-04-20T10:31:26.102 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/data/mon.a.tgz 2024-04-20T10:31:26.102 DEBUG:teuthology.orchestra.run.smithi047:> mktemp 2024-04-20T10:31:26.160 INFO:teuthology.orchestra.run.smithi047.stdout:/tmp/tmp.EhLQTY8nx9 2024-04-20T10:31:26.161 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.EhLQTY8nx9 2024-04-20T10:31:26.345 DEBUG:teuthology.orchestra.run.smithi047:> sudo chmod 0666 /tmp/tmp.EhLQTY8nx9 2024-04-20T10:31:26.495 DEBUG:teuthology.orchestra.remote:smithi047:/tmp/tmp.EhLQTY8nx9 is 381KB 2024-04-20T10:31:26.607 DEBUG:teuthology.orchestra.run.smithi047:> rm -fr /tmp/tmp.EhLQTY8nx9 2024-04-20T10:31:26.666 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/data/mon.b.tgz 2024-04-20T10:31:26.666 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-04-20T10:31:26.724 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.xHWlswgDi7 2024-04-20T10:31:26.724 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xHWlswgDi7 2024-04-20T10:31:26.940 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.xHWlswgDi7 2024-04-20T10:31:27.081 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.xHWlswgDi7 is 503KB 2024-04-20T10:31:27.190 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.xHWlswgDi7 2024-04-20T10:31:27.244 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/data/mon.c.tgz 2024-04-20T10:31:27.244 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-04-20T10:31:27.293 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.ccdGjwiWJV 2024-04-20T10:31:27.293 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ccdGjwiWJV 2024-04-20T10:31:27.487 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.ccdGjwiWJV 2024-04-20T10:31:27.612 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.ccdGjwiWJV is 503KB 2024-04-20T10:31:27.717 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.ccdGjwiWJV 2024-04-20T10:31:27.770 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-20T10:31:27.771 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T10:31:27.778 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-20T10:31:27.923 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T10:31:27.981 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-20T10:31:27.981 INFO:tasks.ceph:Archiving crash dumps... 2024-04-20T10:31:27.984 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/remote/smithi047/crash 2024-04-20T10:31:27.985 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T10:31:28.077 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/remote/smithi175/crash 2024-04-20T10:31:28.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-20T10:31:28.163 INFO:tasks.ceph:Shutting down logrotate 2024-04-20T10:31:28.164 DEBUG:teuthology.orchestra.run.smithi047:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:28.172 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:28.328 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:28.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-20T10:31:28.510 INFO:tasks.ceph:Compressing logs... 2024-04-20T10:31:28.510 DEBUG:teuthology.orchestra.run.smithi047:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T10:31:28.518 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-20T10:31:28.600 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27009.log 2024-04-20T10:31:28.600 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.28654.log 2024-04-20T10:31:28.601 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.28198.log 2024-04-20T10:31:28.602 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35103.log 2024-04-20T10:31:28.602 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.3.28654.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T10:31:28.604 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.28198.log: 92.2% -- replaced with /var/log/ceph/ceph-client.3.28654.log.gz 2024-04-20T10:31:28.604 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22241.log 2024-04-20T10:31:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27009.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.27009.log.gz 2024-04-20T10:31:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32772.log 2024-04-20T10:31:28.604 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37370.log 2024-04-20T10:31:28.604 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35103.log.gz 2024-04-20T10:31:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32772.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32772.log.gz 2024-04-20T10:31:28.606 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35835.log 2024-04-20T10:31:28.606 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.1.28198.log.gz 2024-04-20T10:31:28.606 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T10:31:28.606 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22241.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-20T10:31:28.606 INFO:teuthology.orchestra.run.smithi175.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.22241.log.gz 2024-04-20T10:31:28.607 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.audit.log: 91.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T10:31:28.607 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-20T10:31:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40041.log 2024-04-20T10:31:28.608 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37370.log.gz 2024-04-20T10:31:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35835.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35835.log.gz 2024-04-20T10:31:28.608 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35415.log 2024-04-20T10:31:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40041.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40041.log.gz 2024-04-20T10:31:28.609 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39981.log 2024-04-20T10:31:28.610 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-20T10:31:28.610 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.b.log: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T10:31:28.610 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-20T10:31:28.610 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22452.log 2024-04-20T10:31:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26303.log 2024-04-20T10:31:28.611 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35415.log.gz 2024-04-20T10:31:28.611 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32520.log 2024-04-20T10:31:28.612 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39981.log.gz 2024-04-20T10:31:28.612 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22281.log 2024-04-20T10:31:28.612 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39855.log 2024-04-20T10:31:28.613 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26303.log.gz 2024-04-20T10:31:28.613 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43165.log 2024-04-20T10:31:28.614 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22452.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-20T10:31:28.614 INFO:teuthology.orchestra.run.smithi175.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.22452.log.gz 2024-04-20T10:31:28.615 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.32520.log.gz 2024-04-20T10:31:28.615 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43207.log 2024-04-20T10:31:28.615 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39855.log.gz 2024-04-20T10:31:28.616 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28021.log 2024-04-20T10:31:28.616 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43165.log.gz 2024-04-20T10:31:28.616 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22281.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-20T10:31:28.616 INFO:teuthology.orchestra.run.smithi175.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.22281.log.gz 2024-04-20T10:31:28.616 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-20T10:31:28.617 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43207.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43207.log.gz 2024-04-20T10:31:28.617 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39561.log 2024-04-20T10:31:28.618 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22412.log 2024-04-20T10:31:28.618 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40378.log 2024-04-20T10:31:28.619 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28021.log.gz 2024-04-20T10:31:28.619 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32478.log 2024-04-20T10:31:28.619 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39561.log.gz 2024-04-20T10:31:28.620 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39645.log 2024-04-20T10:31:28.620 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40378.log.gz 2024-04-20T10:31:28.621 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.6.log: /var/log/ceph/ceph-client.admin.22412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22372.log 2024-04-20T10:31:28.622 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30317.log 2024-04-20T10:31:28.622 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.32478.log.gz 2024-04-20T10:31:28.622 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43842.log 2024-04-20T10:31:28.623 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.39645.log.gz 2024-04-20T10:31:28.624 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37178.log 2024-04-20T10:31:28.624 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.30317.log.gz 2024-04-20T10:31:28.625 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22372.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.22412.log.gz 2024-04-20T10:31:28.625 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22492.log 2024-04-20T10:31:28.626 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28829.log 2024-04-20T10:31:28.626 INFO:teuthology.orchestra.run.smithi047.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43842.log.gz 2024-04-20T10:31:28.626 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28493.log 2024-04-20T10:31:28.626 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.37178.log.gz 2024-04-20T10:31:28.627 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35583.log 2024-04-20T10:31:28.627 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.28829.log.gz 2024-04-20T10:31:28.627 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22532.log 2024-04-20T10:31:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43123.log 2024-04-20T10:31:28.628 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28493.log.gz 2024-04-20T10:31:28.628 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41114.log 2024-04-20T10:31:28.629 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.35583.log.gz 2024-04-20T10:31:28.629 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36682.log 2024-04-20T10:31:28.630 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43123.log.gz 2024-04-20T10:31:28.630 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35709.log 2024-04-20T10:31:28.631 INFO:teuthology.orchestra.run.smithi175.stderr: 89.0%/var/log/ceph/ceph-client.admin.22492.log: /var/log/ceph/ceph-client.admin.22532.log: -- replaced with /var/log/ceph/ceph-client.admin.22372.log.gz 2024-04-20T10:31:28.631 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41114.log.gz 2024-04-20T10:31:28.631 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33639.log 2024-04-20T10:31:28.632 INFO:teuthology.orchestra.run.smithi175.stderr:gzip 89.1% -5 --verbose -- /var/log/ceph/ceph-client.admin.22323.log 2024-04-20T10:31:28.632 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.36682.log.gz 2024-04-20T10:31:28.632 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32604.log 2024-04-20T10:31:28.633 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35709.log.gz 2024-04-20T10:31:28.633 INFO:teuthology.orchestra.run.smithi175.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.22492.log.gz 2024-04-20T10:31:28.635 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-20T10:31:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32292.log 2024-04-20T10:31:28.635 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33639.log.gz 2024-04-20T10:31:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39687.log 2024-04-20T10:31:28.635 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.32604.log.gz 2024-04-20T10:31:28.635 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43926.log 2024-04-20T10:31:28.636 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32292.log.gz 2024-04-20T10:31:28.636 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36417.log 2024-04-20T10:31:28.637 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.d.log: -- replaced with /var/log/ceph/ceph-client.admin.22532.log.gz 2024-04-20T10:31:28.637 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39687.log.gz 2024-04-20T10:31:28.637 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36883.log 2024-04-20T10:31:28.637 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43926.log.gz 2024-04-20T10:31:28.638 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39729.log 2024-04-20T10:31:28.638 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36417.log.gz 2024-04-20T10:31:28.639 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.22323.log: gzip -5 --verbose 88.6% -- /var/log/ceph/ceph-mon.b.log 2024-04-20T10:31:28.640 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26625.log 2024-04-20T10:31:28.640 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.36883.log.gz 2024-04-20T10:31:28.640 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39729.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34563.log 2024-04-20T10:31:28.640 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39729.log.gz 2024-04-20T10:31:28.641 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34521.log 2024-04-20T10:31:28.641 INFO:teuthology.orchestra.run.smithi175.stderr: -- replaced with /var/log/ceph/ceph-client.admin.22323.log.gz 2024-04-20T10:31:28.642 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26625.log.gz 2024-04-20T10:31:28.642 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29049.log 2024-04-20T10:31:28.643 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34563.log.gz 2024-04-20T10:31:28.643 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30401.log 2024-04-20T10:31:28.643 INFO:teuthology.orchestra.run.smithi175.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-20T10:31:28.644 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34521.log.gz 2024-04-20T10:31:28.644 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40719.log 2024-04-20T10:31:28.644 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29049.log.gz 2024-04-20T10:31:28.645 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30401.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-20T10:31:28.645 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30401.log.gz 2024-04-20T10:31:28.646 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44596.log 2024-04-20T10:31:28.646 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40719.log.gz 2024-04-20T10:31:28.647 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34137.log 2024-04-20T10:31:28.648 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40336.log 2024-04-20T10:31:28.648 INFO:teuthology.orchestra.run.smithi047.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-20T10:31:28.649 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44596.log.gz 2024-04-20T10:31:28.649 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26967.log 2024-04-20T10:31:28.650 INFO:teuthology.orchestra.run.smithi047.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34137.log.gz 2024-04-20T10:31:28.654 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33024.log 2024-04-20T10:31:28.654 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.40336.log.gz 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42512.log 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.26967.log.gz 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35625.log 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33024.log.gz 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33759.log 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42512.log.gz 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32166.log 2024-04-20T10:31:28.656 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35625.log.gz 2024-04-20T10:31:28.657 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.b.log: 92.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-20T10:31:28.658 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30569.log 2024-04-20T10:31:28.658 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33759.log.gz 2024-04-20T10:31:28.658 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36249.log 2024-04-20T10:31:28.658 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32166.log.gz 2024-04-20T10:31:28.659 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30569.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-20T10:31:28.659 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30569.log.gz 2024-04-20T10:31:28.659 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30191.log 88.8% 2024-04-20T10:31:28.659 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36249.log.gz 2024-04-20T10:31:28.660 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29406.log 2024-04-20T10:31:28.662 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30191.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.30191.log.gz 2024-04-20T10:31:28.662 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43800.log 2024-04-20T10:31:28.663 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34893.log 2024-04-20T10:31:28.663 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29406.log.gz 2024-04-20T10:31:28.664 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32814.log 2024-04-20T10:31:28.665 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43800.log.gz 2024-04-20T10:31:28.665 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30611.log 2024-04-20T10:31:28.665 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34893.log.gz 2024-04-20T10:31:28.666 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34977.log 2024-04-20T10:31:28.666 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32814.log.gz 2024-04-20T10:31:28.667 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28289.log 2024-04-20T10:31:28.668 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30611.log.gz 2024-04-20T10:31:28.668 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42871.log 2024-04-20T10:31:28.668 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34977.log.gz 2024-04-20T10:31:28.669 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27177.log 2024-04-20T10:31:28.670 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.28289.log.gz 2024-04-20T10:31:28.670 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42871.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33471.log 2024-04-20T10:31:28.671 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42871.log.gz 2024-04-20T10:31:28.671 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30781.log 2024-04-20T10:31:28.672 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.27177.log.gz 2024-04-20T10:31:28.673 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32856.log 2024-04-20T10:31:28.673 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33471.log.gz 2024-04-20T10:31:28.674 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35187.log 2024-04-20T10:31:28.674 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.30781.log.gz 2024-04-20T10:31:28.675 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30233.log 2024-04-20T10:31:28.676 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32856.log.gz 2024-04-20T10:31:28.676 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35187.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35187.log.gz 2024-04-20T10:31:28.677 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-20T10:31:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27285.log 2024-04-20T10:31:28.678 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.audit.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.30233.log.gz 2024-04-20T10:31:28.679 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39366.log 2024-04-20T10:31:28.680 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28745.log 2024-04-20T10:31:28.680 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-20T10:31:28.680 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.27285.log.gz 2024-04-20T10:31:28.681 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36291.log 2024-04-20T10:31:28.682 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39366.log.gz 2024-04-20T10:31:28.682 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28745.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.28745.log.gz 2024-04-20T10:31:28.682 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28247.log 2024-04-20T10:31:28.683 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44346.log 2024-04-20T10:31:28.683 INFO:teuthology.orchestra.run.smithi047.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.36291.log.gz 2024-04-20T10:31:28.684 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43520.log 2024-04-20T10:31:28.684 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28247.log.gz 2024-04-20T10:31:28.685 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34647.log 2024-04-20T10:31:28.686 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43520.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44346.log.gz 2024-04-20T10:31:28.686 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43520.log.gz 2024-04-20T10:31:28.687 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29405.log 2024-04-20T10:31:28.688 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29392.log 2024-04-20T10:31:28.688 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.34647.log.gz 2024-04-20T10:31:28.689 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29405.log: gzip 88.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.29405.log.gz --verbose 2024-04-20T10:31:28.689 INFO:teuthology.orchestra.run.smithi047.stderr: -- /var/log/ceph/ceph-client.admin.38502.log 2024-04-20T10:31:28.690 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26925.log 2024-04-20T10:31:28.690 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.29392.log.gz 2024-04-20T10:31:28.691 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35373.log 2024-04-20T10:31:28.691 INFO:teuthology.orchestra.run.smithi175.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-20T10:31:28.692 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38502.log.gz 2024-04-20T10:31:28.692 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39813.log 2024-04-20T10:31:28.692 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26925.log.gz 2024-04-20T10:31:28.693 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27895.log 2024-04-20T10:31:28.694 INFO:teuthology.orchestra.run.smithi047.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35373.log.gz 2024-04-20T10:31:28.694 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28661.log 2024-04-20T10:31:28.695 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39813.log.gz 2024-04-20T10:31:28.695 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27093.log 2024-04-20T10:31:28.696 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27895.log.gz 2024-04-20T10:31:28.697 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38206.log 2024-04-20T10:31:28.697 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28661.log.gz 2024-04-20T10:31:28.697 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44388.log 2024-04-20T10:31:28.698 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27093.log.gz 2024-04-20T10:31:28.698 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29689.log 2024-04-20T10:31:28.699 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38206.log.gz 2024-04-20T10:31:28.700 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40168.log 2024-04-20T10:31:28.700 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44388.log.gz 2024-04-20T10:31:28.701 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33287.log 2024-04-20T10:31:28.701 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29689.log.gz 2024-04-20T10:31:28.701 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32082.log 2024-04-20T10:31:28.702 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40168.log.gz 2024-04-20T10:31:28.703 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31804.log 2024-04-20T10:31:28.703 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.33287.log.gz 2024-04-20T10:31:28.704 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38248.log 2024-04-20T10:31:28.704 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.32082.log.gz 2024-04-20T10:31:28.705 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.31804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32898.log 2024-04-20T10:31:28.705 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.31804.log.gz 2024-04-20T10:31:28.706 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42718.log 2024-04-20T10:31:28.706 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38248.log.gz 2024-04-20T10:31:28.707 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32250.log 2024-04-20T10:31:28.707 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.32898.log.gz 2024-04-20T10:31:28.708 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30527.log 2024-04-20T10:31:28.708 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42718.log.gz 2024-04-20T10:31:28.709 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29740.log 2024-04-20T10:31:28.709 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32250.log.gz 2024-04-20T10:31:28.710 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30149.log 2024-04-20T10:31:28.710 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30527.log.gz 2024-04-20T10:31:28.711 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29740.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-20T10:31:28.711 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29740.log.gz 2024-04-20T10:31:28.712 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30149.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33245.log 2024-04-20T10:31:28.712 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.30149.log.gz 2024-04-20T10:31:28.713 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32208.log 2024-04-20T10:31:28.714 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2024-04-20T10:31:28.715 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33245.log.gz 2024-04-20T10:31:28.716 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32208.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32208.log.gz 2024-04-20T10:31:28.716 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32688.log 2024-04-20T10:31:28.717 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39771.log 2024-04-20T10:31:28.717 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2024-04-20T10:31:28.718 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37938.log 2024-04-20T10:31:28.718 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32688.log.gz 2024-04-20T10:31:28.719 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41240.log 2024-04-20T10:31:28.720 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39771.log.gz 2024-04-20T10:31:28.721 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28451.log 2024-04-20T10:31:28.721 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37938.log.gz 2024-04-20T10:31:28.722 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28619.log 2024-04-20T10:31:28.722 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41240.log.gz 2024-04-20T10:31:28.723 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28451.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.28451.log.gz -5 2024-04-20T10:31:28.723 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42997.log 2024-04-20T10:31:28.724 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28535.log 2024-04-20T10:31:28.724 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28619.log.gz 2024-04-20T10:31:28.725 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39324.log 2024-04-20T10:31:28.725 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42997.log.gz 2024-04-20T10:31:28.726 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38693.log 2024-04-20T10:31:28.726 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.28535.log.gz 2024-04-20T10:31:28.727 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22394.log 2024-04-20T10:31:28.727 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.39324.log.gz 2024-04-20T10:31:28.728 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32730.log 2024-04-20T10:31:28.729 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38693.log.gz 2024-04-20T10:31:28.730 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.22394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33555.log 2024-04-20T10:31:28.730 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.22394.log.gz 2024-04-20T10:31:28.731 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42675.log 2024-04-20T10:31:28.731 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32730.log.gz 2024-04-20T10:31:28.732 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33555.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33555.log.gz 2024-04-20T10:31:28.732 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-20T10:31:28.733 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38989.log 2024-04-20T10:31:28.733 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42675.log.gz 2024-04-20T10:31:28.734 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29635.log 2024-04-20T10:31:28.736 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.38989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38122.log 2024-04-20T10:31:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29635.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.38989.log.gz 2024-04-20T10:31:28.737 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41384.log 2024-04-20T10:31:28.738 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29635.log.gz 2024-04-20T10:31:28.738 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28063.log 2024-04-20T10:31:28.739 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38122.log: /var/log/ceph/ceph-client.admin.41384.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38122.log.gz 2024-04-20T10:31:28.740 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30065.log 2024-04-20T10:31:28.740 INFO:teuthology.orchestra.run.smithi047.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.41384.log.gz 2024-04-20T10:31:28.741 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29397.log 2024-04-20T10:31:28.741 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.28063.log.gz 2024-04-20T10:31:28.742 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35499.log 2024-04-20T10:31:28.743 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.30065.log.gz 2024-04-20T10:31:28.743 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29397.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.29397.log.gz -5 2024-04-20T10:31:28.743 INFO:teuthology.orchestra.run.smithi047.stderr: --verbose -- /var/log/ceph/ceph-client.admin.40803.log 2024-04-20T10:31:28.744 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36813.log 2024-04-20T10:31:28.745 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35499.log: /var/log/ceph/ceph-client.admin.40803.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35499.log.gz 2024-04-20T10:31:28.746 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28373.log 2024-04-20T10:31:28.746 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40803.log.gz 2024-04-20T10:31:28.747 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39189.log 2024-04-20T10:31:28.747 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36813.log.gz 2024-04-20T10:31:28.748 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28373.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.28373.log.gz 2024-04-20T10:31:28.748 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44304.log 2024-04-20T10:31:28.748 INFO:teuthology.orchestra.run.smithi175.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-20T10:31:28.749 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31305.log 2024-04-20T10:31:28.750 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39189.log: /var/log/ceph/ceph-client.admin.44304.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39189.log.gz 2024-04-20T10:31:28.751 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30653.log 2024-04-20T10:31:28.751 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44304.log.gz 2024-04-20T10:31:28.752 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.31305.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-20T10:31:28.752 INFO:teuthology.orchestra.run.smithi047.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.31305.log.gz 2024-04-20T10:31:28.753 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30653.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-20T10:31:28.753 INFO:teuthology.orchestra.run.smithi047.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-20T10:31:28.754 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.1.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.30653.log.gz 2024-04-20T10:31:28.754 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30107.log 2024-04-20T10:31:28.755 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2024-04-20T10:31:28.757 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30107.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28787.log 2024-04-20T10:31:28.758 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.30107.log.gz 2024-04-20T10:31:28.758 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28577.log 2024-04-20T10:31:28.759 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2024-04-20T10:31:28.759 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34689.log 2024-04-20T10:31:28.760 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28787.log.gz 2024-04-20T10:31:28.761 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36501.log 2024-04-20T10:31:28.762 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.28577.log.gz 2024-04-20T10:31:28.762 INFO:teuthology.orchestra.run.smithi047.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-20T10:31:28.762 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40942.log 2024-04-20T10:31:28.762 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34689.log.gz 2024-04-20T10:31:28.763 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36501.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28205.log 2024-04-20T10:31:28.764 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.36501.log.gz 2024-04-20T10:31:28.765 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40942.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-20T10:31:28.765 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40942.log.gz 2024-04-20T10:31:28.766 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35145.log 2024-04-20T10:31:28.766 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28205.log.gz 2024-04-20T10:31:28.767 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40210.log 2024-04-20T10:31:28.768 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34379.log 2024-04-20T10:31:28.768 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.35145.log.gz 2024-04-20T10:31:28.769 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41030.log 2024-04-20T10:31:28.769 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40210.log.gz 2024-04-20T10:31:28.770 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35247.log 2024-04-20T10:31:28.771 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.34379.log.gz 2024-04-20T10:31:28.772 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26583.log 2024-04-20T10:31:28.772 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41030.log.gz 2024-04-20T10:31:28.773 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37052.log 2024-04-20T10:31:28.774 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35247.log.gz 2024-04-20T10:31:28.774 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36968.log 2024-04-20T10:31:28.775 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26583.log.gz 2024-04-20T10:31:28.775 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44514.log 2024-04-20T10:31:28.776 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37052.log.gz 2024-04-20T10:31:28.777 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33329.log 2024-04-20T10:31:28.777 INFO:teuthology.orchestra.run.smithi047.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36968.log.gz 2024-04-20T10:31:28.778 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35751.log 2024-04-20T10:31:28.779 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44514.log.gz 2024-04-20T10:31:28.779 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36724.log 2024-04-20T10:31:28.779 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.33329.log.gz 2024-04-20T10:31:28.781 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41282.log 2024-04-20T10:31:28.781 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35751.log.gz 2024-04-20T10:31:28.782 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36061.log 2024-04-20T10:31:28.782 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36724.log.gz 2024-04-20T10:31:28.783 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42098.log 2024-04-20T10:31:28.783 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41282.log.gz 2024-04-20T10:31:28.784 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29681.log 2024-04-20T10:31:28.785 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.36061.log.gz 2024-04-20T10:31:28.786 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41324.log 2024-04-20T10:31:28.786 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42098.log.gz 2024-04-20T10:31:28.787 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26417.log 2024-04-20T10:31:28.787 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29681.log.gz 2024-04-20T10:31:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41324.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41324.log.gz 2024-04-20T10:31:28.788 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28331.log 2024-04-20T10:31:28.789 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38637.log 2024-04-20T10:31:28.789 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26417.log.gz 2024-04-20T10:31:28.790 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42097.log 2024-04-20T10:31:28.791 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.28331.log.gz 2024-04-20T10:31:28.792 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34179.log 2024-04-20T10:31:28.792 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.38637.log.gz 2024-04-20T10:31:28.793 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34851.log 2024-04-20T10:31:28.793 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34179.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42097.log.gz 2024-04-20T10:31:28.794 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34179.log.gz 2024-04-20T10:31:28.794 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29045.log 2024-04-20T10:31:28.795 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.34851.log: /var/log/ceph/ceph-client.admin.40126.log 2024-04-20T10:31:28.796 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.34851.log.gz 2024-04-20T10:31:28.797 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36631.log 2024-04-20T10:31:28.797 INFO:teuthology.orchestra.run.smithi047.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.29045.log.gz 2024-04-20T10:31:28.798 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28871.log 2024-04-20T10:31:28.798 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40126.log.gz 2024-04-20T10:31:28.799 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36631.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.36631.log.gz 2024-04-20T10:31:28.801 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28871.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.28871.log.gz 2024-04-20T10:31:28.801 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29048.log 2024-04-20T10:31:28.802 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32376.log 2024-04-20T10:31:28.804 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35019.log 2024-04-20T10:31:28.804 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.29048.log.gz 2024-04-20T10:31:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32376.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.32376.log.gz 2024-04-20T10:31:28.805 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37136.log 2024-04-20T10:31:28.806 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38022.log 2024-04-20T10:31:28.806 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.35019.log.gz 2024-04-20T10:31:28.807 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40847.log 2024-04-20T10:31:28.807 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.37136.log.gz 2024-04-20T10:31:28.808 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42955.log 2024-04-20T10:31:28.808 INFO:teuthology.orchestra.run.smithi047.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.38022.log.gz 2024-04-20T10:31:28.809 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43351.log 2024-04-20T10:31:28.810 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40847.log.gz 2024-04-20T10:31:28.811 INFO:teuthology.orchestra.run.smithi175.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-20T10:31:28.811 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33429.log 2024-04-20T10:31:28.811 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42955.log.gz 2024-04-20T10:31:28.812 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37854.log 2024-04-20T10:31:28.812 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43351.log.gz 2024-04-20T10:31:28.813 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33429.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.33429.log.gz 2024-04-20T10:31:28.814 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.27135.log 2024-04-20T10:31:28.814 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29391.log 2024-04-20T10:31:28.815 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.37854.log.gz 2024-04-20T10:31:28.816 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27135.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.43716.log 2024-04-20T10:31:28.816 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27135.log.gz 2024-04-20T10:31:28.817 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27937.log 2024-04-20T10:31:28.817 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29391.log.gz 2024-04-20T10:31:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33161.log 2024-04-20T10:31:28.819 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43716.log.gz 2024-04-20T10:31:28.819 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27937.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.35061.log 2024-04-20T10:31:28.819 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.27937.log.gz 2024-04-20T10:31:28.820 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26839.log 2024-04-20T10:31:28.821 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33161.log.gz 2024-04-20T10:31:28.822 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40988.log 2024-04-20T10:31:28.822 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35061.log.gz 2024-04-20T10:31:28.823 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38459.log 2024-04-20T10:31:28.823 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.26839.log.gz 2024-04-20T10:31:28.824 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40988.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-20T10:31:28.825 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40988.log.gz 2024-04-20T10:31:28.825 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43478.log 2024-04-20T10:31:28.826 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.38459.log.gz 2024-04-20T10:31:28.826 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26498.log 2024-04-20T10:31:28.828 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44472.log 2024-04-20T10:31:28.828 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43478.log.gz 2024-04-20T10:31:28.829 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40891.log 2024-04-20T10:31:28.830 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.26498.log.gz 2024-04-20T10:31:28.831 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28105.log 2024-04-20T10:31:28.831 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44472.log.gz 2024-04-20T10:31:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40891.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40891.log.gz 2024-04-20T10:31:28.832 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43039.log 2024-04-20T10:31:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.28105.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40252.log 2024-04-20T10:31:28.834 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.28105.log.gz 2024-04-20T10:31:28.834 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43039.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43039.log.gz 2024-04-20T10:31:28.835 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34095.log 2024-04-20T10:31:28.836 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36543.log 2024-04-20T10:31:28.836 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40252.log.gz 2024-04-20T10:31:28.837 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43081.log 2024-04-20T10:31:28.838 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34095.log.gz 2024-04-20T10:31:28.839 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36543.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.36543.log.gz 2024-04-20T10:31:28.840 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43081.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43081.log.gz 2024-04-20T10:31:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35961.log 2024-04-20T10:31:28.841 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29739.log 2024-04-20T10:31:28.843 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32982.log 2024-04-20T10:31:28.843 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35961.log.gz 2024-04-20T10:31:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29739.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.29739.log.gz 2024-04-20T10:31:28.844 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44010.log 2024-04-20T10:31:28.846 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34421.log 2024-04-20T10:31:28.846 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.32982.log.gz 2024-04-20T10:31:28.847 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33843.log 2024-04-20T10:31:28.847 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44010.log.gz 2024-04-20T10:31:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34421.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34421.log.gz 2024-04-20T10:31:28.848 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34053.log 2024-04-20T10:31:28.850 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40761.log 2024-04-20T10:31:28.850 INFO:teuthology.orchestra.run.smithi047.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33843.log.gz 2024-04-20T10:31:28.851 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41198.log 2024-04-20T10:31:28.852 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.34053.log.gz 2024-04-20T10:31:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40761.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.40761.log.gz 2024-04-20T10:31:28.853 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37094.log 2024-04-20T10:31:28.854 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40553.log 2024-04-20T10:31:28.854 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.41198.log.gz/var/log/ceph/ceph-client.admin.37094.log: 2024-04-20T10:31:28.859 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27219.log 89.4% -- replaced with /var/log/ceph/ceph-client.admin.37094.log.gz 2024-04-20T10:31:28.860 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T10:31:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40553.log: 88.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26753.log 2024-04-20T10:31:28.860 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40553.log.gz 2024-04-20T10:31:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26222.log 2024-04-20T10:31:28.860 INFO:teuthology.orchestra.run.smithi047.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.27219.log.gz 2024-04-20T10:31:28.860 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30485.log 2024-04-20T10:31:28.860 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.26753.log.gz 2024-04-20T10:31:28.861 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26222.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.26222.log.gz 2024-04-20T10:31:28.862 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5/var/log/ceph/ceph-client.admin.30485.log: --verbose -- /var/log/ceph/ceph-client.admin.42340.log 2024-04-20T10:31:28.862 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.30485.log.gz 2024-04-20T10:31:28.863 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.15608.log 2024-04-20T10:31:28.865 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42340.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40633.log 2024-04-20T10:31:28.865 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42340.log.gz 2024-04-20T10:31:28.866 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph.tmp-client.admin.15608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29052.log 2024-04-20T10:31:28.866 INFO:teuthology.orchestra.run.smithi047.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.15608.log.gz 2024-04-20T10:31:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40633.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40633.log.gz 2024-04-20T10:31:28.867 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22494.log 2024-04-20T10:31:28.868 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43758.log 2024-04-20T10:31:28.869 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29052.log.gz 2024-04-20T10:31:28.870 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.22494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44052.log 2024-04-20T10:31:28.870 INFO:teuthology.orchestra.run.smithi047.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.22494.log.gz 2024-04-20T10:31:28.871 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43758.log: gzip 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43758.log.gz 2024-04-20T10:31:28.871 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.29042.log 2024-04-20T10:31:28.872 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32562.log 2024-04-20T10:31:28.872 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44052.log.gz 2024-04-20T10:31:28.874 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38080.log 2024-04-20T10:31:28.874 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29042.log.gz 2024-04-20T10:31:28.875 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32562.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32562.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43674.log 2024-04-20T10:31:28.875 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T10:31:28.876 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38080.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38080.log.gz 2024-04-20T10:31:28.877 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33597.log 2024-04-20T10:31:28.878 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35541.log 2024-04-20T10:31:28.878 INFO:teuthology.orchestra.run.smithi047.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43674.log.gz 2024-04-20T10:31:28.879 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33597.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33597.log.gz 2024-04-20T10:31:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32940.log 2024-04-20T10:31:28.880 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35541.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35541.log.gz 2024-04-20T10:31:28.881 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27051.log 2024-04-20T10:31:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40473.log 2024-04-20T10:31:28.883 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.32940.log.gz 2024-04-20T10:31:28.883 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27051.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.27051.log.gz 2024-04-20T10:31:28.884 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33927.log 2024-04-20T10:31:28.885 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36770.log 2024-04-20T10:31:28.885 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40473.log.gz 2024-04-20T10:31:28.887 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37648.log 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33927.log.gz 2024-04-20T10:31:28.887 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T10:31:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36770.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.36770.log.gz 2024-04-20T10:31:28.888 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37220.log 2024-04-20T10:31:28.890 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.37648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41156.log 2024-04-20T10:31:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37220.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.37220.log.gz 2024-04-20T10:31:28.891 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41156.log: 92.1% -- replaced with /var/log/ceph/ceph-client.0.37648.log.gz 2024-04-20T10:31:28.892 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41156.log.gz 2024-04-20T10:31:28.893 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41996.log 2024-04-20T10:31:28.894 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30275.log 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40675.log 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41996.log.gz 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T10:31:28.896 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30275.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30275.log.gz 2024-04-20T10:31:28.897 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34463.log 2024-04-20T10:31:28.898 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29051.log 2024-04-20T10:31:28.898 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40675.log.gz 2024-04-20T10:31:28.900 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26797.log 2024-04-20T10:31:28.900 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.34463.log.gz 2024-04-20T10:31:28.901 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34337.log 2024-04-20T10:31:28.901 INFO:teuthology.orchestra.run.smithi047.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.29051.log.gz 2024-04-20T10:31:28.903 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26797.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.26797.log.gz 2024-04-20T10:31:28.904 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.34337.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36207.log 2024-04-20T10:31:28.904 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34337.log.gz 2024-04-20T10:31:28.905 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43968.log 2024-04-20T10:31:28.906 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36207.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36459.log 2024-04-20T10:31:28.906 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36207.log.gz 2024-04-20T10:31:28.908 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30443.log 2024-04-20T10:31:28.908 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43968.log.gz 2024-04-20T10:31:28.909 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36459.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.36459.log.gz 2024-04-20T10:31:28.909 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29036.log 2024-04-20T10:31:28.910 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30443.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.30443.log.gz 2024-04-20T10:31:28.911 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33066.log 2024-04-20T10:31:28.912 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42277.log 2024-04-20T10:31:28.912 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29036.log.gz 2024-04-20T10:31:28.913 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32646.log 2024-04-20T10:31:28.913 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33066.log.gz 2024-04-20T10:31:28.915 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42277.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.2.31700.log 2024-04-20T10:31:28.915 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42277.log.gz 2024-04-20T10:31:28.916 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34011.log 2024-04-20T10:31:28.916 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.32646.log.gz 2024-04-20T10:31:28.919 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.2.31700.log: /var/log/ceph/ceph-client.admin.34011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27979.log 2024-04-20T10:31:28.919 INFO:teuthology.orchestra.run.smithi047.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.2.31700.log.gz 2024-04-20T10:31:28.919 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.34011.log.gz 2024-04-20T10:31:28.919 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33717.log 2024-04-20T10:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr:gzip/var/log/ceph/ceph-client.admin.27979.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.35331.log 2024-04-20T10:31:28.921 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.27979.log.gz 2024-04-20T10:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33717.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33717.log.gz 2024-04-20T10:31:28.922 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41072.log 2024-04-20T10:31:28.924 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34253.log 2024-04-20T10:31:28.924 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35331.log.gz 2024-04-20T10:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38388.log 2024-04-20T10:31:28.925 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41072.log.gz 2024-04-20T10:31:28.926 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34253.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34253.log.gz 2024-04-20T10:31:28.927 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35667.log 2024-04-20T10:31:28.928 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38388.log: 89.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36925.log 2024-04-20T10:31:28.928 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38388.log.gz 2024-04-20T10:31:28.930 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32124.log 2024-04-20T10:31:28.930 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.35667.log.gz 2024-04-20T10:31:28.931 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27722.log 2024-04-20T10:31:28.931 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.36925.log.gz 2024-04-20T10:31:28.932 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29733.log 2024-04-20T10:31:28.932 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.32124.log.gz 2024-04-20T10:31:28.933 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.27722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42095.log 2024-04-20T10:31:28.935 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29733.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.29733.log.gz 2024-04-20T10:31:28.936 INFO:teuthology.orchestra.run.smithi047.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.27722.log.gz 2024-04-20T10:31:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38164.log 2024-04-20T10:31:28.936 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42095.log: 88.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42913.log 2024-04-20T10:31:28.936 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42095.log.gz 2024-04-20T10:31:28.938 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43291.log 2024-04-20T10:31:28.938 INFO:teuthology.orchestra.run.smithi047.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38164.log.gz 2024-04-20T10:31:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42913.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.42913.log.gz 2024-04-20T10:31:28.939 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33371.log 2024-04-20T10:31:28.941 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43394.log 2024-04-20T10:31:28.941 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43291.log.gz 2024-04-20T10:31:28.942 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42446.log 2024-04-20T10:31:28.942 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33371.log.gz 2024-04-20T10:31:28.943 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43394.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43394.log.gz 2024-04-20T10:31:28.943 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29404.log 2024-04-20T10:31:28.944 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38326.log 2024-04-20T10:31:28.945 INFO:teuthology.orchestra.run.smithi047.stderr: 91.0%/var/log/ceph/ceph-client.admin.29404.log: -- replaced with /var/log/ceph/ceph-client.admin.42446.log.gz 2024-04-20T10:31:28.946 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 88.2% --verbose -- /var/log/ceph/ceph-client.admin.37980.log 2024-04-20T10:31:28.946 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29404.log.gz 2024-04-20T10:31:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38326.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.38326.log.gz 2024-04-20T10:31:28.947 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34731.log 2024-04-20T10:31:28.948 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43884.log 2024-04-20T10:31:28.949 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.37980.log.gz 2024-04-20T10:31:28.950 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34731.log: gzip -5 --verbose -- 88.7% -- replaced with /var/log/ceph/ceph-client.admin.34731.log.gz 2024-04-20T10:31:28.950 INFO:teuthology.orchestra.run.smithi047.stderr: /var/log/ceph/ceph-client.admin.29407.log 2024-04-20T10:31:28.951 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43884.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43884.log.gz 2024-04-20T10:31:28.952 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36165.log 2024-04-20T10:31:28.953 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37262.log 2024-04-20T10:31:28.953 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.29407.log.gz 2024-04-20T10:31:28.954 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2024-04-20T10:31:28.954 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36165.log.gz 2024-04-20T10:31:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37262.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.37262.log.gz 2024-04-20T10:31:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33885.log 2024-04-20T10:31:28.956 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42386.log 2024-04-20T10:31:28.958 INFO:teuthology.orchestra.run.smithi047.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2024-04-20T10:31:28.958 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39408.log 2024-04-20T10:31:28.958 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.33885.log.gz 2024-04-20T10:31:28.959 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42386.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42386.log.gz 2024-04-20T10:31:28.960 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36333.log 2024-04-20T10:31:28.961 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32434.log 2024-04-20T10:31:28.961 INFO:teuthology.orchestra.run.smithi047.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.39408.log.gz 2024-04-20T10:31:28.962 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31846.log 2024-04-20T10:31:28.963 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.36333.log.gz 2024-04-20T10:31:28.964 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31998.log 2024-04-20T10:31:28.964 INFO:teuthology.orchestra.run.smithi047.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.32434.log.gz 2024-04-20T10:31:28.965 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.31846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35877.log 2024-04-20T10:31:28.965 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31846.log.gz 2024-04-20T10:31:28.967 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.31998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29743.log 2024-04-20T10:31:28.967 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.31998.log.gz 2024-04-20T10:31:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35877.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.35877.log.gz 2024-04-20T10:31:28.968 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34809.log 2024-04-20T10:31:28.970 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5/var/log/ceph/ceph-client.admin.29743.log: --verbose -- /var/log/ceph/ceph-client.admin.39450.log 2024-04-20T10:31:28.970 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29743.log.gz/var/log/ceph/ceph-client.admin.34809.log: 2024-04-20T10:31:28.970 INFO:teuthology.orchestra.run.smithi047.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.34809.log.gz 2024-04-20T10:31:28.971 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38548.log 2024-04-20T10:31:28.972 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34935.log 2024-04-20T10:31:28.972 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39450.log.gz 2024-04-20T10:31:28.973 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.38548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32334.log 2024-04-20T10:31:28.974 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38548.log.gz 2024-04-20T10:31:28.974 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34935.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.34935.log.gz 2024-04-20T10:31:28.975 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33801.log 2024-04-20T10:31:28.976 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39897.log 2024-04-20T10:31:28.977 INFO:teuthology.orchestra.run.smithi047.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.32334.log.gz 2024-04-20T10:31:28.978 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35793.log 2024-04-20T10:31:28.978 INFO:teuthology.orchestra.run.smithi047.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.33801.log.gz 2024-04-20T10:31:28.979 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39897.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37304.log 2024-04-20T10:31:28.979 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39897.log.gz 2024-04-20T10:31:28.980 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36587.log 2024-04-20T10:31:28.981 INFO:teuthology.orchestra.run.smithi047.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35793.log.gz 2024-04-20T10:31:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44220.log 2024-04-20T10:31:28.982 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36587.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.37304.log.gz 2024-04-20T10:31:28.983 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43249.log 2024-04-20T10:31:28.983 INFO:teuthology.orchestra.run.smithi047.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.36587.log.gz 2024-04-20T10:31:28.985 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35289.log 2024-04-20T10:31:28.985 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44220.log.gz 2024-04-20T10:31:28.986 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29396.log 2024-04-20T10:31:28.986 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43249.log.gz 2024-04-20T10:31:28.987 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35289.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35289.log.gz 2024-04-20T10:31:28.987 INFO:teuthology.orchestra.run.smithi047.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33203.log 2024-04-20T10:31:28.989 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37010.log 2024-04-20T10:31:28.989 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.29396.log.gz 2024-04-20T10:31:28.990 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33203.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26540.log 2024-04-20T10:31:28.990 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.33203.log.gz 2024-04-20T10:31:28.992 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37010.log: 88.6%gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-20T10:31:28.992 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37010.log.gz 2024-04-20T10:31:28.993 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37896.log 2024-04-20T10:31:28.993 INFO:teuthology.orchestra.run.smithi047.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.26540.log.gz 2024-04-20T10:31:28.994 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40084.log 2024-04-20T10:31:28.996 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.37896.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.37896.log.gz 2024-04-20T10:31:28.997 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28163.log 2024-04-20T10:31:28.997 INFO:teuthology.orchestra.run.smithi047.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40084.log.gz 2024-04-20T10:31:28.998 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.31194.log 2024-04-20T10:31:29.000 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28163.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.28163.log.gz 2024-04-20T10:31:29.001 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41450.log/var/log/ceph/ceph-client.0.31194.log: 2024-04-20T10:31:29.002 INFO:teuthology.orchestra.run.smithi047.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.31194.log.gz 2024-04-20T10:31:29.003 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34295.log/var/log/ceph/ceph-client.admin.41450.log: 2024-04-20T10:31:29.004 INFO:teuthology.orchestra.run.smithi047.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.41450.log.gz 2024-04-20T10:31:29.005 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38590.log 2024-04-20T10:31:29.007 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.34295.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34295.log.gz 2024-04-20T10:31:29.007 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30023.log/var/log/ceph/ceph-client.admin.38590.log: 2024-04-20T10:31:29.009 INFO:teuthology.orchestra.run.smithi047.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.38590.log.gz 2024-04-20T10:31:29.009 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32040.log 2024-04-20T10:31:29.010 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30023.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.30023.log.gz 2024-04-20T10:31:29.011 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26709.log 2024-04-20T10:31:29.013 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.32040.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.32040.log.gz 2024-04-20T10:31:29.014 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29939.log 2024-04-20T10:31:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26709.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.26709.log.gz 2024-04-20T10:31:29.015 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34605.log 2024-04-20T10:31:29.017 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29939.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.29939.log.gz 2024-04-20T10:31:29.018 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5/var/log/ceph/ceph-client.admin.34605.log: --verbose -- /var/log/ceph/ceph-client.admin.43436.log 2024-04-20T10:31:29.018 INFO:teuthology.orchestra.run.smithi047.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.34605.log.gz 2024-04-20T10:31:29.019 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31956.log 2024-04-20T10:31:29.021 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.43436.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43436.log.gz 2024-04-20T10:31:29.022 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.31956.log: /var/log/ceph/ceph-client.admin.39282.log 2024-04-20T10:31:29.023 INFO:teuthology.orchestra.run.smithi047.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.31956.log.gz 2024-04-20T10:31:29.023 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42633.log 2024-04-20T10:31:29.025 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39282.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39282.log.gz 2024-04-20T10:31:29.026 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39939.log 2024-04-20T10:31:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42633.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42633.log.gz 2024-04-20T10:31:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41728.log 2024-04-20T10:31:29.028 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39939.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39939.log.gz 2024-04-20T10:31:29.029 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33969.log 2024-04-20T10:31:29.032 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.0.41728.log: /var/log/ceph/ceph-client.admin.33969.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33969.log.gz 2024-04-20T10:31:29.033 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30359.log 2024-04-20T10:31:29.035 INFO:teuthology.orchestra.run.smithi047.stderr: 92.3% -- replaced with /var/log/ceph/ceph-client.0.41728.log.gz 2024-04-20T10:31:29.036 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.30359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42591.log 2024-04-20T10:31:29.037 INFO:teuthology.orchestra.run.smithi047.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.30359.log.gz 2024-04-20T10:31:29.038 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39603.log 2024-04-20T10:31:29.039 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42591.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42591.log.gz 2024-04-20T10:31:29.040 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40294.log 2024-04-20T10:31:29.041 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39603.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39603.log.gz 2024-04-20T10:31:29.042 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22126.log 2024-04-20T10:31:29.043 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.40294.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40294.log.gz 2024-04-20T10:31:29.044 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39146.log 2024-04-20T10:31:29.045 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.22126.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.22126.log.gz 2024-04-20T10:31:29.046 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44430.log 2024-04-20T10:31:29.047 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.39146.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39146.log.gz 2024-04-20T10:31:29.048 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44554.log 2024-04-20T10:31:29.050 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44430.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44430.log.gz 2024-04-20T10:31:29.050 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26667.log 2024-04-20T10:31:29.051 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44554.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44554.log.gz 2024-04-20T10:31:29.052 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36375.log 2024-04-20T10:31:29.053 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26667.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.26667.log.gz 2024-04-20T10:31:29.054 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29742.log 2024-04-20T10:31:29.055 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.36375.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.36375.log.gz 2024-04-20T10:31:29.056 INFO:teuthology.orchestra.run.smithi047.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.a.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44262.log 2024-04-20T10:31:29.056 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T10:31:29.057 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29742.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.29742.log.gz 2024-04-20T10:31:29.058 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35457.log 2024-04-20T10:31:29.058 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44262.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.44262.log.gz 2024-04-20T10:31:29.059 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28703.log 2024-04-20T10:31:29.062 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44094.log 2024-04-20T10:31:29.062 INFO:teuthology.orchestra.run.smithi047.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35457.log.gz 2024-04-20T10:31:29.063 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.28703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29981.log 2024-04-20T10:31:29.063 INFO:teuthology.orchestra.run.smithi047.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.28703.log.gz 2024-04-20T10:31:29.063 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29053.log 2024-04-20T10:31:29.063 INFO:teuthology.orchestra.run.smithi047.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44094.log.gz 2024-04-20T10:31:29.065 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41917.log 2024-04-20T10:31:29.065 INFO:teuthology.orchestra.run.smithi047.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.29981.log.gz 2024-04-20T10:31:29.066 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.29053.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.29053.log.gz 2024-04-20T10:31:29.066 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44136.log 2024-04-20T10:31:29.067 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.41917.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.41917.log.gz 2024-04-20T10:31:29.068 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42760.log 2024-04-20T10:31:29.069 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.44136.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33513.log 2024-04-20T10:31:29.069 INFO:teuthology.orchestra.run.smithi047.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44136.log.gz 2024-04-20T10:31:29.070 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.42760.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.42760.log.gz 2024-04-20T10:31:29.071 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.33513.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.33513.log.gz 2024-04-20T10:31:29.072 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35919.log 2024-04-20T10:31:29.073 INFO:teuthology.orchestra.run.smithi047.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26881.log 2024-04-20T10:31:29.074 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.35919.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.35919.log.gz 2024-04-20T10:31:29.075 INFO:teuthology.orchestra.run.smithi047.stderr:/var/log/ceph/ceph-client.admin.26881.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.26881.log.gz 2024-04-20T10:31:29.286 INFO:teuthology.orchestra.run.smithi047.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-20T10:31:29.420 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-20T10:31:29.423 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-20T10:31:29.444 INFO:teuthology.orchestra.run.smithi175.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-20T10:31:29.453 INFO:teuthology.orchestra.run.smithi175.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-20T10:31:29.456 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-04-20T10:31:29.456 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m0.886s 2024-04-20T10:31:29.456 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m3.455s 2024-04-20T10:31:29.457 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.388s 2024-04-20T10:31:29.572 INFO:teuthology.orchestra.run.smithi047.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-20T10:31:29.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-04-20T10:31:29.586 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-04-20T10:31:29.672 INFO:teuthology.orchestra.run.smithi047.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-20T10:31:29.699 INFO:teuthology.orchestra.run.smithi047.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-20T10:31:29.711 INFO:teuthology.orchestra.run.smithi047.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-20T10:31:29.716 INFO:teuthology.orchestra.run.smithi047.stderr: 2024-04-20T10:31:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:real 0m1.149s 2024-04-20T10:31:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:user 0m4.153s 2024-04-20T10:31:29.716 INFO:teuthology.orchestra.run.smithi047.stderr:sys 0m1.312s 2024-04-20T10:31:29.717 INFO:tasks.ceph:Archiving logs... 2024-04-20T10:31:29.717 DEBUG:teuthology.misc:Transferring archived files from smithi047:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/remote/smithi047/log 2024-04-20T10:31:29.718 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T10:31:30.546 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/remote/smithi175/log 2024-04-20T10:31:30.547 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-20T10:31:31.107 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-20T10:31:31.116 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-20T10:31:31.116 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T10:31:31.124 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-20T10:31:31.271 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-20T10:31:31.280 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-20T10:31:31.280 DEBUG:teuthology.orchestra.run.smithi047:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:31:31.288 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-20T10:31:31.337 INFO:teuthology.orchestra.run.smithi047.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:31:31.344 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-04-20T10:31:31.347 INFO:teuthology.orchestra.run.smithi047.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:31:31.347 INFO:teuthology.orchestra.run.smithi047.stdout:=============================================================================== 2024-04-20T10:31:31.348 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +364us[ +364us] +/- 89ms 2024-04-20T10:31:31.348 INFO:teuthology.orchestra.run.smithi047.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 +61us[ +48us] +/- 79ms 2024-04-20T10:31:31.348 INFO:teuthology.orchestra.run.smithi047.stdout:^* hv03.front.sepia.ceph.com 3 6 377 36 -262us[ -275us] +/- 61ms 2024-04-20T10:31:31.348 INFO:teuthology.orchestra.run.smithi047.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:31:31.354 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-20T10:31:31.354 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-04-20T10:31:31.354 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 35 +380us[ +380us] +/- 89ms 2024-04-20T10:31:31.354 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 36 +66us[ +66us] +/- 79ms 2024-04-20T10:31:31.355 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv03.front.sepia.ceph.com 3 6 377 36 -239us[ -254us] +/- 61ms 2024-04-20T10:31:31.355 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-20T10:31:31.356 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-20T10:31:31.366 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-20T10:31:31.366 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-20T10:31:31.375 DEBUG:teuthology.orchestra.run.smithi047:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T10:31:31.563 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-20T10:31:31.724 DEBUG:teuthology.orchestra.run.smithi047:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:31:31.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:31:31.815 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-20T10:31:31.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:31:31.906 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-20T10:31:31.915 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-20T10:31:31.924 INFO:teuthology.task.internal:Duration was 575.100314 seconds 2024-04-20T10:31:31.924 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-20T10:31:31.932 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-20T10:31:31.933 DEBUG:teuthology.orchestra.run.smithi047:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T10:31:31.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-20T10:31:32.078 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:31:32.091 INFO:teuthology.orchestra.run.smithi047.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-20T10:31:32.469 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-20T10:31:32.470 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi047.front.sepia.ceph.com 2024-04-20T10:31:32.470 DEBUG:teuthology.orchestra.run.smithi047:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T10:31:32.533 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-04-20T10:31:32.533 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-20T10:31:32.609 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-20T10:31:32.609 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T10:31:32.617 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-20T10:31:32.704 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-20T10:31:32.704 DEBUG:teuthology.orchestra.run.smithi047:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T10:31:32.710 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-20T10:31:32.857 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-20T10:31:32.867 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-20T10:31:32.867 DEBUG:teuthology.orchestra.run.smithi047:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T10:31:32.960 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-20T10:31:33.046 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-20T10:31:33.056 DEBUG:teuthology.orchestra.run.smithi047:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:31:33.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:31:33.157 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-04-20T10:31:33.160 INFO:teuthology.orchestra.run.smithi047.stdout:kernel.core_pattern = core 2024-04-20T10:31:33.208 DEBUG:teuthology.orchestra.run.smithi047:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:31:33.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:31:33.259 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-20T10:31:33.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-20T10:31:33.314 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-20T10:31:33.324 INFO:teuthology.task.internal:Transferring archived files... 2024-04-20T10:31:33.325 DEBUG:teuthology.misc:Transferring archived files from smithi047:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/remote/smithi047 2024-04-20T10:31:33.325 DEBUG:teuthology.orchestra.run.smithi047:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T10:31:33.420 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-04-20_09:50:15-fs-wip-rishabh-testing-20240416.193735-testing-default-smithi/7664463/remote/smithi175 2024-04-20T10:31:33.421 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-20T10:31:33.627 INFO:teuthology.task.internal:Removing archive directory... 2024-04-20T10:31:33.627 DEBUG:teuthology.orchestra.run.smithi047:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T10:31:33.635 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-20T10:31:33.693 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-20T10:31:33.749 INFO:teuthology.task.internal:Not uploading archives. 2024-04-20T10:31:33.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-20T10:31:33.760 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-20T10:31:33.760 DEBUG:teuthology.orchestra.run.smithi047:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T10:31:33.768 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-20T10:31:33.821 INFO:teuthology.orchestra.run.smithi047.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 10:31 /home/ubuntu/cephtest 2024-04-20T10:31:33.824 INFO:teuthology.orchestra.run.smithi175.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 20 10:31 /home/ubuntu/cephtest 2024-04-20T10:31:33.827 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-20T10:31:33.838 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-20T10:31:33.860 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate 3-modules} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v2 tasks/auto-repair} duration: 575.1003139019012 flavor: default mon.a-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 mon.b-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 owner: scheduled_rishabh@teuthology success: true 2024-04-20T10:31:33.860 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-20T10:31:33.935 INFO:teuthology.run:pass